#Mllm
Showing 48 of 48 repositories tagged #mllm, ranked by stars
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
Agent S: an open agentic framework that uses computers like a human
Mobile-Agent: The Powerful GUI Agent Family
From Chain-of-Thought prompting to OpenAI o1 and DeepSeek-R1 🍓
Code and models for ICML 2024 paper, NExT-GPT: Any-to-Any Multimodal Large Language Model
Eagle: Frontier Vision-Language Models with Data-Centric Strategies
InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions
Cambrian-1 is a family of multimodal LLMs with a vision-centric design.
🚀🚀🚀 A collection of some awesome public YOLO object detection series projects and the related object detection datasets.
Official Repo For Pixel-LLM Codebase: Sa2VA (Arxiv-25), SAMTok (CVPR-26), VRT, SaSaSa2VA (1-st solution for LSVOS)
OpenAI and Anthropic compatible server for Apple Silicon. Run LLMs and vision-language models (Llama, Qwen-VL, LLaVA) with continuous batching, MCP tool calling, and multimodal support. Native MLX backend, 400+ tok/s. Works with Claude Code.
Fully Open Framework for Democratized Multimodal Training
A family of lightweight multimodal models.
OpenEMMA, a permissively licensed open source "reproduction" of Waymo’s EMMA model.
[NeurIPS 2025] 4KAgent: Agentic Any Image to 4K Super-Resolution. An intelligent computer vision agent that can magically restore any image to perfect-4K!
🚀🚀🚀A collection of some awesome public projects about Large Language Model(LLM), Vision Language Model(VLM), Vision Language Action(VLA), AI Generated Content(AIGC), the related Datasets and Applications.
✨✨Woodpecker: Hallucination Correction for Multimodal Large Language Models
[ECCV2024] Grounded Multimodal Large Language Model with Localized Visual Tokenization
Custom ComfyUI nodes for Vision Language Models, Large Language Models, Image to Music, Text to Music, Consistent and Random Creative Prompt Generation
[ICLR2026] Official implementation for "JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigation"
🔥🔥🔥 A curated list of papers on LLMs-based multimodal generation (image, video, 3D and audio).
A curated collection of resources, tools, and frameworks for developing GUI Agents.
This project is the official implementation of 'LLMGA: Multimodal Large Language Model based Generation Assistant', ECCV2024 Oral
[Fully open] [Encoder-free MLLM] Vision as LoRA
Awesome_Multimodel is a curated GitHub repository that provides a comprehensive collection of resources for Multimodal Large Language Models (MLLM). It covers datasets, tuning techniques, in-context learning, visual reasoning, foundational models, and more. Stay updated with the latest advancement.
EVE Series: Encoder-Free Vision-Language Models from BAAI
🏭 Mega Scale Multimodal DataPipeline for SOTA Foundation Models
AUITestAgent is the first automatic, natural language-driven GUI testing tool for mobile apps, capable of fully automating the entire process of GUI interaction and function verification.
A framework for standardizing evaluations of large foundation models, beyond single-score reporting and rankings.
[CVPR 2026 Highlight] PersonaVLM: Long-Term Personalized Multimodal LLMs
[ICLR'26] Grasp Any Region: Towards Precise, Contextual Pixel Understanding for Multimodal LLMs
mPLUG-HalOwl: Multimodal Hallucination Evaluation and Mitigating
[ICLR'26] Traceable Evidence Enhanced Visual Grounded Reasoning: Evaluation and Methodology
Official repository of the paper "Generalist Virtual Agents: A Survey on Autonomous Agents Across Digital Platforms"
本项目以应用为主出发，结合了从基础的机器学习、深度学习到目标检测以及目前最新的大模型，采用目前成熟的 第三方库、开源预训练模型以及相关论文的最新技术，目的是记录学习的过程同时也进行分享以供更多人可以直接进行使用。
Latest Advances on (RL based) Multimodal Reasoning and Generation in Multimodal LLMs
A collection of visual instruction tuning datasets.
Official Repo For PerceptionDLM Codebase
【AIGC 实战入门笔记 —— AIGC 摩天大楼】分享 大语言模型（LLMs），大模型高效微调（SFT）,检索增强生成（RAG），智能体（Agent），PPT自动生成, 角色扮演，文生图（Stable Diffusion） ，图像文字识别（OCR），语音识别（ASR），语音合成（TTS），人像分割（SA），多模态（VLM），Ai 换脸(Face Swapping), 文生视频(VD)，图生视频（SVD），Ai 动作迁移，Ai 虚拟试衣，数字人，全模态理解（Omni），Ai音乐生成 干货学习 等 实战与经验。
[ICCV'25] Ross3D: Reconstructive Visual Instruction Tuning with 3D-Awareness
[NeurIPS 2025] Deep Memory Backtracking for Long Video Understanding
Evaluation framework for paper "VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?"
Official implementation of DiTFuse （TPAMI 2026）
在千问最新的多模态image-text模型Qwen3-VL-4B-Instruct 进行多种lora微调对比效果，通过langchain+RAG+多智能体(Multi-Agent)进行部署
[CVPR 2026 Highlight] ReAG: Reasoning-Augmented Generation for Knowledge-based Visual Question Answering
[CVPR 2025] Code for "Notes-guided MLLM Reasoning: Enhancing MLLM with Knowledge and Visual Notes for Visual Question Answering".
Datasets, case studies and benchmarks for extracting structured information from PDFs, HTML files or images, created by the Parsee.ai team. Datasets also on Hugging Face: https://huggingface.co/parsee-ai
Awesome-HCI （Ubiquitous, LLM, MLLM, Agent, RAG, Embodied-AI, RLHF)