#Vision-language-model
Showing 60 of 84 repositories tagged #vision-language-model, ranked by stars
[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.
[CVPR 2024 Oral] InternVL Family: A Pioneering Open-Source Alternative to GPT-4o. 接近GPT-4o表现的开源多模态对话模型
Effortless data labeling with AI support from Segment Anything and other awesome models.
👀「大模型」2小时从0训练65M参数的视觉多模态VLM！Train a 65M-parameter VLM from scratch in just 2h!
The official repo of Qwen-VL (通义千问-VL) chat & pretrained large vision language model proposed by Alibaba Cloud.
MineContext is your proactive context-aware AI partner（Context-Engineering+ChatGPT Pulse）
MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.
The official repo of MiniMax-Text-01 and MiniMax-VL-01, large-language-model & vision-language-model based on Linear Attention
Collection of AWESOME vision-language models for vision tasks
InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions
The Cradle framework is a first attempt at General Computer Control (GCC). Cradle supports agents to ace any computer task by enabling strong reasoning abilities, self-improvment, and skill curation, in a standardized general environment with minimal requirements.
A collection of original, innovative ideas and algorithms towards Advanced Literate Machinery. This project is maintained by the OCR Team in the Language Technology Lab, Tongyi Lab, Alibaba Group.
Seed1.5-VL, a vision-language foundation model designed to advance general-purpose multimodal understanding and reasoning, achieving state-of-the-art performance on 38 out of 60 public benchmarks.
日本語LLMまとめ - Overview of Japanese LLMs
OpenAI and Anthropic compatible server for Apple Silicon. Run LLMs and vision-language models (Llama, Qwen-VL, LLaVA) with continuous batching, MCP tool calling, and multimodal support. Native MLX backend, 400+ tok/s. Works with Claude Code.
Fine-tune LLMs on your Mac with Apple Silicon. SFT, DPO, GRPO, Vision, TTS, STT, Embedding, and OCR fine-tuning — natively on MLX. Unsloth-compatible API.
This series will take you on a journey from the fundamentals of NLP and Computer Vision to the cutting edge of Vision-Language Models.
Fully Open Framework for Democratized Multimodal Training
Parsing-free RAG supported by VLMs
[CVPR 2024 🔥] Grounding Large Multimodal Model (GLaMM), the first-of-its-kind model capable of generating natural language responses that are seamlessly integrated with object segmentation masks.
[CVPR 2024 Highlight🔥] Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding
About This repository is a curated collection of the most exciting and influential CVPR 2025 papers. 🔥 [Paper + Code + Demo]
[CVPR 2024] Alpha-CLIP: A CLIP Model Focusing on Wherever You Want
A curated list of 3D Vision papers relating to Robotics domain in the era of large models i.e. LLMs/VLMs, inspired by awesome-computer-vision, including papers, codes, and related websites
OmniVinci is an omni-modal LLM for joint understanding of vision, audio, and language.
Evaluating text-to-image/video/3D models with VQAScore
🎉 PILOT: A Pre-trained Model-Based Continual Learning Toolbox
[ECCV2024] Grounded Multimodal Large Language Model with Localized Visual Tokenization
Chatbot Arena meets multi-modality! Multi-Modality Arena allows you to benchmark vision-language models side-by-side while providing images as inputs. Supports MiniGPT-4, LLaMA-Adapter V2, LLaVA, BLIP-2, and many more!
Flame is an open-source multimodal AI system designed to translate UI design mockups into high-quality React code. It leverages vision-language modeling, automated data synthesis, and structured training workflows to bridge the gap between design and front-end development.
Cambrian-S: Towards Spatial Supersensing in Video
An open-source implementation for training LLaVA-NeXT.
Code for RoboFlamingo
[CVPR 2025 Highlight] Official code for "Olympus: A Universal Task Router for Computer Vision Tasks"
[CVPR 2024 Highlight] OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation
🖨️ Automated scanner document processor with AI-powered naming and WebDav integration. Receives scans via FTP, extracts text using Vision AI, generates intelligent filenames with Ollama AI, and uploads to your cloud storage.
Stream-Omni is a GPT-4o-like language-vision-speech chatbot that simultaneously supports interaction across various modality combinations.
Official repo of "Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokens"
[NeurIPS 2025] NAUTILUS: A Large Multimodal Model for Underwater Scene Understanding
Recipes for learning, fine-tuning, and adapting ColPali to your multimodal RAG use cases. 👨🏻🍳
Seamlessly integrate state-of-the-art transformer models into robotics stacks
A Simple, Lightweight, and Extensible Serving Framework for X-AnyLabeling
Vision Document Retrieval (ViDoRe): Benchmark. Evaluation code for the ColPali paper.
Bilingual (中文+EN) ML / LLM / diffusion / agent interview cheat sheets for AI 秋招 — generated by ARIS /interview-cheatsheet, rendered by /render-html into single-file HTML, reads anywhere — plus a CV→DBLP-fact-checked academic homepage generator and hand-authored long-form blogs 🌱
From scratch implementation of a vision language model in pure PyTorch
[ECCV 2024] ShareGPT4V: Improving Large Multi-modal Models with Better Captions
Foundation models based medical image analysis
a comprehensive and critical synthesis of the emerging role of GenAI across the full autonomous driving stack
[NeurIPS-2023] Annual Conference on Neural Information Processing Systems
LitePali is a minimal, efficient implementation of ColPali for image retrieval and indexing, optimized for cloud deployment.
Vision Search Assistant: Empower Vision-Language Models as Multimodal Search Engines
On-device AI recipes for building private, real-time applications with production-grade voice, language, and vision SDKs. Ready-to-use, open-source code examples.
Sleek, mobile-friendly web UI for NVIDIA LocateAnything-3B — open-vocabulary object detection & grounding on your own GPU, via one docker compose up.
[ECCV 2024] Official PyTorch Implementation of "How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMs"
[IGARSS 2024 ORAL] Official implementation of "Composed Image Retrieval for Remote Sensing".
AI-powered browser automation for Go — describe tasks in plain English, let the agent handle the clicks.
[CVPR 2025] DivPrune: Diversity-based Visual Token Pruning for Large Multimodal Models
AI-powered visual reasoning tools for broadcast & ProAV. PTZ camera tracking, object detection, scene analysis using Moondream VLM. By StreamGeeks & PTZOptics.
LabVLA: Grounding Vision–Language–Action Models in Scientific Laboratories
🏆 Official implementation of LangCoop: Collaborative Driving with Natural Language