#Vlm
Showing 60 of 102 repositories tagged #vlm, ranked by stars
๐ค Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
SGLang is a high-performance serving framework for large language models and multimodal models.
Production ready toolkit to run AI locally
A collection of tutorials on state-of-the-art computer vision models and techniques. Explore everything from foundational architectures like ResNet to cutting-edge models like RF-DETR, YOLO11, SAM 3, and Qwen3-VL.
Anomaly detection related books, papers, videos, and toolboxes. Last update late 2025 for LLM and VLM works!
Run frontier LLMs and VLMs locally on Qualcomm devices across NPU, GPU, and CPU with a few lines of code
The official repository for ERNIE 4.5 and ERNIEKit โ its industrial-grade development toolkit based on PaddlePaddle.
The end of web parsing. The beginning of scalable pixel-native search. link: https://pixelrag.ai/
Solve Visual Understanding with Reinforced VLMs
A Low-Code MCP Framework for Building Complex and Innovative RAG Pipelines
๐100+ ๅๅ LLM / RL ๅ็ๅพ๐๏ผใๅคงๆจกๅ็ฎๆณใไฝ่ ๅทจ็ฎ๏ผ๐ฅ๏ผ100+ LLM/RL Algorithm Maps ๏ผ
StarVector is a foundation model for SVG generation that transforms vectorization into a code generation task. Using a vision-language modeling architecture, StarVector processes both visual and textual inputs to produce high-quality SVG code with remarkable precision.
[CVPR 2026] PromptEnhancer is a prompt-rewriting tool, refining prompts into clearer, structured versions for better image generation.
The official repo of MiniMax-Text-01 and MiniMax-VL-01, large-language-model & vision-language-model based on Linear Attention
An AI-powered file management tool that ensures privacy by organizing local texts, images. Using Llama3.2 3B and Llava v1.6 models with the Nexa SDK, it intuitively scans, restructures, and organizes files for quick, seamless access and easy retrieval.
Skywork-R1V is an advanced multimodal AI model series developed by Skywork AI, specializing in vision-language reasoning.
A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.
[NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
Open-Source AI Camera Skills Platform, AI NVR & CCTV Surveillance. Local VLM video analysis with Qwen, DeepSeek, SmolVLM, LLaVA, YOLO26. LLM-powered agentic security camera agent โ watches, understands, remembers & guards your home via Telegram, Discord or Slack. Pluggable AI skills. OpenAI, Google, Anthropic or local AI. Runs on Mac Mini & AI PC.
[EMNLP-2024] Build multimodal language agents for fast prototype and production
The Cradle framework is a first attempt at General Computer Control (GCC). Cradle supports agents to ace any computer task by enabling strong reasoning abilities, self-improvment, and skill curation, in a standardized general environment with minimal requirements.
ChatGPT Plus/Team/Pro ่ฎข้ ๅ่ฎฎ็ซฏๅฐ็ซฏ้ๆพๅทฅๅ ท้ ยท hCaptcha ่ง่งๆฑ่งฃๅจ ยท ๅๆฌบ่ฏๆบๅถๅฎ่ฏ็ ็ฉถ / End-to-end protocol replay toolkit for ChatGPT Plus/Team/Pro subscription with from-scratch hCaptcha solver and empirical anti-fraud research
Open source implementation and extension of Google Researchโs PaperBanana for automated academic figures, diagrams, and research visuals, expanded to new domains like slide generation.
A reading list for large models safety, security, and privacy (including Awesome LLM Security, Safety, etc.).
๐๐๐ A collection of some awesome public YOLO object detection series projects and the related object detection datasets.
The NVIDIA VSS Blueprint is a suite of reference architectures for building GPU-accelerated vision agents and AI-powered video analytics applications.
Declarative way to run AI models in React Native on device, powered by ExecuTorch.
TokenSpeed is a speed-of-light LLM inference engine.
Awesome-Jailbreak-on-LLMs is a collection of state-of-the-art, novel, exciting jailbreak methods on LLMs. It contains papers, codes, datasets, evaluations, and analyses.
Camera monitoring with VLM
Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.
A family of lightweight multimodal models.
MindSpore + ๐คHuggingface: Run any Transformers/Diffusers model on MindSpore with seamless compatibility and acceleration.
ใๆฐๅขๆบ่ฝไฝๆจกๅผใๅฎๅ็ซฏๅ จๅบๆฏGPTๅฉๆ๏ผๅฏ็จ้ณ้้ฎๅค่ตทๅนถ่ฟ่ก่ฏญ้ณไบคๆต๏ผๆฏๆ่็ฝใๆ็ งใๆจกๆฟใ้ไปถ่งฃๆใๆบ่ฝไฝๆจกๅผ็ญ | GPT assistant for Android, activated via volume keys for voice interaction, supporting features such as networking, taking photos, templates, parsing PDF and Office documents, and agent mode.
A curated list of 3D Vision papers relating to Robotics domain in the era of large models i.e. LLMs/VLMs, inspired by awesome-computer-vision, including papers, codes, and related websites
๐๐๐A collection of some awesome public projects about Large Language Model(LLM), Vision Language Model(VLM), Vision Language Action(VLA), AI Generated Content(AIGC), the related Datasets and Applications.
Official Repository of "LLM ร DATA" Survey Paper
[CVPR 2026๐ฅ] ๐งโ๐จ OmniLottie, an open-sourced multi-modal instructed vector animation generator that produces Lottie JSONs.
Dingo: A Comprehensive AI Data, Model and Application Quality Evaluation Tool
MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research ยท ๆต่งๅจ้่ฟ่ก็ๅฎๅๆจกๆๅจ ยท Browser-hosted Android Simulator ยท Verifiable Evaluation ยท Scalable Online RL Training
๐ Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support
[ECCV 2026] SparkVSR: Interactive Video Super-Resolution via Sparse Keyframe Propagation
This repo contains the code for "VLM2Vec" [ICLR 2025], "VLM2Vec-V2 [TMLR 2026]", and "MMEB-V3"
ScreenAgent: A Computer Control Agent Driven by Visual Language Large Model (IJCAI-24)
Custom ComfyUI nodes for Vision Language Models, Large Language Models, Image to Music, Text to Music, Consistent and Random Creative Prompt Generation
Flame is an open-source multimodal AI system designed to translate UI design mockups into high-quality React code. It leverages vision-language modeling, automated data synthesis, and structured training workflows to bridge the gap between design and front-end development.
A hub for various industry-specific schemas to be used with VLMs.
Use the reMarkable2 as an interface to vision-LLMs (ChatGPT, Claude, Gemini). Ghost in the machine!
RAI is a vendor agnostic agentic framework for Physical AI robotics, utilizing ROS 2 tools to perform complex actions, defined scenarios, free interface execution, log summaries, voice interaction and more.
Node Creative Coding / 3D / Image Processing tool inspired by Quartz Composer
OpenGUI is an Android GUI agent framework for phone-use AI that can see, plan, and operate real mobile apps through the GUI.
Go with your own intelligence - Go applications that directly integrate llama.cpp for local inference using hardware acceleration.
๐๐๐ This repository lists some awesome public CUDA, cuda-python, cuBLAS, cuDNN, CUTLASS, TensorRT, TensorRT-LLM, Triton, TVM, MLIR, PTX and High Performance Computing (HPC) projects.
An Asynchronous Reinforcement Learning Engine for Omni-Modal Post-Training at Scale
๐ Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future
LightRFT: Light, Efficient, Omni-modal & Reward-model Driven Reinforcement Fine-Tuning Framework
Open Android AI agent runtime for phone control, app automation, VLM screen reading, skill routing, mini apps, and Mihomo VPN workflows.
Reliable Automation Agents at Scale
[Fully open] [Encoder-free MLLM] Vision as LoRA
Official repo of "Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokens"