#Multimodal-learning
Showing 35 of 35 repositories tagged #multimodal-learning, ranked by stars
Reading list for research topics in multimodal machine learning
Become a cracked AI/ML Research Engineer
An open-source framework for training large multimodal models.
[CVPR 2024 & TPAMI 2025] UniRepLKNet
ICCV 2023-2025 Papers: Discover cutting-edge research from ICCV 2023-25, the leading computer vision conference. Stay updated on the latest in computer vision and deep learning, with code included. โญ support visual intelligence development!
Papers, code and datasets about deep learning and multi-modal learning for video analysis
[NeurIPS 2021] Multiscale Benchmarks for Multimodal Representation Learning
This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI"
A Survey on Multimodal Retrieval-Augmented Generation
Official Pytorch implementation of "OmniNet: A unified architecture for multi-modal multi-task learning" | Authors: Subhojeet Pramanik, Priyanka Agrawal, Aman Hussain
Multi-modality pre-training
Knowledge-Aware machine LEarning (KALE): accessible machine learning from multiple sources for interdisciplinary research, part of the ๐ฅPyTorch ecosystem. โญ Star to support our work!
[CVPR'24 Highlight] GPT4Point: A Unified Framework for Point-Language Understanding and Generation.
Official repo of "Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokens"
Research Trends in LLM-guided Multimodal Learning.
A curated list of awesome Multimodal studies.
Projects based on SigLIP (Zhai et. al, 2023) and Hugging Face transformers integration ๐ค
Reliable, minimal and scalable library for pretraining foundation and world models
From scratch implementation of a vision language model in pure PyTorch
[ICLR 2023] MultiViz: Towards Visualizing and Understanding Multimodal Models
A Context-aware Visual Attention-based training pipeline for Object Detection from a Webpage screenshot!
PyTorch implementation of our graph convolutional network (GCN) for human motion generation from music. Also with paired dance-music data for training!
A paper list of pre-trained language models (PLMs).
my solution with 0.67 accuracy
Quick Long Video Understanding [TMLR2025]
Latest Advances on (RL based) Multimodal Reasoning and Generation in Multimodal LLMs
LoFS ้ขๅๆฌๅฐ็ฅ่ฏๅบ็ๆก้ข็บงๆไปถ็ฎก็ไธไฟกๆฏๆฃ็ดขๅๅฏน่ฏๅบ็จ
Hub for researchers exploring VLMs and Multimodal Learning:)
Awesome Mixture of Experts (MoE): A Curated List of Mixture of Experts (MoE) and Mixture of Multimodal Experts (MoME)
A Unified Framework for Benchmarking Generative Electrocardiogram-Language Models (ELMs)
Implementation of AutoRT: "AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents"
[MLHC 2025] ECG-Byte: A Tokenizer for End-to-End Generative Electrocardiogram Language Modeling
[CVPR 2025] Code for "Notes-guided MLLM Reasoning: Enhancing MLLM with Knowledge and Visual Notes for Visual Question Answering".
Build production-ready AI agents with the Strands Agents SDK and AWS services. This repository demonstrates how you can create multi-modal systems with persistent memory in minimal code. Progress from your first agent to production-ready systems through hands-on chapters.