#Knowledge-distillation
Showing 40 of 40 repositories tagged #knowledge-distillation, ranked by stars
A treasure chest for visual classification and recognition powered by PaddlePaddle
Awesome Knowledge Distillation
Collection of AWESOME vision-language models for vision tasks
AI Powered Knowledge Graph Generator
把书、长视频、播客等高价值内容蒸馏成可执行的 Agent Skills
EasyNLP: A Comprehensive and Easy-to-use NLP Toolkit
A curated list for Efficient Large Language Models
A PyTorch implementation for exploring deep and shallow knowledge distillation (KD) experiments with flexibility
Improving Convolutional Networks via Attention Transfer (ICLR 2017)
NLP DNN Toolkit - Building Your NLP DNN Models Like Playing Lego
This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vertical Distillation of LLMs.
Collection of recent methods on (deep) neural network compression and acceleration.
The official implementation of [CVPR2022] Decoupled Knowledge Distillation https://arxiv.org/abs/2203.08679 and [ICCV2023] DOT: A Distillation-Oriented Trainer https://openaccess.thecvf.com/content/ICCV2023/papers/Zhao_DOT_A_Distillation-Oriented_Trainer_ICCV_2023_paper.pdf
An Extendible (General) Continual Learning Framework based on Pytorch - official codebase of Dark Experience for General Continual Learning
A Pytorch Knowledge Distillation library for benchmarking and extending works in the domains of Knowledge Distillation, Pruning, and Quantization.
From teacher to tiles — a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous batching, speculative decoding, a Rust gateway, a JAX oracle, and interpretability tooling.
A curated collection of papers, technical reports, frameworks, and tools for on-policy distillation (OPD) of large language models
Infrastructures™ for Machine Learning Training/Inference in Production.
Official pytorch Implementation of Relational Knowledge Distillation, CVPR 2019
A curated collection of papers and resources on On-Policy Distillation for Large Language Models.
大模型/LLM推理和部署理论与实践
[CVPR 2024 Highlight] Logit Standardization in Knowledge Distillation
Explore concepts like Self-Correct, Self-Refine, Self-Improve, Self-Contradict, Self-Play, and Self-Knowledge, alongside o1-like reasoning elevation🍓 and hallucination alleviation🍄.
Repository to track the progress in model compression and acceleration
大师.skill — 输入行业，自动调研 6 轨[行业大佬 / 工具地图 / 工作流 / 知识正典 / 信息源 / 术语标准] → 提炼为可运行的行业 Master OS skill；装到任意 Claude Code / OpenClaw / Codex / Hermes agent 即让其进入「这一行的资深人」模式。MIT，Python + Shell。
Code and pretrained models for paper: Data-Free Adversarial Distillation
A collection of resources on using BERT (https://arxiv.org/abs/1810.04805 ) and related Language Models in production environments.
[ICLR 2022] Code for Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillation (GLNN)
(NeurIPS 2022) Official Implementation of "Preservation of the Global Knowledge by Not-True Distillation in Federated Learning"
Code for ML Doctor
Awesome-3D/Multimodal-Anomaly-Detection-and-Localization/Segmentation/3D-KD/3D-knowledge-distillation
Knowledge Distillation Toolkit
This resposity maintains a collection of important papers on knowledge distillation (awesome-knowledge-distillation)).
(CVPR-Oral 2021) PyTorch implementation of Knowledge Evolution approach and Split-Nets
easy-bert是一个中文NLP工具，提供诸多bert变体调用和调参方法，极速上手；清晰的设计和代码注释，也很适合学习
Gather research papers, corresponding codes (if having), reading notes and any other related materials about Hot🔥🔥🔥 fields in Computer Vision based on Deep Learning.
Compressing Representations for Self-Supervised Learning
The official implementation for ECCV22 paper: "FOSTER: Feature Boosting and Compression for Class-Incremental Learning" in PyTorch.
Multi-Label Image Classification via Knowledge Distillation from Weakly-Supervised Detection (ACM MM 2018)
Official code of Why Knowledge Distillation Works in Generative Models: A Minimal Working Explanation (NeurIPS 2025)