#Self-distillation

Tebmer

Awesome-Knowledge-Distillation-of-LLMs

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vertical Distillation of LLMs.

chrisliu298

awesome-on-policy-distillation

A curated collection of papers, technical reports, frameworks, and tools for on-policy distillation (OPD) of large language models

THU-BPM

RLCSD

Source code of paper "RLCSD: Reinforcement Learning with Contrastive On-Policy Self-Distillation"