#Pipeline-parallelism

deepspeedai

DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

hpcaitech

ColossalAI

Making large AI models cheaper, faster and more accessible

bigscience-workshop

petals

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

kakaobrain

torchgpipe

A GPipe implementation in PyTorch

ai-decentralized

BloomBee

Decentralized LLMs fine-tuning and inference with offloading