#Data-parallelism

deepspeedai

DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

hpcaitech

ColossalAI

Making large AI models cheaper, faster and more accessible

cerndb

dist-keras

Distributed Deep Learning, with a focus on distributed training, using Keras and Apache Spark.

dkeras-project

dkeras

Distributed Keras Engine, Make Keras faster with only one line of code.

hkproj

pytorch-transformer-distributed

Distributed training (multi-node) of a Transformer model