#Collective-communication

Showing 1 of 1 repositories tagged #collective-communication, ranked by stars

pytorch-transformer-distributed

Distributed training (multi-node) of a Transformer model

★ 99 ⑂ 43 —

Related Topics

#data-parallelism #deep-learning #distributed-data-parallel #distributed-training #gradient-accumulation #machine-learning #model-parallelism #pytorch #tutorial