#Distributed-training
Showing 35 of 35 repositories tagged #distributed-training, ranked by stars
Learn how to develop, deploy and iterate on production-grade ML applications.
The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNetV4, MobileNet-V3 & V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more
PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）
Easy-to-use and powerful LLM and SLM library with awesome model zoo.
Run, manage, and scale AI workloads on any AI infrastructure. Use one system to access & manage all AI compute (Kubernetes, Slurm, 20+ clouds, on-prem).
Build, Manage and Deploy AI/ML Systems
Democratizing Reinforcement Learning for LLMs
FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.
A high performance and generic framework for distributed DNN training
Fast and flexible AutoML with learning guarantees.
Determined is an open-source machine learning platform that simplifies distributed training, hyperparameter tuning, experiment tracking, and resource management. Works with PyTorch and TensorFlow.
Training and serving large-scale neural networks with auto parallelization.
Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.
DLRover: An Automatic Distributed Deep Learning System
Library for Fast and Flexible Human Pose Estimation
🌾 OAT: A research-friendly framework for LLM online alignment, including reinforcement learning, preference learning, etc.
An Asynchronous Reinforcement Learning Engine for Omni-Modal Post-Training at Scale
Resource-adaptive cluster scheduler for deep learning training.
Collection of best practices, reference architectures, model training examples and utilities to train large models on AWS.
TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.
Project Tapestry aims to give every nation and participant frontier AI they can call their own — uniting a global consortium to train a shared frontier model from which partners build and own sovereign models aligned to their national, socio-cultural, and industrial needs.
A lightweight runtime health check for PyTorch training runs.
Distributed training (multi-node) of a Transformer model
TePDist (TEnsor Program DISTributed) is an HLO-level automatic distributed system for DL models.
This repo covers Kubeflow Environment with LABs: Kubeflow GUI, Jupyter Notebooks on pods, Kubeflow Pipelines, Experiments, KALE, KATIB (AutoML: Hyperparameter Tuning), KFServe (Model Serving), Training Operators (Distributed Training), Projects, etc.
Pytorch分布式训练框架
Next-gen AI-native tensor-network-based quantum software framework
The LUMI AI Guide is designed to assist users in migrating their machine learning applications from smaller-scale computing environments to the LUMI supercomputer.
Machine learning library, Distributed training, Deep learning, Reinforcement learning, Models, TensorFlow, PyTorch
Create, List, Update, Delete Amazon EKS clusters. Deploy and manage software on EKS. Run distributed model training and inference examples.
Dynamic training with Apache MXNet reduces cost and time for training deep neural networks by leveraging AWS cloud elasticity and scale. The system reduces training cost and time by dynamically updating the training cluster size during training, with minimal impact on model training accuracy.
Large-scale Auto-Distributed Training/Inference Unified Framework | Memory-Compute-Control Decoupled Architecture | Multi-language SDK & Heterogeneous Hardware Support
All about large language models
A simple and intuitive library for diffusion models using Flax and Jax. Includes detailed notebooks on DDPM, DDIM, and EDM with simplified mathematical explanations. Made as part of my journey for learning and experimenting with generative AI.