#Distributed-computing
Showing 60 of 92 repositories tagged #distributed-computing, ranked by stars
Making large AI models cheaper, faster and more accessible
Machine Learning、Deep Learning、PostgreSQL、Distributed System、Node.Js、Golang
Automate code & data workflows with interactive Elixir notebooks
High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale
Proto Actor - Ultra fast distributed actors for Go, C# and Java/Kotlin
Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.
A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
Accelerated deep learning R&D
Training and serving large-scale neural networks with auto parallelization.
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.
zenoh unifies data in motion, data in-use, data at rest and computations. It carefully blends traditional pub/sub with geo-distributed storages, queries and computations, while retaining a level of time and space efficiency that is well beyond any of the mainstream stacks.
Open-source software for volunteer computing and grid computing.
A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.
Everything about federated learning, including research papers, books, codes, tutorials, videos and beyond
Ultrafast serverless GPU inference, sandboxes, and background jobs
Python-based research interface for blackbox and hyperparameter optimization, based on the internal Google Vizier Service.
Distributed Deep learning with Keras & Spark
Implementation of Communication-Efficient Learning of Deep Networks from Decentralized Data
The current, performant & industrial strength version of Holochain on Rust.
Unified Interface for Constructing and Managing Workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow.
Distributed Stockfish analysis for lichess.org
Bagua Speeds up PyTorch
GPU worker client for the Talos network. Pairs with your Talos account, serves open-model inference jobs over a WebSocket, and reports uptime for payouts.
Framework for large distributed pipelines
A library to train, evaluate, interpret, and productionize decision forest models such as Random Forest and Gradient Boosted Decision Trees.
Backend.AI is a streamlined, container-based computing cluster platform that hosts popular computing/ML frameworks and diverse programming languages, with pluggable heterogeneous accelerator support including CUDA GPU, ROCm GPU, Gaudi NPU, Google TPU, GraphCore IPU and other NPUs.
An experimental platform for federated learning.
User-friendly Scheduler for sub-node tasks for HPC systems
Fast Raft framework using the Redis protocol for Go
Kubernetes Operator for MPI-based applications (distributed training, HPC, etc.)
Auto Tune Models - A multi-tenant, multi-data system for automated machine learning (model selection and tuning).
Harmonious distributed data analysis in Rust.
:books: :eyeglasses: A collection of research papers, codes, tutorials and blogs on Federated Computing/Learning.
A curated list of awesome Distributed Deep Learning resources.
High performance distributed framework for training deep learning recommendation models based on PyTorch.
Self-hosted AI agent OS. Your memory, chat, agents, and files stay on hardware you own, offline by default, cloud by choice. Offline AI memory (taOSmd), self-hosted multi-framework group chat, a full web desktop + app store, and auto-clustering across the consumer hardware you already have (Orange/Raspberry Pi, Mac mini, gaming PC).
😇 A Docker Compose bundle to run on servers with spare CPU, RAM, disk, and bandwidth to help the world. Includes Tor, ArchiveWarrior, BOINC, and more...
Notebooks for the O'Reilly book "Learning Ray"
An easy-to-use WebUI for crypto plotting and farming. Offers Bladebit, Gigahorse, MadMax, Chiadog and Plotman in a Docker container. Supports Chia, MMX, Chives, Flax, and HDDCoin among others.
Python for Data Science (Seminar Course at UC Berkeley; AY 250)
An advanced distributed task flow management on top of Celery
A Clojure dataframe library that runs on Spark
Arkouda (αρκούδα): Interactive Data Analytics at Supercomputing Scale :bear:
Fleex makes it easy to create multiple VPS on cloud providers and use them to distribute workloads.
Distributed tensors and Machine Learning framework with GPU and MPI acceleration in Python
Modern web-based distributed hashcracking solution, built on hashcat
A front-end GUI for interacting with the AI Horde / Stable Diffusion distributed cluster
Distributed DAG (Directed acyclic graph) framework for machine learning with UI
The default client software for volunteering to generate images for the crowdsourced AI-Horde
A distributed block-based data storage and compute engine
Post-money economy simulator in the form of a free fictional game based on gift economy that tests the hypothesis of having a market system without any exchange be it barters or money.
A Web Crawler based on LLMs implemented with Ray and Huggingface. The embeddings are saved into a vector database for fast clustering and retrieval. Use it for your RAG.
A Lightweight, Cloud-Native Stateful Distributed Dataflow Engine
TePDist (TEnsor Program DISTributed) is an HLO-level automatic distributed system for DL models.
Trader is an algorithmic trading framework and host built on Microsoft Orleans.
An analytical performance modeling tool for deep neural networks.
PySpark Algorithms Book: https://www.amazon.com/dp/B07X4B2218/ref=sr_1_2
Automated Parallelization System and Infrastructure for Multiple Ecosystems
Legacy code connected to the high-assurance implementation of the Ouroboros protocol family