#Map-reduce

chrislusf

gleam

Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.

numaproj

numaflow

Kubernetes-native platform to run massively parallel data/streaming jobs

Qihoo360

poseidon

A search engine which can hold 100 trillion lines of log data.

phelps-sg

python-bigdata

Data science and Big Data with Python

asavinov

prosto

Prosto is a data processing toolkit radically changing how data is processed by heavily relying on functions and operations with functions - an alternative to map-reduce and join-groupby

neochaotic

leoflow

GitOps-first, container-native workflow orchestrator in Go (Airflow-UI compatible).