#Streaming-data

Showing 34 of 34 repositories tagged #streaming-data, ranked by stars

oxnr
oxnr
awesome-bigdata

A curated list of awesome big data frameworks, ressources and other awesomeness.

Score
100
β˜… 14.5k β‘‚ 2.6k +8/day
johnkerl
johnkerl
miller

Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON

Score
67
β˜… 9.9k β‘‚ 239 +1/day
Go
redpanda-data
redpanda-data
connect

Fancy stream processing made operationally mundane

Score
100
β˜… 8.7k β‘‚ 947 +3/day
Go
MaterializeInc
MaterializeInc
materialize

The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL

Score
100
β˜… 6.3k β‘‚ 513 β€”
Rust
online-ml
online-ml
river

🌊 Online machine learning in Python

Score
100
β˜… 5.9k β‘‚ 643 +1/day
Python
readysettech
readysettech
readyset

Readyset is a MySQL and Postgres wire-compatible caching layer that sits in front of existing databases to speed up queries and horizontally scale read throughput. Under the hood, ReadySet caches the results of cached select statements and incrementally updates these results over time as the underlying data changes.

Score
33
β˜… 5.3k β‘‚ 160 +5/day
Rust
fluvio-community
fluvio-community
fluvio

πŸ¦€ event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.

Score
67
β˜… 5.2k β‘‚ 529 +2/day
Rust
piskvorky
piskvorky
smart_open

Utils for streaming large files (S3, HDFS, gzip, bz2...)

Score
0
β˜… 3.5k β‘‚ 389 β€”
Python
bytewax
bytewax
bytewax

Python Stream Processing

Score
0
β˜… 2.0k β‘‚ 111 +4/day
Python
quixio
quixio
quix-streams

Python Streaming DataFrames for Kafka

Score
89
β˜… 1.6k β‘‚ 107 β€”
Python
zpl-c
zpl-c
zpl

πŸ“ Pushing the boundaries of simplicity

Score
100
β˜… 1.1k β‘‚ 49 β€”
C
scikit-multiflow
scikit-multiflow
scikit-multiflow

A machine learning package for streaming data in Python. The other ancestor of River.

Score
100
β˜… 798 β‘‚ 188 β€”
Python
streamdal
streamdal
streamdal

Code-Native Data Privacy

Score
0
β˜… 615 β‘‚ 16 β€”
TypeScript
kLabUM
kLabUM
rrcf

🌲 Implementation of the Robust Random Cut Forest algorithm for anomaly detection on streams

Score
0
β˜… 523 β‘‚ 119 β€”
Python
Sinotrade
Sinotrade
Shioaji

Shioaji all new cross platform api for trading ( θ·¨εΉ³ε°θ­‰εˆΈδΊ€ζ˜“API )

Score
100
β˜… 481 β‘‚ 38 β€”
Dockerfile
ankur-anand
ankur-anand
unisondb

A streaming multimodal database for Edge AI, and Edge Computing.

Score
33
β˜… 419 β‘‚ 21 β€”
Go
lightbend
lightbend
cloudflow

Cloudflow enables users to quickly develop, orchestrate, and operate distributed streaming applications on Kubernetes.

Score
0
β˜… 328 β‘‚ 90 β€”
Scala
Chulong-Li
Chulong-Li
Real-time-Sentiment-Tracking-on-Twitter-for-Brand-Improvement-and-Trend-Recognition

A real-time interactive web app based on data pipelines using streaming Twitter data, automated sentiment analysis, and MySQL&PostgreSQL database (Deployed on Heroku)

Score
67
β˜… 319 β‘‚ 126 β€”
Jupyter Notebook
AbubakrChan
AbubakrChan
crewai-UI-business-product-launch

Streamlit UI for crewai | crewai ui

Score
0
β˜… 191 β‘‚ 50 β€”
Python
wso2
wso2
product-integrator-si

A stream processing runtime that allows connecting any streaming data source to any destination and act on it

Score
100
β˜… 112 β‘‚ 55 β€”
Python
sonhmai
sonhmai
data-systems-design

System Design, Solution Architecture, Data Systems Practice

Score
22
β˜… 81 β‘‚ 23 β€”
systemaccounting
systemaccounting
mxfactorial

a payment application intended for deployment by the united states treasury that replaces banking with accounting

Score
75
β˜… 55 β‘‚ 26 β€”
Rust
lensesio
lensesio
agentic-engineering-for-apache-kafka

AI agent skills for building, operating and troubleshooting Apache Kafka applications. Topic audit, consumer lag, schema review, security, connectors and DLQ

Score
78
β˜… 51 β‘‚ 13 β€”
Python
vales98
vales98
typewriter-gpt-im

A simple case of implementing streaming and displaying content word by word, similar to the official website, using the ChatGPT API interface. Developed using Vue.js and TypeScript.

Score
100
β˜… 45 β‘‚ 10 β€”
Vue
yennanliu
yennanliu
DE-100-days

data engineering 100 days πŸ€– 🧲 🦾 | #DE

Score
11
β˜… 40 β‘‚ 16 β€”
Menziess
Menziess
slipstream-async

Slipstream provides a data-flow model to simplify development of stateful streaming applications.

Score
44
β˜… 39 β‘‚ 2 β€”
Python
xdgrulez
xdgrulez
kafi

The Swiss Army Knife for Kafka

Score
56
β˜… 33 β‘‚ 6 β€”
Python
sdpython
sdpython
pandas-streaming

Streaming API for pandas applied to big datasets

Score
33
β˜… 31 β‘‚ 9 β€”
Python
thammo4
thammo4
uvatradier

Python wrapper for the Tradier brokerage API

Score
50
β˜… 29 β‘‚ 19 β€”
Python
certeu
certeu
morio

Connect - Stream - Observe - Respond | Morio provides the plumbing for your observability needs

Score
0
β˜… 28 β‘‚ 3 β€”
NucleusEngineering
NucleusEngineering
hack-your-pipe

Efficient streaming data ingestion, transformation & activation

Score
0
β˜… 28 β‘‚ 3 β€”
Python
slimandslam
slimandslam
schwab-client-js

A modern wrapper around the Schwab financial API for NodeJS Typescript and Javascript projects. Join our Discord: https://discord.gg/Q9z8EnB8xD

Score
25
β˜… 26 β‘‚ 16 β€”
TypeScript
stensoosaar
stensoosaar
IBKit

Interactive Brokers TWS/Gateway API in Swift

Score
0
β˜… 24 β‘‚ 15 β€”
Swift
aramisfacchinetti
aramisfacchinetti
streaming-json-parser

Streaming JSON parser designed to process JSON data incrementally. The primary goal is to handle potentially incomplete JSON data streams, such as those produced by Large Language Models (LLMs), and return the current state of the parsed object at any time.

Score
0
β˜… 13 β‘‚ 2 β€”
Python
Related Topics
#stream-processing#python#data-engineering#streaming#kafka#data-science#rust#machine-learning#database#trading#data-processing#go

Β© 2026 GitRepoTrend Β· GitHub repositories by topic Β· Updated weekly