#Kafka
Showing 60 of 320 repositories tagged #kafka, ranked by stars
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. The next cohort starts in January 2026. Join the course here ๐๐ผ
Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
Sarama is a Go library for Apache Kafka.
Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
Building event-driven applications the easy way in Go.
Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.
Fancy stream processing made operationally mundane
Kafka library in Go
1000+ DevOps Bash Scripts - AWS, GCP, Kubernetes, Docker, CI/CD, APIs, SQL, PostgreSQL, MySQL, Hive, Impala, Kafka, Hadoop, Jenkins, GitHub, GitLab, BitBucket, Azure DevOps, TeamCity, Spotify, MP3, LDAP, Code/Build Linting, pkg mgmt for Linux, Mac, Python, Perl, Ruby, NodeJS, Golang, Advanced dotfiles: .bashrc, .vimrc, .gitconfig, .screenrc, tmux..
Free and open log management
Python Stream Processing
The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL
Apache Camel is an open source integration framework with 350+ connectors. Write routes in Java, YAML, or XML. Run on Spring Boot, Quarkus, or standalone. Apache License 2.0.
Kafka Web UI
Python client for Apache Kafka
Apache Kafkaยฎ running on Kubernetes
๐ ไธชไบบๅ่ฟ็ๆๆฏๅไบซ... ( topic: ้ซๆง่ฝๆถๆ, ไธญ้ดไปถๅ็ๅฎ่ทต๏ผไบๅ็๏ผGolang ็ญ)
FastStream is an asynchronous Python framework for building event-driven applications. It brings together message broker integration, dependency injection, validation, testing utilities, and AsyncAPI documentation generation in a single toolkit
๐ฅๅบไบgo-zero(go zero) ๅพฎๆๅกๅ จๆๆฏๆ ๅผๅๆไฝณๅฎ่ทต้กน็ฎใDevelop best practice projects based on the full technology stack of go zero (go zero) microservices.
Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)
Distributed stream processing engine in Rust
Orchestrate end-to-end encryption, cryptographic identities, mutual authentication, and authorization policies between distributed applications โ at massive scale.
Redpanda Console is a developer-friendly UI for managing your Kafka/Redpanda workloads. Console gives you a simple, interactive approach for gaining visibility into your topics, masking data, managing consumer groups, and exploring real-time data with time-travel debugging.
Learn Agentic AI using Dapr Agentic Cloud Ascent (DACA) Design Pattern and Agent-Native Cloud Technologies: OpenAI Agents SDK, Memory, MCP, A2A, Knowledge Graphs, Dapr, Rancher Desktop, and Kubernetes.
A Microservice Toolkit from The New York Times
docker compose files to create a fully working kafka stack
Alink is the Machine Learning algorithm platform based on Flink, developed by the PAI team of Alibaba computing platform.
Usage-based pricing and billing for developers ๐ Cloud or self-hosted โ๏ธ No-code UI ๐ฐ Realtime usage metering ๐ Credits & top-ups ๐ Control feature access
The open source and enterprise-ready webhooks service ๐ฆ
Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage
franz-go is a feature complete, pure Go library for Kafka from 0.8.0 through 4.2+. Producing, consuming, transacting, administrating, etc.
Goka is a compact yet powerful distributed stream processing library for Apache Kafka written in Go.
ๆทฑๅณๅฐ้ๅคงๆฐๆฎๅฎขๆตๅๆ็ณป็ป๐๐๐
Full-stack .Net 10 Clean Architecture (Microservices, Modular Monolith, Monolith), Blazor, Angular 22, React 19, Vue 3.5, BFF with YARP, NextJs 16, Domain-Driven Design, CQRS, SOLID, Asp.Net Core Identity Custom Storage, OpenID Connect, EF Core, OpenTelemetry, SignalR, Background Services, Health Checks, Rate Limiting, Clouds (Azure, AWS, GCP), ...
Flow collector, enricher and visualizer
2023ๅนดๆๆฐๆด็ c++ๅ็ซฏๅผๅ๏ผ1000็ฏไผ็งๅๆ๏ผๅซๅ ๅญ๏ผ็ฝ็ป๏ผๆถๆ่ฎพ่ฎก๏ผ้ซๆง่ฝ๏ผๆฐๆฎ็ปๆ๏ผๅบ็ก็ปไปถ๏ผไธญ้ดไปถ๏ผๅๅธๅผ็ธๅ ณ
A swiss army knife CLI tool for interacting with Kafka, RabbitMQ and other messaging systems.
Metering and Billing for AI, API and DevOps. Collect and aggregate millions of usage events in real-time and enable usage-based billing.
Kafka Docker for development. Kafka, Zookeeper, Schema Registry, Kafka-Connect, , 20+ connectors
Apache Kafka, Apache Flink and Confluent Platform examples and demos
A fully asynchronous, futures-based Kafka client library for Rust based on librdkafka
This repo contains my notes on working with Go and computer systems.
๐ฆญ Distributed log streaming engine built from first principles
Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning
Vanus is a Serverless, event streaming system with processing capabilities. It easily connects SaaS, Cloud Services, and Databases to help users build next-gen Event-driven Applications.
More than 2000+ Data engineer interview questions.
Event Sourcing for Go!
Trench โ Open-Source Analytics Infrastructure. A single production-ready Docker image built on ClickHouse, Kafka, and Node.js for tracking events. Easily build product analytics dashboards, LLM RAGs, observability platforms, or any other analytics product.
Communication infrastructure for the AI era โ one binary, one broker, one storage layer, any protocol
Python Streaming DataFrames for Kafka
100xDevs Cohort 3 ๐ by Harkirat Singh. This repository contains Lecture Codes, Notes, Assignments, Assignment Solution Code and Some Good Articles/Blogs Link. ๐ฅ
Test your code without writing mocks with ephemeral Docker containers ๐ฆ Setup popular services with just a couple lines of code โฑ๏ธ No bash, no yaml, only code ๐ป
Machine Learning Platform and Recommendation Engine built on Kubernetes
Rust client for Apache Kafka
Example end to end data engineering project.
High performance mqtt broker
50+ DockerHub public images for Docker & Kubernetes - DevOps, CI/CD, GitHub Actions, CircleCI, Jenkins, TeamCity, Alpine, CentOS, Debian, Fedora, Ubuntu, Hadoop, Kafka, ZooKeeper, HBase, Cassandra, Solr, SolrCloud, Presto, Apache Drill, Nifi, Spark, Consul, Riak
:fireworks: Flexible and extendable containerized workspaces. Now. with free offline chat GPT!!! ๐๐๐
Wait4X allows you to wait for a port or a service to enter the requested state.