#Lakehouse
Showing 34 of 34 repositories tagged #lakehouse, ranked by stars
ClickHouse® is a real-time analytics database management system
Data Agent Ready Warehouse : One for Analytics, Search, AI, Python Sandbox. — rebuilt from scratch. Unified architecture on your S3.
LakeSoul is an end-to-end, realtime cloud-native Lakehouse framework for fast data ingestion, concurrent updates, incremental analytics, multimodal data processing and vector search — powering next-generation BI and AI workloads.
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
Real-time analytics on Postgres tables
Lakekeeper is an Apache-Licensed, secure, fast and easy to use Apache Iceberg REST Catalog written in Rust.
Lakehouse native graph engine with git-style workflows
A curated list of open source tools used in analytics platforms and data engineering ecosystem
End-to-end Data Lakehouse project built on Databricks, following the Medallion Architecture (Bronze, Silver, Gold). Covers real-world data engineering and analytics workflows using Spark, PySpark, SQL, Delta Lake, and Unity Catalog. Designed for learning, portfolio building, and job interviews.
Examples of using Terraform to deploy Databricks resources
Use SQL to build ELT pipelines on a data lakehouse.
The Lakehouse Engine is a configuration driven Spark framework, written in Python, serving as a scalable and distributed engine for several lakehouse algorithms, data flows and utilities for Data Products.
A modern data marketplace that makes collaboration among diverse users (like business, analysts and engineers) easier, increasing efficiency and agility in data projects on AWS.
Quick start: pip install jsoniq ⛈️ RumbleDB 2.1.0 "Cedrus Libani" 🌳 for Apache Spark | Run queries on your large-scale, messy datasets (JSON, text, CSV, Parquet, Delta...) | Data Lakehouse with Updates, Scripting, Declarative Machine Learning and more
Data Forge — a modern data stack playground to practice flows and best practices, not just tools. Spark, Trino, Kafka, Iceberg, ClickHouse, Airflow, MinIO, Superset — all wired together locally with Docker Compose.
🧊 A curated list of OLAP databases, data lake tools, columnar engines, and analytics frameworks for data engineers.
Modern serverless lakehouse implementing HOOK methodology, Unified Star Schema (USS), and Analytical Data Storage System (ADSS) principles on Adventure Works. Features programmatic model generation, event-enhanced Puppini bridges, and temporal resolution across DAS/DAB/DAR layers.
Experimental version. A BYOC option for Snowflake workloads
🦆 Batch data pipeline with Airflow, DuckDB, Delta Lake, Trino, MinIO, and Metabase. Full observability and data quality.
📡 Real-time data pipeline with Kafka, Flink, Iceberg, Trino, MinIO, and Superset. Ideal for learning data systems.
Docker Compose environments for developing modern data platform architectures using Kafka, Flink, Spark, Iceberg, OpenLineage, OpenMetadata, Pinot, ClickHouse, StarRocks + Kpow & Flex by Factor House
600+ free patterns and concepts for data engineers on Azure Databricks, Microsoft Fabric, and PySpark. 12 books covering the full stack. Free forever.
Databricks DLT Apparel Pipeline Project: Learn medallion architecture, streaming, and data engineering with Delta Live Tables. Includes synthetic data, step-by-step guide, and certification prep.
A local-first lakehouse reference architecture for production-minded data platform engineering.
100 essential Databricks concepts for data engineers, organized by category with difficulty levels and self-assessment scoring
170+ curated resources every Databricks Data Engineer should bookmark - tools, courses, creators, labs, and communities
Production-style real-time e-commerce lakehouse with Kafka, Airflow, Databricks, Medallion architecture, data quality, quarantine, Terraform, and Dash analytics.
AI-Lake Format: self-contained Parquet + HNSW index — unifying tabular data, embeddings, and vector search in a single Iceberg-compatible file
Big Data CONFERENCE Europe 2025 🇱🇹 AI, Cloud and Data Conference - Build Your First End-to-End Lakehouse Solution (aka.ms/fabconlake)
Rust-native time-series table format with gap/overlap tracking and SQL queries
Automated setup of Apache Iceberg on Amazon S3 using Terraform and AWS Glue Data Catalog. Explore the power of a Lakehouse architecture for data management and analysis, featuring schema discovery, metadata management, and efficient querying with Amazon Athena.
Data Lakehouse moderno com MinIO, DuckDB, dbt, metabase e airflow
Real-time fraud detection lakehouse with Kafka, medallion pipelines, data quality, explainable scoring, and dashboards.
A comprehensive educational resource hub dedicated to mastering Microsoft Fabric, offering in-depth tutorials, real-world use cases, and hands-on guides for seamless end-to-end analytics