#Datalake
Showing 20 of 20 repositories tagged #datalake, ranked by stars
Chat with your database or your datalake (SQL, CSV, parquet). PandasAI makes data analysis conversational using LLMs and RAG.
Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.
lakeFS - Data version control for your data lake | Git for data
Scalable master data management, identity resolution, entity resolution, and deduplication using ML
A comprehensive guide to building a modern data warehouse with SQL Server, including ETL processes, data modeling, and analytics.
Continuously updated paper list on advancements in Data Agents. Companion repo to our paper "A Survey of Data Agents: Emerging Paradigm or Overstated Hype?"
A curated list of open source tools used in analytics platforms and data engineering ecosystem
Use SQL to build ELT pipelines on a data lakehouse.
A Data Platform built for AWS, powered by Kubernetes.
🧊 A curated list of OLAP databases, data lake tools, columnar engines, and analytics frameworks for data engineers.
Icebird: JavaScript Iceberg Client
This repository will help you to learn about databricks concept with the help of examples. It will include all the important topics which we need in our real life experience as a data engineer. We will be using pyspark & sparksql for the development. At the end of the course we also cover few case studies.
Skardi is an agent data plane that gives AI agents data autonomy.
Awesome list for datapipeline
A platform for extracting and shipping security value from your data lake to Sentinel.
The goal of this project is to illustrate Extract Transform Load (ETL) using Python and SQL. ETL is a process commonly done in computing, which takes raw data, cleans it and stores it for later use. The extraction phase targets and retrieves the data. Transform manipulates and cleans the data. Then load stores the data, typically in a data warehouse.
Code/Notes for the Data Engineering Zoomcamp by DataTalksClub
An Iceberg destination for DLT that supports REST catalogs
Building a poor man's data lake: Exploring the Power of Polars and Delta Lake