#Elt-pipeline
Showing 6 of 6 repositories tagged #elt-pipeline, ranked by stars
ETL / ELT / Reverse ETL Framework powered by DuckDB, designed to seamlessly integrate and process data from diverse sources. It leverages Markdown as a configuration medium, where YAML blocks define metadata for each data source, and embedded SQL blocks specify the extraction, transformation, and loading logic.
A Pub/Sub for Tables based data integration platform, to discover, publish, modify and consume data effortlessly.
An end-to-end data pipeline which extracts divvy bikeshare data from web loads it into data lake and datawarehouse transforms it using dbt and finally , a dashboard to visualize the data using looker studio, the pipeline is orchestrated using prefect
Frontend & BFF (Backend for frontend) for Olake. This includes the UI code and backend code for storing the configuration of sync and orchestrating it.
A Python package to load complex XML files into a relational database
End-to-end ELT batch pipeline | Medallion Architecture (Bronze → Silver → Gold) | dbt + Snowflake + AWS S3 + Apache Airflow + Docker | 3.5M+ records