#Dataops
Showing 47 of 47 repositories tagged #dataops, ranked by stars
Open Lakehouse Format for Multimodal AI. Convert from Parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..
Redpanda Console is a developer-friendly UI for managing your Kafka/Redpanda workloads. Console gives you a simple, interactive approach for gaining visibility into your topics, masking data, managing consumer groups, and exploring real-time data with time-travel debugging.
Scalable and efficient data transformation framework - backwards compatible with dbt.
An open-source data logging library for machine learning models and data pipelines. π Provides visibility into data quality & model performance over time. π‘οΈ Supports privacy-preserving data collection, ensuring safety & robustness. π
Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
The dbt-native data observability solution for data & analytics engineers. Monitor your data pipelines in minutes. Available as self-hosted or cloud service with premium features.
Kafka Docker for development. Kafka, Zookeeper, Schema Registry, Kafka-Connect, , 20+ connectors
Cloud Native DataOps & AIOps Platform | δΊεηζ°ζΊθΏη»΄εΉ³ε°
π Awesome Data Catalogs and Observability Platforms.
Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.
Tenzir is the data pipeline engine for security teams.
DataOps for Microsoft Data Platform technologies. https://aka.ms/dataops-repo
A list of tools for annotating data, managing annotations, etc.
Engine for AI/ML/Data tracking, visualization, explainability, drift detection, and dashboards for Polyaxon.
One framework to develop, deploy and operate data workflows with Python and SQL.
Titan Core - Snowflake infrastructure-as-code. Provision environments, automate deploys, CI/CD. Manage RBAC, users, roles, and data access. Declarative Python Resource API. Change Management tool for the Snowflake data warehouse.
The data-validation toolkit for enhanced dbt (data build tool) PR review
Power BI DevOps & Source Control Tool
:sunglasses: A curated list of awesome DataOps tools
Resources for video demonstrations and blog posts related to DataOps on AWS
Open Source Data Quality Monitoring.
Transform your pythonic research to an artifact that engineers can deploy easily.
A curated list of awesome open source tools and commercial products for ML Experiment Tracking and Management π
β€for real-time DataOps - where the application and data fabric blends - Lenses
Installer for DataKitchen's Open Source Data Observability Products. Data breaks. Servers break. Your toolchain breaks. Ensure your team is the first to know and the first to solve with visibility across and down your data estate. Save time with simple, fast data quality test generation and execution. Trust your data, tools, and systems end to end.
A curated list of awesome open source tools and commercial products for monitoring data quality, monitoring model performance, and profiling data π
Beneath is a serverless real-time data platform β‘οΈ
Interactive computing for complex data processing, modeling and analysis in Python 3
DataOps Data Quality TestGen is part of DataKitchen's Open Source Data Observability. DataOps TestGen delivers simple, fast data quality test generation and execution by data profiling, Β new dataset hygiene review, AI generation of data quality validation tests, ongoing testing of data refreshes, & continuous anomaly monitoring
In-deprecation. For Lenses please check lensesio/lenses-helm-charts. Soon Stream Reactor will also get its own Helm repository.
Duke MIDS: Data Engineering and DataOps Course
End-to-end DataOps platform deployed by Terraform.
DataOps framework for Machine Learning projects.
DataOps Observability is part of DataKitchen's Open Source Data Observability. DataOps Observability monitors every data journey from data source to customer value, from any team development environment into production, across every tool, team, environment, and customer so that problems are detected, localized, and understood immediately.
A curated list of awesome open source and commercial platforms for serving models in production π
Build, Test and Deploy ETL solutions using AWS Glue and AWS CDK based CI/CD pipelines
A curated list of awesome open source tools and commercial products to catalog, version, and manage data π
Love your Data. Love the Environment. Love VULKΠ.
This repository is home to the original helm charts for products throughout the Raystack ecosystem.
DevSquad In a Day - DataOps. A hands-on lab for exercising DataOps practices on Azure.
Never sift through endless dbtβ’ logs again. dbt Command Center is a free, open-source, local web application that provides a user-friendly interface to monitor and manage dbt runs.
A framework for rapid development of robust data pipelines following a simple design pattern
Awesome list of dataops products, open source and resources
Your own personal data engineer for OpenClaw.
Kedro plugin to support running pipelines on Dagster
Collection of useful Helm Charts. Well test with KinD and Kubeconform
