#Parquet
Showing 60 of 67 repositories tagged #parquet, ranked by stars
High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale
Commandline tool for running SQL queries against JSON, CSV, Excel, Parquet, and more.
Official Rust implementation of Apache Arrow
Create full-fledged APIs for slowly moving datasets without writing a single line of code.
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
The fastest business intelligence tool for humans and agents.
Parseable is an open source, unified infrastructure observability platform built in Rust on a data lake architecture. It tracks logs, metrics, traces, and events across apps, agents, and systems, reducing storage costs by up to 90% through columnar telemetry compression.
Real-time analytics on Postgres tables
Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.
One SQL interface for 60+ tools (e.g., GitHub, Notion, Airtable). Plug into any LLM through MCP.
Tonbo is an embedded database for serverless and edge runtimes.
cryo is the easiest way to extract blockchain data to parquet, csv, json, or python dataframes
Quilt is a Scientific Data Management Platform on AWS that helps teams and AI find, trust, and reuse data through deeply versioned, context-rich data packages.
Fast, interactive geospatial data visualization in Jupyter.
80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Functions, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.
High-performance Go package to read and write Parquet files
Graph Data Science: an abstraction layer in Python for building knowledge graphs, integrated with popular graph libraries – atop Pandas, NetworkX, RAPIDS, RDFlib, pySHACL, PyVis, morph-kgc, pslpython, pyarrow, etc.
Copy to/from Parquet in S3, Azure Blob Storage, Google Cloud Storage, http(s) stores, local files or standard inout stream from within PostgreSQL
Rust-based WebAssembly bindings to read and write Apache Parquet data
High-performance analytical database. 19.9M records/sec ingestion, 8.4M+ rows/sec queries. Ingestion, compaction, SQL, retention, continuous queries — one binary. Open Parquet on your storage. S3/Azure native. Air-gap ready. No vendor lock-in. AGPL-3.0.
Fast data store for Pandas time-series data
Data Preview 🈸 extension for importing 📤 viewing 🔎 slicing 🔪 dicing 🎲 charting 📊 & exporting 📥 large JSON array/config, YAML, Apache Arrow, Avro, Parquet & Excel data files
Unified MySQL, Postgres & FlightSQL Server, Powered by DuckDB.
select * from logs; Tailpipe is an open source SIEM for instant log insights, powered by DuckDB. Analyze millions of events in seconds, right from your terminal.
A fast viewer for CSV/Parquet files and databases such as DuckDB, SQLite, PostgreSQL, MySQL, Clickhouse, etc., base on Tauri
sql driver for CSV, TSV, LTSV, JSON, Parquet, Excel with gzip, bzip2, xz, zstd support.
Comprehensive Vector Data Tooling. The universal interface for all vector database, datasets and RAG platforms. Easily export, import, backup, re-embed (using any model) or access your vector data from any vector databases or repository.
A command line tool to query an ODBC data source and write the result into a parquet file.
Quick start: pip install jsoniq ⛈️ RumbleDB 2.1.0 "Cedrus Libani" 🌳 for Apache Spark | Run queries on your large-scale, messy datasets (JSON, text, CSV, Parquet, Delta...) | Data Lakehouse with Updates, Scripting, Declarative Machine Learning and more
Firebolt Core is a free, self-hosted edition of Firebolt's distributed query engine (https://www.firebolt.io/); it provides high-performance data warehousing capabilities that can be deployed anywhere from a single laptop to enterprise datacenters.
Edit database tables like Vim buffers. Staged mutations + live SQL preview, transaction undo, schema browser + ER diagrams + DDL, FK navigation, cross-database federation, data profiling, SQL notebooks, AI SQL via Anthropic/OpenAI/Gemini/Ollama, Parquet/CSV/remote files. PostgreSQL · SQLite · MySQL · DuckDB · MotherDuck
executes SQL against CSV, TSV, LTSV, JSON, JSONL, Parquet, Microsoft Excel™ , ACH, and Fedwire files with shell.
Icebird: JavaScript Iceberg Client
Export PostgreSQL table or query into Parquet file
Fast parquet command line tool with many functions, nailed it!
Fast psql-style database CLI/workbench for PostgreSQL, MySQL, SQLite, ClickHouse, MongoDB, Elasticsearch, and SQL over Parquet/CSV/JSON — with optional AI, SSH/Vault, Python & Django.
Open-source SQL workbench powered by DuckDB. Query local files, BigQuery, and Snowflake right in your browser. Drop-in UI for duckdb -ui. AI SQL assistant included.
Token-efficient data serialization for LLM/AI. 50% fewer tokens than JSON, 93% better value/token. Rust, schema validation, LSP.
Datos públicos de Chile curados, normalizados y validados: geografía, demografía, economía, salud, educación y distritos electorales. Listos para consumir en una línea de código con Polars, DuckDB, SQLite y Excel.
Local-first AI memory MCP server — query your ChatGPT, Claude Code, Cursor & Codex history from any LLM. DuckDB+parquet.
A fast, local-first web GUI for exploring large CSV/Parquet/JSONL files. Powered by VisiData’s engine. Opens millions of rows instantly. Works with local files, SSH servers, and Jupyter. Your new data debugging lens.
Common Lisp CFFI wrapper around the DuckDB C API
Not just an alternative to Pandas in Go. A next-generation data analysis library for Golang with factor analysis, Parquet, CSV, JSON, Excel, and seamless Python integration.
Parquet file reader and editor in Jupyterlab, built with `perspective` for pivoting, filtering, aggregating, etc
DGGS indexer for raster data
A local-first lakehouse reference architecture for production-minded data platform engineering.
Fast search engine on object storage, with full text search, vectors, and SQL, natively on Parquet.
Squey is an open-source cross-platform visualization software designed to interactively explore and understand large amounts of tabular data (this is the read-only mirror of https://gitlab.com/squey/squey)
Market data acquisition, storage, and update workflows for machine learning for trading.
Dataframe viewer for Jupyter over Arrow
A fast, keyboard-driven terminal explorer for tabular data. Open CSV, JSON, Parquet, Excel and SQLite files directly in your terminal — filter, sort, pivot, compute new columns, and visualise distributions without leaving the shell
A high-performance spatial query layer for Polars
A project for exploring how Great Expectations can be used to ensure data quality and validate batches within a data pipeline defined in Airflow.
AI-Lake Format: self-contained Parquet + HNSW index — unifying tabular data, embeddings, and vector search in a single Iceberg-compatible file
Cursor for Data — a local-first AI workspace that understands your datasets: discovers relationships, builds semantic models, and answers questions with DuckDB. CLI + browser app.
DBConvert Streams: Database IDE, Federated SQL, Real-time CDC & AI assistants via MCP — explore, query, and replicate data across databases and files
Big Data CONFERENCE Europe 2025 🇱🇹 AI, Cloud and Data Conference - Build Your First End-to-End Lakehouse Solution (aka.ms/fabconlake)
Example Jupyter notebooks hosted on Kaggle that demonstrate how to work with US energy data from PUDL.
Analyze and generate unstructured data using LLMs, from quick experiments to billion token jobs.