#Data-processing-pipelines

NVIDIA-NeMo

Curator

Scalable data pre processing and curation toolkit for LLMs

graphbookai

graphbook

Visual AI development framework for training and inference of ML models, scaling pipelines, and automating workflows with Python

westandskif

convtools

convtools is a specialized Python library for dynamic, declarative data transformations with automatic code generation

edrewitz

WxData

A Python package of end-to-end weather data clients & raw data clients with VPN/PROXY support, data processors that decode variable keys from GRIB format into a plain-language format & various tools for assisting Python automated workflows, querying meteorological datasets and filling gaps in meteorological data.