#Data-collection
Showing 33 of 33 repositories tagged #data-collection, ranked by stars
A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/网页爬虫软件，可以无代码图形化的设计和执行爬虫任务。别名：ServiceWrapper面向Web应用的智能化服务封装系统。
Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
This GitHub repo is a powerhouse collection of APIs you can start using immediately to build everything from simple automations to full-scale applications. One of the most valuable API lists on GitHub—period. 💪
Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.
Jitsu is an open-source Segment alternative. Fully-scriptable data ingestion engine for modern data teams. Set-up a real-time data pipeline in minutes, not days
A powerful Model Context Protocol (MCP) server that provides an all-in-one solution for public web access.
Concurrent Python made simple
Player Analytics plugin for Minecraft Server platforms - View player activity of your server with ease. :calendar:
🌊 Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, by managing your data flows with Estuary. 🌊
ODK Collect is an Android app for filling out forms. It's been used to collect billions of data points in challenging environments around the world. Contribute and make the world a better place! ✨📋✨
IoT DC3 — open-source, cloud-native industrial IoT platform. Multi-protocol device access, data collection & management with 28 drivers, Spring Cloud, gRPC, and native LLM integration.
99+ CLI tools to build, browse, and blend your media library
How to disable Firefox Telemetry and Data Collection
📱🌐📋 wq: a modular framework supporting web / native apps for mobile surveys and geospatial data collection. Powered by Django REST Framework, Redux, React, and Material UI.
CHEK EGO Miner by Qingkong Technology: use CHEK mobile and Qingkong Miker desktop to collect first-person EGO data for embodied AI.
💻📱 wq's app library: a JavaScript framework powering offline-first web & native apps for geospatial data collection, mobile surveys, and citizen science. Powered by Redux, React, Material UI and Maplibre GL.
Official Oxylabs MCP integration
A Client for the Radio Browser API
Goblyn is a Python tool focused to enumeration and capture of website files metadata.
Cryptocurrencies price data collection, price tickers, signals notifications, charts, Telegram bot and more.
Development of AI audio app to compare the cough of a Coronavirus (COVID-19) infected individual with the cough of an uninfected individual.
Open, extensible Python injector/profiler/analyzer
Python program for automated electron diffraction data collection
A Content Discovery and Development Platform. Empowering Cybersecurity, AI, Marketing, and Finance professionals and researchers to discover, analyze, and interact with the web in all its dimensions.
Index any EVM chain and query in SQL
Grabs all your Perplexity conversations data, spits it out into a nice file folder structure and allows you to find your old conversations again.
Local-first Chrome extension for collecting, organizing, and exporting X/Twitter posts for research, AI workflows, and personal knowledge management.
rtdl makes it easy to build and maintain a real-time data lake
知乎内容爬虫 | Web scraper for Zhihu content extraction
OpenSnowcat Enricher (Apache 2.0 License)
Zomato Data Exploration and Analysis with SQL (SQL SERVER)
An IoT-based water quality monitoring and notification system for fish farmers to determine the physio-chemical parameters of aqua-cultured sites such as fish ponds. Built with esp32-arduino microcontroller, sensors (temperature, turbidity, pH, ultrasonic), Python Flask and SQLite Database). Literature work: https://drive.google.com/file/d/1_ctEz1cAZmnwtB1LCnVKh2_sHrVP6vUU/view
🩺 Machine Learning diabetes prediction model using Support Vector Machine (SVM) classifier. Analyzes 8 medical features (glucose, BMI, age, etc.) from Pima Indian dataset to predict diabetes risk with 75-80% accuracy. Built with Python, scikit-learn, pandas. Includes data preprocessing, model training, and prediction system for diabetes..