#Data-extraction
Showing 53 of 53 repositories tagged #data-extraction, ranked by stars
The API to search, scrape, and interact with the web at scale. 🔥
🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!
Python scraper based on AI
🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes 🔥
Declarative data automation language and Go runtime for structured extraction workflows.
Extract Keywords from sentence or Replace keywords in sentences.
Browser automation CLI built for AI agents. Break through anti-bot walls, hand off to humans across platforms when stuck. Parallel multi-task execution, independent multi-session operation, isolated multi-account browsing.
A powerful Model Context Protocol (MCP) server that provides an all-in-one solution for public web access.
The undetected self-hosted browser automation platform. Powered by Camoufox (Firefox) for 0% detection rates. Built for speed, privacy, and scalability.
ContextGem: Effortless LLM extraction from documents
Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.
:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
A beginner-friendly yet powerful Python toolkit for financial analysis and automation — built to make modern investing accessible to everyone
Lightweight library for scraping web-sites with LLMs
Local-first, open-source AI assistant for your data. Unify tasks, notes, docs, photos, and bookmarks. Private, self-hosted, and extensible via APIs.
The fastest PDF library for Python and Rust. Text extraction, image extraction, markdown conversion, PDF creation & editing. 0.8ms mean, 5× faster than industry leaders, 100% pass rate on 3,830 PDFs. MIT/Apache-2.0.
:newspaper: Let ChatGPT Summarize Hacker News for You
🤖 AI-powered web scraping editor with visual workflow builder. Build, test & deploy web scrapers using natural language. Powered by ScrapeGraphAI & LangGraph.
Open source web infrastructure for AI. Scrape, crawl, and automate the web, clean markdown, browser sessions, ready for your agents.
Pure Python, lightweight, Pillow-based solver for Amazon's text captcha.
Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.
A tool for scraping emails, social media accounts, and much more information from websites using Google Search Results.
Pure Rust PDF library for AI/RAG: structure-aware chunking, no ML, no C deps.
Apify SDK for Python—The official library for building Apify Actors: serverless cloud programs for web scraping, browser automation, data processing, and AI agents. Manages the Actor lifecycle, storages (datasets, key-value stores, request queues), events, proxies, and pay-per-event monetization. Built on top of the the Apify API Client.
PDFStract - Extract, Chunking and Embedding Layer in Your RAG Pipeline - Available as CLI - WEBUI - API
Data processing and modelling framework for automating tasks (incl. Python & SQL transformations).
wxpath - declarative web crawling with XPath; a Web Query Language (WQL)
DocWire SDK: Award-winning modern data processing in C++20. SourceForge Community Choice & Microsoft support. AI-driven processing. Supports nearly 100 data formats, including email boxes and OCR. Boost efficiency in text extraction, web data extraction, data mining, document analysis. Offline processing is possible for security and confidentiality
Claude Code-powered end-to-end meta-analysis automation: AI-assisted literature review, screening, extraction, analysis, and manuscript generation for systematic reviews and clinical evidence synthesis
A self-healing web scraper built for hostile sites: selectors repair themselves, browser rendering kicks in when needed, and Chrome TLS fingerprinting evades bot detection. Ships with an MCP server so any LLM can drive a full crawl through conversation.
Golang Keyword extraction/replacement Datastructure using Tries instead of regexes
OpenClaw plugin to search tweets, search replies, post tweets, export followers, manage media, monitor X/Twitter, and run giveaway draws via Xquik.
Powerful Google Maps Crawler / Scraper tool with REST API, Docker support & multi-format export
⚡ Build structured YouTube datasets at scale — effortlessly fetch transcripts and rich metadata for NLP, ML, and AI workflows.
PDF DataSource for Apache Spark, allow to read PDF files directly to the DataFrame and ocr it
Reduce HTML and XML to JSON from the command line, using an expressive query language inspired by CSS selectors.
Extract structured data from PDFs, Word docs and images. Embeddable directly into your application, regardless of the stack.
The fastest Office document library for Python, Rust, Go, JS/TS, C# and WASM. DOCX, XLSX, PPTX, DOC, XLS, PPT. Up to 100× faster than python-docx/openpyxl/python-pptx. 100% pass rate on valid Office files. MIT/Apache-2.0.
An EVM-compatible Solidity Smart Contract Storage/Slot Analyzer and Data Extractor.
Extract receipt info
AI-powered form filling and data extraction browser extension
A modern Python GUI application to batch extract YouTube comments and metadata to CSV. Features spam filtering, relevance sorting, and a dark-mode interface.
Batch-convert pdf to text, extract data from pdf in python
Data Engineering Project: Extracting music video metrics of Twice using YouTube API, AWS, and Tableau
The objective of this assignment is to extract textual data articles from the given URL and perform text analysis to compute variables that are explained
WildBerries Parser is a Python script that extracts item information from Wildberries.ru and saves it in an Excel file. It supports parsing by directory or search keyword, collecting data like link, ID, name, brand, pricing, rating, reviews, and sales. Ideal for market research and analysis. MIT licensed. Contributions welcome.
Real-time Google Search API for AI Agents & RAG pipelines. Get structured SERP data instantly using remote browsers.
Understand the relationships between various features in relation with the sale price of a house using exploratory data analysis and statistical analysis. Applied ML algorithms such as Multiple Linear Regression, Ridge Regression and Lasso Regression in combination with cross validation. Performed parameter tuning, compared the test scores and suggested a best model to predict the final sale price of a house. Seaborn is used to plot graphs and scikit learn package is used for statistical analysis.
Download all content from a Telegram channel
Extract Keywords from sentence or Replace keywords in sentences. @ https://github.com/vi3k6i5/flashtext
Moroccan housing data pipeline using scrapy, mongodb , zyte and digitalocean cloud