#Web-scraper
Showing 36 of 36 repositories tagged #web-scraper, ranked by stars
The API to search, scrape, and interact with the web at scale. 🔥
🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!
Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
Python scraper based on AI
🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes 🔥
Self-hosted webscraper.
scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
A list of practical knowledge-building projects.
A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama
The process of extracting product data from Amazon using Python, including titles, ratings, prices, images, and descriptions.
Web Scraper in Go, similar to BeautifulSoup
Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.
CLI tool for saving a faithful copy of a complete web page in a single HTML file (based on SingleFile)
Faster requests on Python 3
An ergonomic, censorship-resistant Rust HTTP Client
100% free and full open-source edge Firecrawl alternative with better links extraction for agents - that you can deploy to cloudflare or vercel by yourself.
A multithreaded 🕸️ web crawler that recursively crawls a website and creates a 🔽 markdown file for each page, designed for LLM RAG
A desktop app for tracking and batch downloading anime
Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.
A Reddit bot that summarizes news articles written in Spanish or English. It uses a custom built algorithm to rank words and sentences.
Nástroj pro hlídání nových nabídek nemovitostí na populárních realitních serverech. Nabídky jsou vypisovány do Discord roomky.
A self-healing web scraper built for hostile sites: selectors repair themselves, browser rendering kicks in when needed, and Chrome TLS fingerprinting evades bot detection. Ships with an MCP server so any LLM can drive a full crawl through conversation.
A Router WiFi key recovery/cracking tool with a twist.
Go web crawler to scrape documentation sites and convert content to clean Markdown for LLM ingestion (RAG, training data).
Common utilities for wreq
Generate thousands of temporary emails within seconds!
Simple webscraper for Craigslist.
Web scraper, written in Elixir, that pulls cybersecurity job postings from public hiring APIs, isolates the genuine cybersecurity roles, and reports which certifications employers ask for most. 💜
Web Scraping using Python Data mining , Data Analyzing & Data Visualization of the collected Data, The python script is written to fetch all the individual categories the website , The code is written for fetching the data from the first page and it iterates to each and every pages of website ( activities, categories, count of bought), and I used statistical techniques for mathematically analysis and presenting the data into visualization
Developing a long/short equity investment portfolio with Machine Learning predictions using data acquired from web-scraping. Flatiron Module 5 Project.
AI-powered web scraping agent built with LangGraph, LangSmith, Firecrawl, and Anthropic AI. Automates intelligent crawling, structured data extraction, and LLM-powered content formatting. Efficiently handles anti-bot mechanisms, error recovery, and batch processing. 🚀
Convert the public web into AI-ready Markdown with a local Python CLI/SDK/MCP crawler.
Data ETL & Analysis on thousands of job listings from the official Mexican job board (2020 edition).
Self-hosted multi-agent AI platform, orchestrate specialized agents across Telegram, WhatsApp, and web with 90+ tools, 15 autonomous scrapers, vector memory, and real-time market streaming
The Lat-Epig interface allows you to query the EDCS and save the search result in a TSV file and plot the results on a map of the Roman Empire without any prior knowledge of programming.
Web Scraping and EDA from iFood website data.