#Webcrawler
Showing 12 of 12 repositories tagged #webcrawler, ranked by stars
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
新闻网页正文通用抽取器 Beta 版.
HTTP API for Scrapy spiders
Uscrapper Vanta: Dive deeper into the web with this powerful open-source tool. Extract valuable insights with ease and efficiency, from both surface and deep web sources. Empower your data mining and analysis with Vanta's advanced capabilities. Fast, reliable, and user-friendly, Uscrapper Vanta is the ultimate choice for researchers and analysts.
《Python爬虫开发 从入门到实战》配套源代码。
AI-native web scraper. Single binary with a bundled Claude Code skill. MIT-licensed alternative to Firecrawl.
A Web Crawler based on LLMs implemented with Ray and Huggingface. The embeddings are saved into a vector database for fast clustering and retrieval. Use it for your RAG.
Stick to doing something interesting and valuable.
The data and code that used in my book.
Social Scraper is a python tool meant for Detection of Child Predators/Cyber Harassers on Social Media
Deep web crawler and search engine
Webanator: Webcam scraper written in python that will scrape any countries web cameras.