#Webscraping
Showing 60 of 115 repositories tagged #webscraping, ranked by stars
The API to search, scrape, and interact with the web at scale. 🔥
🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!
Create agents that monitor and act on your behalf. Your agents are standing by!
Python scraper based on AI
An autonomous agent that conducts deep research on any data using any LLM providers
Stealth Chromium that passes every bot detection test. Drop-in Playwright replacement with source-level fingerprint patches. 30/30 tests passed.
🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes 🔥
A cli tool to browse and play anime
📊 APIs for web automation, testing, and bypassing bot-detection.
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
Analysis of Bot Protection systems with available countermeasures 🚿. How to defeat anti-bot system 👻 and get around browser fingerprinting scripts 🕵️♂️ when scraping the web?
Self-hosted webscraper.
Undetected version of the Playwright testing and automation library.
A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama
Make your job hunt easy by automating your application process with this Auto Applier
Web Scraper in Go, similar to BeautifulSoup
LinkedIn enumeration tool to extract valid employee names from an organization through search engine scraping
Undetected Python version of the Playwright testing and automation library.
Lightweight library for scraping web-sites with LLMs
Watch everything from your terminal.
This code is used to perform web scraping and data extraction from Google Maps. It is particularly designed for obtaining information about businesses, including their name, address, website, phone number, reviews, and more.
Scalable Python web scraping scripts for +40 popular domains
Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.
Suck the InTernet
Make a ZIM file from any Web site and surf offline!
A python bot to automatically apply all Linkedin,Glassdoor, etc Easy Apply jobs based on your preferences. Auto login, auto fill additional questions, apply automatically!
A modular, terminal-based toolkit for OSINT, reconnaissance, and scraping - built in Python, runs on Linux and Windows.
Uscrapper Vanta: Dive deeper into the web with this powerful open-source tool. Extract valuable insights with ease and efficiency, from both surface and deep web sources. Empower your data mining and analysis with Vanta's advanced capabilities. Fast, reliable, and user-friendly, Uscrapper Vanta is the ultimate choice for researchers and analysts.
🔖 Web & Electron based Anime Streaming App for 🐈s
Undetected NodeJS version of the Playwright testing and automation library.
🤖 AI-powered web scraping editor with visual workflow builder. Build, test & deploy web scrapers using natural language. Powered by ScrapeGraphAI & LangGraph.
Powerful and flexible Instagram scraping library for Python, providing easy-to-use and expressive tools for accessing data programmatically
Extract price and indicator data from TradingView charts to create ML datasets
🗽 A Simple Demonstration of the New York Times App 📱 using Jsoup web crawler with MVVM Architecture 🔥
Take the hassle out of web scraping
Learn everything web scraping with David Teather Codes on YouTube
Free, open-source anti-detect browser launcher for web scraping and multi-accounting. By the ProxyShard team. Engine-level fingerprint spoofing in Chromium 148 (WebGL / WebGPU / Client Hints / fonts / TLS), 170+ device profiles bundled, stable QUIC + WebRTC over SOCKS5.
a class that uses scraped proxies to make http GET/POST requests (Python requests)
Crawl telegra.ph searching for nudes!
A streaming app with no ADs.
Use LLMs to robustly extract web data
This AI Smart Speaker uses speech recognition, TTS (text-to-speech), and STT (speech-to-text) to enable voice and vision-driven conversations, with additional web search capabilities via OpenAI and Langchain agents.
📚 This is an adapted version of Jina AI's Reader for local deployment using Docker. Convert any URL to an LLM-friendly input with a simple prefix http://127.0.0.1:3000/https://website-to-scrape.com/
A project to deploy an online app that predicts the win probability for each NBA game every day. Demonstrates end-to-end Machine Learning deployment.
Turn Webpage to LLM friendly input text. Similar to Firecrawl and Jina Reader API. Makes RAG, AI web scraping, image & webpage links extraction easy.
Extract public Spotify data — tracks, albums, artists, playlists, podcasts & lyrics — without the official API. Sync + async, typed models, one dependency.
AniWorld Downloader is a cross-platform tool for streaming and downloading anime from aniworld.to, as well as series from s.to. It runs on Windows, macOS, and Linux, providing a seamless experience for offline viewing or instant playback. If you enjoy using it, feel free to leave a ⭐!
Lego AI Parser is an open-source application that uses OpenAI to parse visible text of HTML elements.
Perform Google Dork search with Dorkify
A low-code data extractor for websites with built in proxy and parsing capabilities. Great for testing and debugging css selectors
A tool for scraping emails, social media accounts, and much more information from websites using Google Search Results.
DPULSE - Tool for complex approach to domain OSINT
A next-generation GUI automation framework for Web and Desktop Application Testing and Automation.
Android Automation Framework for Python on emulators (BlissOs, BlueStacks, LDPlayer, Memu, Mumu, Android Studio ...) and rooted devices WITHOUT ADB!
Terminal-based PornHub browser inspired by ani-cli. Browse categories, search videos, and stream instantly in mpv without opening a browser.
Daily podcasts generated by AI 🤖
AI-native web scraper. Single binary with a bundled Claude Code skill. MIT-licensed alternative to Firecrawl.
Automatic sync from IMDb to Trakt (watchlist, lists, ratings and history) using GitHub actions.
ani-cli but for adult videos
A project to automate the process of downloading images and other attachment files from platforms like Fantia and more!