#Webscraper
Showing 28 of 28 repositories tagged #webscraper, ranked by stars
Self-hosted webscraper.
AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts structured SERP results from Google/Bing/Baidu/etc. Native multi-threading for bulk processing.
Web Scraper in Go, similar to BeautifulSoup
Scalable Python web scraping scripts for +40 popular domains
Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.
a class that uses scraped proxies to make http GET/POST requests (Python requests)
An AI assistant tool that integrates coding, writing, and reading functions. For better alternatives see https://monica.im/desktop
📚 This is an adapted version of Jina AI's Reader for local deployment using Docker. Convert any URL to an LLM-friendly input with a simple prefix http://127.0.0.1:3000/https://website-to-scrape.com/
Python class to scrape data from rightmove.co.uk and return listings in a pandas DataFrame object
Extract public Spotify data — tracks, albums, artists, playlists, podcasts & lyrics — without the official API. Sync + async, typed models, one dependency.
Lego AI Parser is an open-source application that uses OpenAI to parse visible text of HTML elements.
Financial Web Scraper & Sentiment Classifier
A lightweight API for Pornhub
A tkinter GUI collating various data
A command line tool for checking your website for GDPR compliance.
:earth_americas: :hotel: Scrape Booking.com :hotel: :earth_americas:
This Python script will download chapters from novels availaible on wuxiaworld.com saves then into the .epub format
Osint tool to scraper websites
Finds Jobs on LinkedIn using web-scraping
Automatically scrape the web data of people profiles on Linkedin based on a specific search query
It uses selenium to automate Whatsapp for various different functionality like SMS bombing , Simultaneously sending multiple user's same messages, profile opening, status view and more, .
Library packs for google cloud functions
Scraper for Trading Economics
cryptopanic news feed scraper
The spider crawls moneycontrol.com and economictimes.com to fetch news of input companies and also scores and classifies the companies to raise an early warning signal
Scrap BTC addresses and balances from bitkeys.work
A python package to extract historical market data of cryptocurrencies and to calculate technical price indicators.
UNMAINTAINED | R-package providing access to fundamental data and valuation metrics for thousands of publicly traded companies worldwide.