firecrawl
Web Scraping
The index of self-hostable projects — ranked, categorized and served with live updates.
Own your data and stop giving it to big tech.
Self-hosted web scraping software runs the crawling and extraction on your own machines instead of a third-party service. More about Web Scraping
Web Scraping
Web Scraping
Browser Automation
Automation
AI Chat
Archiving & Preservation
Web Scraping
Browser Automation
Web Scraping
Proxy
Web Scraping
RSS
Web Scraping
Web Scraping
Search Engines
Web Scraping
Personal Productivity
Web Scraping
Web Scraping
Web Scraping
Search Engines
Monitoring & observability
Web Scraping
Browser Automation
AI Agents
RSS
Web Scraping
Proxy
You control where the requests go, what data is stored, and how the output is used. Crawl4AI is an open-source web crawler and scraper for LLMs and AI agents that converts any website into clean, LLM-ready Markdown, and it can be run yourself. changedetection.io focuses on monitoring: it tracks website changes, price drops, restock alerts and content updates. Maxun turns any website into a structured API so you can extract, automate, search and monitor the web. Browserless lets you deploy headless browsers in Docker on your own infrastructure.
33 self-hosted Web Scraping projects are listed here, ranked by popularity, maintenance and documentation.
| Project | License | Language | Stars | Docker | Setup | Latest release | Score |
|---|---|---|---|---|---|---|---|
| firecrawl | AGPL-3.0 | Typescript | 199 668 | Yes | Advanced | v2.11.0 | 91 |
| Crawl4AI | Apache-2.0 | Python | 87 085 | Yes | Moderate | v0.9.5 | 87 |
| Lightpanda | AGPL-3.0 | Zig | 41 589 | Yes | Easy | 1.0.3 | 84 |
| Huginn | MIT | Ruby | 50 410 | Yes | Moderate | v2026.11.05.2 | 82 |
| gpt-researcher | Apache-2.0 | Python | 30 560 | Yes | Moderate | v3.7.0 | 80 |
| ArchiveBox | MIT | Python | 28 974 | Yes | Easy | v0.9.77 | 79 |
| Obscura | Apache-2.0 | Rust | 30 543 | Yes | Easy | v0.2.6 | 76 |
| camofox-browser | MIT | JavaScript | 12 201 | — | Easy | v1.18.1 | 74 |
| RSS-Bridge | Unlicense | PHP | 9 338 | Yes | Moderate | 2025-08-05 | 73 |
| maxun | AGPL-3.0 | Typescript | 18 168 | Yes | Moderate | v0.1.0 | 73 |
| flaresolverr | MIT | Python | 16 162 | Yes | Easy | v3.5.2 | 73 |
| browserless | Other | Typescript | 13 915 | Yes | Easy | v2.60.0 | 73 |