Self-host searcharvester
2.1.0Self-hosted search and markdown harvester for AI agents
- Stars
- 282
- Ports
- 8000 (adapter: /search, /extract, /research, /media), 9762 (web UI), 8011 (search-admin, used by the settings page)
- Requires
- SearXNG, Valkey (Redis)
ghcr.io/vakovalskii/searcharvester
What is searcharvester?
Searcharvester is a self-hosted stack that gives AI agents a Tavily-compatible search API backed by SearXNG with over 100 engines, a URL-to-markdown extractor using multiple extraction engines with size presets and pagination, and a deep research agent that plans, searches, reads, and synthesizes cited markdown reports. It includes a React web UI on port 9762 to start jobs, pick models per agent role, watch agents live, and browse sources and media. The stack runs with one docker compose up, pulling the adapter image from GHCR and building the UI and search-admin locally. No API keys or quotas for search; LLM credentials (any OpenAI-compatible endpoint) are only needed for the research feature. The adapter listens on 127.0.0.1:8000, SearXNG on 8999, search-admin on 8011, and the UI on 9762. Licensed AGPL-3.0 with MIT on project code.
Should you self-host searcharvester?
searcharvester scores 47/100 on selfhost.directory (how scoring works) and ranks #123 of 209 in AI Agents. It has 282 GitHub stars.
There was no stable release in the past 12 months, so check the repository's recent activity before you rely on it.
Setup is moderate: expect some configuration beyond a single command. An official Docker image is available. It is AGPL-3.0-licensed and written mostly in Python.
Its closest alternative on selfhost.directory is crw (58/100, 1 192 stars). It scores higher overall, so compare the two before you pick. See searcharvester vs crw side by side.
Features
- Citation tracking
- Deep Research Agent
- Image and video search
- Loop guard and budgets
- Model per agent role
- Multiple content extractors
- Optional Cloudflare WARP proxyOptional profile, may not work on all networks
- Search settings UI
- Size presets and pagination
- SSRF protection
- Tavily-compatible search API
- URL to markdown extraction
- Web UI
searcharvester README
More in AI Agents
self-hosted-ai-starter-kitThis open-source Docker Compose template spins up a self-hosted AI stack with n8n, Ollama, Qdrant and PostgreSQL, auto-pulling Llama3.2 for local workflows.
sempreSelf-hosted multi-agent AI system where a General Manager delegates tasks to worker agents with isolated tool stacks, shared knowledge base, and n8n automation.
Serena-MCPSelf-hosted, open-source MCP toolkit that gives AI coding agents IDE-level symbol retrieval, editing, refactoring and debugging across 40+ languages.
shaiOpen-source terminal coding agent written in Rust, with an interactive TUI, headless piping, an OpenAI-compatible HTTP server, MCP tools and skills.
silosSelf-hosted dashboard for OpenClaw AI agents with real-time monitoring, chat, skills, and omnichannel messaging via WhatsApp, Telegram, Discord, Slack.
SimOpen-source, self-hosted AI workspace for teams designing, launching and observing agent workflows, with 1,000+ app connectors, tables, knowledge bases and run logs.
SkalesSelf-hosted AI agent harness: background goals, coding, voice and browser automation on desktop plus native phone apps, with 27 providers or fully local inference.
skrunSkrun is a self-hosted agent runtime turning Agent Skills into POST /run APIs. Supports multiple LLMs, MCP, SSE, webhooks, dashboard. MIT, Node.js, SQLite/Postgres
skyfloDeprecated open-source AI agent that runs in your Kubernetes cluster to automate infrastructure via a deterministic approval-gated control loop.
skypilot-code-sandboxA self-hosted sandbox that lets LLM agents run code securely on your own cloud using SkyPilot, supporting multiple languages and MCP clients.
skyvernSelf-hosted, open-source AI browser automation: vision LLM agents run tasks and no-code workflows on any site, with Python/TypeScript SDKs and Docker Compose.
SoulFlow-OrchestratorSelf-hosted AI agent runtime with 9 LLM backends, 141-node workflow engine, HITL gates, AES-256-GCM vault, Slack/Telegram/Discord/web chat. All data on your server.
0 comments