Best Firecrawl Alternatives Wey You Fit Host for VPS

Wetin self-hosted Firecrawl alternative suppose do

Self-hosted Firecrawl alternative get one main work: make e take URL and return the page as clean markdown wey agent fit read. Hosted APIs dey charge for each page, so bill go increase as your agent dey curious. VPS wey you already dey pay for fit do the same work. The projects differ for one question: headless browser (real browser engine wey dey run without window) need start for your machine?

This answer determine memory footprint, cost for each page, and which pages go return empty. This guide compare Draco, Hound and self-hosted Firecrawl release. E install the lightest one for pinned version, then connect am to agent through MCP (model context protocol).

Di four projects, and wetin each one really be

Draco na one binary wey dem write for Rust, and MIT or Apache-2.0 license cover am. Dem publish Release v0.20.5 on 16 July 2026. draco scrape <url> dey print markdown go stdout. draco serve dey run daemon wey dey answer for 127.0.0.1:3002, na the port Firecrawl dey use. E no ship any container image, and e no start browser.

Firecrawl self-hosted na the engine behind the hosted product, and AGPL-3.0 cover am. E docker-compose.yaml define seven services: playwright-service, api, redis, rabbitmq, nuq-postgres, foundationdb and foundationdb-init. You go get the real crawl queue, but you go need run small distributed system.

Hound dey inside master-fetch repository, and e dey ship go PyPI as hound-mcp. MIT license cover am, and version na 13.0.1 as of 3 August 2026. E need Python 3.11 or newer. E first be MCP server, then fetcher: e dey try plain HTTP, and e start Patchright browser only when plain fetch return blocked.

Trawl dey here because people dey meet am when dem dey search for the others, and e dey do different work. E dey solve JavaScript challenges and CAPTCHAs with fingerprint-patched Firefox, as replacement for FlareSolverr for an *arr media stack. E no be markdown extractor. The section about etiquette below explain why this difference decide whether e fit enter your agent stack at all.

Browser pool na where small VPS box dey die

Each browser tab wey dey open na separate renderer process wey dey hold its own DOM (document object model) and its own JavaScript heap. So memory dey increase based on pages wey dey open at the same time, no be pages wey system fetch per day. Two of these projects write this cost for their own compose files.

[
  {
    "label": "Firecrawl api",
    "memory_limit_gb": 8
  },
  {
    "label": "Firecrawl playwright",
    "memory_limit_gb": 4
  },
  {
    "label": "Hound (browser included)",
    "memory_limit_gb": 3
  }
]

Firecrawl compose file limit its api container to 8 GB and its Playwright container to 4 GB, with matching swap limits. Hound compose set 3 GB for one container wey carry bundled Chromium. These na limits wey the projects choose and publish; dem no be measurements from quiet system. Redis, RabbitMQ, PostgreSQL and FoundationDB still need their own share on top of the Firecrawl figures.

If limit pass the RAM wey you get, e no solve anything. When the box run out of memory, kernel out-of-memory killer go end one process, so container go disappear from docker compose ps without any error for application log. Read dmesg -T | tail after any restart wey you no fit explain. Reserve 8 GB for the complete Firecrawl stack, and treat 4 GB as the minimum for test box. Setting these numbers per service dey explained for memory limits for Docker Compose.

One more browser detail fit make person spend whole evening. Docker dey give container 64 MB shared memory for /dev/shm, and Chromium dey put renderer buffers there, so e fit crash for heavy pages. Both browser stacks increase this value: Hound compose get shm_size: "1gb". Copy that line enter any image wey you build around Playwright.

Quality wey extraction dey get for JavaScript-heavy pages

For static HTML, server-rendered blog, documentation page, or news article, all of dem dey return almost the same markdown, so the fastest one win. Difference dey show for client-rendered pages, where delivered HTML na empty shell and JavaScript bring the text after page load.

Draco dey escalate by tiers. Tier 0 and tier 1 parse HTML without any JavaScript. Tier 2 run the page own JavaScript inside an in-process V8 isolate. Na JavaScript engine be this, but browser no dey around am. README talk say page code no get host capability bindings for there. This fit cover plenty single-page applications while e use only small part of browser memory. When Draco meet wall wey e no fit pass, draco scrape dey exit with code 3, needs_browser. Check am for scripts, because empty file wey get zero exit code na the failure wey dey quietly spoil agent context:

draco scrape https://example.com > page.md
echo "exit=$?"

Firecrawl's playwright-service dey control real Chromium, so e render wetin browser dey render. But self-hosted build still no be the hosted product. Documentation talk say self-hosted instances no get access to Fire Engine. So anti-blocking and IP rotation wey cloud service get no dey available, and /agent plus /browser endpoints no dey supported. Hound dey sit for middle by design. E fetch over HTTP and escalate for each request. E warm browser dey close after idle timeout, so quiet machine dey stay near baseline.

Install Draco with version wey you pin

README dey document one-line installer. Read wetin e dey do before you pipe am go shell: e install to $HOME/.draco/bin/draco, e always collect latest release, and e no check any signature or hash. For server, pin the version and verify the download.

cd /tmp
curl -fsSLO https://github.com/0xchasercat/draco/releases/download/v0.20.5/draco-linux-x86-64.tar.gz
curl -fsSLO https://github.com/0xchasercat/draco/releases/download/v0.20.5/SHA256SUMS
sha256sum --ignore-missing -c SHA256SUMS

E go print draco-linux-x86-64.tar.gz: OK. FAILED line mean say the bytes wey you get no be the same bytes wey project publish, so delete dem and start again.

mkdir -p draco-v0.20.5
tar -xzf draco-linux-x86-64.tar.gz -C draco-v0.20.5
sudo install -m 755 "$(find draco-v0.20.5 -type f -name draco | head -n1)" /usr/local/bin/draco
draco scrape https://example.com

The last command go print the example page as markdown well under one second. find no be decoration: archive layout no dey inside project public contract, and official installer dey locate the binary the same way.

Run the daemon under im own account instead of your login user. Write /etc/systemd/system/draco.service:

[Unit]
Description=Draco fetch daemon
After=network-online.target
Wants=network-online.target

[Service]
User=draco
ExecStart=/usr/local/bin/draco serve --host 127.0.0.1 --port 3002 --max-concurrency 4
Restart=on-failure
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
PrivateTmp=true

[Install]
WantedBy=multi-user.target

sudo useradd --system --no-create-home --shell /usr/sbin/nologin draco
sudo systemctl daemon-reload
sudo systemctl enable --now draco
curl -s http://127.0.0.1:3002/health

/health go answer as soon as daemon dey listen. Connection refused mean say e no dey listen, so read journalctl -u draco -n 50. The usual cause na another process wey already dey hold 3002, because na Firecrawl default port too, and --port go move either one. Unit files in more depth: systemd service units and timers.

Now fetch am the way your agent go:

curl -X POST http://127.0.0.1:3002/v1/scrape \
  -H 'content-type: application/json' \
  -d '{"url": "https://example.com", "formats": ["markdown"]}'

Keep fetch daemon off public internet

Fetch API wey no get authentication na open proxy. Anybody wey fit reach the port fit make your server request any URL under your IP address, and abuse report go go your provider, no be the person wey cause am. Draco documented serve flags include no API key, so na network go provide the protection. Keep the default 127.0.0.1 bind when agent dey run for the same box. When agent dey elsewhere, put both ends for private tunnel. WireGuard VPN wey you host yourself na the usual answer. Bind to the tunnel address instead of 0.0.0.0. Then check from another machine say public IP no dey answer anything. ufw firewall basics and least privilege user accounts cover the two sides of that.

Your agent code go change? API compatibility for real life

Draco dey answer Firecrawl v1 routes: /v1/scrape, /v1/map, /v1/crawl, /v1/batch/scrape and /v1/search, and e README talk say e dey accept unknown fields and ignore dem. Agent wey already dey send request to /v1/scrape need new base URL only; nothing else. Pay attention to wetin change for the other side: Firecrawl own self-hosting page now dey test with /v2/crawl, and current SDKs dey use v2. So, if you point v2 client to Draco, e go request route wey Draco no publish. Test every call with curl before you edit agent code, and read the JSON body instead of only the status code, because na field names dey differ between these implementations.

Robots.txt, rate limits, and the boundary wey you no suppose cross

Draco dey read robots.txt by default, and --ignore-robots dey off am. Firecrawl document the same default. Leave both as dem be. Then set your own speed: --delay dey put milliseconds between requests, while --max-concurrency dey limit parallel jobs, with 8 as the daemon default. Two to four dey gentler on shared VPS link and e rarely slow things down overall, because if site start rate limiting you, the extra minutes go pass the concurrency wey you save. Cache wetin you fetch, so second agent run no cost the source anything. That one also na the cheapest line item for controlling wetin AI agent dey cost you.

Challenge walls na another matter, and Trawl na specifically for this work: Cloudflare Turnstile, reCAPTCHA, hCaptcha and GeeTest. Challenge wall mean say site dey plainly reject automated traffic. If you work around am, you dey go against the site's terms of use, and for some places, against the law too, so this guide only cover fetching infrastructure and stop there. The same techniques wey fit pass wall na the ones site owners dey monitor and block, so any pipeline wey depend on dem go fragile and rude. When source matter reach that level, look for its RSS feed, public API, or bulk export. Each one cheaper to run, and none go break the week the wall change.

Wire am go agent through MCP

MCP (model context protocol) na the interface wey agent dey use to call tool. Draco carry MCP server inside the same binary, through stdio:

{ "mcpServers": { "draco": { "command": "draco", "args": ["mcp"] } } }

The tools go then show for agent as draco_scrape, draco_search and the draco_interact_* set. Stdio only work when agent process and binary dey for the same machine, because transport na that process standard input. For agent wey dey another host, Hound serve MCP through HTTP instead: hound --http --host 127.0.0.1 --port 8765 publish endpoint for http://127.0.0.1:8765/mcp, wey you fit reach through the tunnel. Transport choices and wetin to expose dey for running MCP servers on a VPS.

Fetching pairs with search. Agent wey fit only fetch go wait make you provide URLs. Add a self-hosted SearXNG search instance and e fit find dem by itself, for the same pattern as the browser search skill built on SearXNG. Once the daemon don start, na one shared service for any of the self-hosted AI agents wey you run.

FAQ

I need headless browser to fetch pages for AI agent?

For most pages, no. Documentation, blogs and news articles wey server dey render dey come back complete with ordinary HTTP fetch plus HTML-to-markdown step. Na this Draco dey use for im lower tiers, around browser-free 300 ms per page according to the project figures. Browser dey useful for client-rendered applications, where the HTML wey server deliver na empty shell. Draco V8 isolate fit handle plenty of this middle ground without browser process. E dey exit with code 3, needs_browser, when e no fit.

How much RAM self-hosted Firecrawl need for VPS?

Im compose file set 8 GB limit for the api container and 4 GB for the Playwright container. The same stack still start Redis, RabbitMQ, PostgreSQL and FoundationDB. Plan for 8 GB. For 2 GB machine, kernel out-of-memory killer go remove containers when load high. The first sign fit be restarted container for docker compose ps, with no useful information for application log. Confirm am with dmesg -T | tail.

Draco na drop-in replacement for Firecrawl API?

For the v1 endpoints, e dey close. E serve /v1/scrape, /v1/map, /v1/crawl, /v1/batch/scrape and /v1/search. E ignore request fields wey e no know, so client wey dem write against Firecrawl v1 usually only need new base URL. E no be the hosted product. No managed proxy pool dey behind am, and Firecrawl newer v2 routes no dey part of the available interface. First verify every call wey your agent dey make with curl.

If I self-host scraper, I fit ignore robots.txt?

No. Where the code dey run no change wetin site publish or wetin its terms allow. Draco and Firecrawl both respect robots.txt by default. The override flag dey for sites wey you own or where you get written permission to crawl. Rate limits still dey enforced for the other end, so polite --delay with low concurrency go help your IP address keep working. If stack only work by defeating challenge wall, e go break without warning.