Self-hosted Firecrawl alternatives for a VPS

What a self-hosted Firecrawl alternative must do

A self-hosted Firecrawl alternative has one job: take a URL and hand back the page as clean markdown an agent can read. The hosted APIs charge per page, so the bill grows with how curious your agent is, and a VPS you already pay for can do the same work. The projects split on one question: does a headless browser (a real browser engine running with no window) have to start on your box?

That answer decides the memory footprint, the cost of each page, and which pages come back empty. This guide compares Draco, Hound and the self-hosted Firecrawl release, installs the lightest one at a pinned version, and wires it to an agent over MCP (model context protocol).

The four projects, and what each one actually is

Draco is one binary, written in Rust, licensed MIT or Apache-2.0. Release v0.20.5 was published on 16 July 2026. draco scrape <url> prints markdown to stdout. draco serve runs a daemon that answers on 127.0.0.1:3002, the port Firecrawl uses. It ships no container image and starts no browser.

Firecrawl self-hosted is the engine behind the hosted product, under AGPL-3.0. Its docker-compose.yaml defines seven services: playwright-service, api, redis, rabbitmq, nuq-postgres, foundationdb and foundationdb-init. You get the real crawl queue, at the price of running a small distributed system.

Hound lives in the master-fetch repository and ships to PyPI as hound-mcp, MIT licensed, version 13.0.1 as of 3 August 2026. It needs Python 3.11 or newer. It is an MCP server first and a fetcher second: it tries plain HTTP, and starts a Patchright browser only when the plain fetch comes back blocked.

Trawl is here because people meet it while searching for the others, and it does a different job. It solves JavaScript challenges and CAPTCHAs with a fingerprint-patched Firefox, as a replacement for FlareSolverr in an *arr media stack. It is not a markdown extractor. The section on etiquette below explains why that distinction decides whether it belongs in your agent stack at all.

Why the browser pool is where small VPS boxes die

Each open browser tab is a separate renderer process holding its own DOM (document object model) and its own JavaScript heap. Memory therefore scales with pages open at the same moment, not with pages fetched per day. Two of these projects write that cost into their own compose files.

[
  {
    "label": "Firecrawl api",
    "memory_limit_gb": 8
  },
  {
    "label": "Firecrawl playwright",
    "memory_limit_gb": 4
  },
  {
    "label": "Hound (browser included)",
    "memory_limit_gb": 3
  }
]

The Firecrawl compose file caps its api container at 8 GB and its Playwright container at 4 GB, with matching swap limits. Hound's compose sets 3 GB for one container that carries a bundled Chromium. These are ceilings the projects chose, published figures rather than measurements of a quiet system, and Redis, RabbitMQ, PostgreSQL and FoundationDB still want their share on top of the Firecrawl numbers.

A ceiling above the RAM you own does nothing. When the box runs out, the kernel out-of-memory killer ends a process, so a container vanishes from docker compose ps with no error written in the application log. Read dmesg -T | tail after any restart you cannot explain. Budget 8 GB for the full Firecrawl stack and treat 4 GB as a test-box floor. Setting the numbers per service is covered in memory limits in Docker Compose.

One more browser detail costs people an evening. Docker gives a container 64 MB of shared memory at /dev/shm, and Chromium puts renderer buffers there, so it crashes on heavy pages. Both browser stacks raise it: Hound's compose carries shm_size: "1gb". Copy that line into any image you build around Playwright.

Extraction quality on JavaScript-heavy pages

On static HTML, a server-rendered blog, a documentation page, a news article, all of these return near-identical markdown and the fastest one wins. The difference appears on client-rendered pages, where the delivered HTML is an empty shell and the text arrives from JavaScript after load.

Draco escalates in tiers. Tier 0 and tier 1 parse the HTML with no JavaScript at all. Tier 2 runs the page's own JavaScript inside an in-process V8 isolate, which is the JavaScript engine without a browser around it, and the README states page code gets no host capability bindings there. That covers many single-page applications for a fraction of a browser's memory. When Draco meets a wall it cannot pass, draco scrape exits with code 3, needs_browser. Check it in scripts, because an empty file with a zero exit code is the failure that poisons an agent's context quietly:

draco scrape https://example.com > page.md
echo "exit=$?"

Firecrawl's playwright-service drives a real Chromium, so it renders what a browser renders. The self-hosted build is still not the hosted product: the documentation states self-hosted instances have no access to Fire Engine, so the anti-blocking and IP rotation of the cloud service are missing, and the /agent and /browser endpoints are unsupported. Hound sits in between on purpose. It fetches over HTTP and escalates per request, and its warm browser closes after an idle timeout, so a quiet box stays near baseline.

Install Draco with a pinned version

The README documents a one-line installer. Read what it does before piping it to a shell: it installs to $HOME/.draco/bin/draco, it always takes the latest release, and it checks no signature or hash. On a server, pin the version and verify the download.

cd /tmp
curl -fsSLO https://github.com/0xchasercat/draco/releases/download/v0.20.5/draco-linux-x86-64.tar.gz
curl -fsSLO https://github.com/0xchasercat/draco/releases/download/v0.20.5/SHA256SUMS
sha256sum --ignore-missing -c SHA256SUMS

That prints draco-linux-x86-64.tar.gz: OK. A FAILED line means the bytes you hold are not the bytes the project published, so delete them and start again.

mkdir -p draco-v0.20.5
tar -xzf draco-linux-x86-64.tar.gz -C draco-v0.20.5
sudo install -m 755 "$(find draco-v0.20.5 -type f -name draco | head -n1)" /usr/local/bin/draco
draco scrape https://example.com

The last command prints the example page as markdown in well under a second. The find is not decoration: the archive layout is not part of the project's public contract, and the official installer locates the binary the same way.

Run the daemon under its own account instead of your login user. Write /etc/systemd/system/draco.service:

[Unit]
Description=Draco fetch daemon
After=network-online.target
Wants=network-online.target

[Service]
User=draco
ExecStart=/usr/local/bin/draco serve --host 127.0.0.1 --port 3002 --max-concurrency 4
Restart=on-failure
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
PrivateTmp=true

[Install]
WantedBy=multi-user.target

sudo useradd --system --no-create-home --shell /usr/sbin/nologin draco
sudo systemctl daemon-reload
sudo systemctl enable --now draco
curl -s http://127.0.0.1:3002/health

/health answers as soon as the daemon is listening. Connection refused means it is not, so read journalctl -u draco -n 50. The usual cause is another process already holding 3002, since that is also Firecrawl's default port, and --port moves either one. Unit files in more depth: systemd service units and timers.

Now fetch the way your agent will:

curl -X POST http://127.0.0.1:3002/v1/scrape \
  -H 'content-type: application/json' \
  -d '{"url": "https://example.com", "formats": ["markdown"]}'

Keep the fetch daemon off the public internet

A fetch API with no authentication is an open proxy. Anyone who can reach the port makes your server request any URL under your IP address, and the abuse report goes to your provider, not to them. Draco's documented serve flags include no API key, so the protection has to be the network. Keep the default 127.0.0.1 bind when the agent runs on the same box. When the agent is elsewhere, put both ends on a private tunnel, a WireGuard VPN you host yourself is the usual answer, and bind to the tunnel address rather than 0.0.0.0. Then check from another machine that the public IP answers nothing. ufw firewall basics and least privilege user accounts cover the two halves of that.

Will your agent code change? API compatibility in practice

Draco answers the Firecrawl v1 routes: /v1/scrape, /v1/map, /v1/crawl, /v1/batch/scrape and /v1/search, and its README states that unknown fields are accepted and ignored. An agent that already posts to /v1/scrape needs a new base URL and nothing else. Watch what moved on the other side: Firecrawl's own self-hosting page now tests with /v2/crawl, and current SDKs speak v2, so a v2 client pointed at Draco asks for a route Draco does not publish. Test each call with curl before you edit agent code, and read the JSON body rather than the status code, because field names are where these implementations drift apart.

Robots.txt, rate limits, and the line to stay behind

Draco reads robots.txt by default, and --ignore-robots turns that off. Firecrawl documents the same default. Leave both alone. Then set your own pace: --delay puts milliseconds between requests and --max-concurrency bounds parallel jobs, with 8 as the daemon default. Two to four is kinder on a shared VPS link and rarely slower overall, because a site that starts rate limiting you costs more minutes than the concurrency saved. Cache what you fetch, so a second agent run costs the source nothing. That is also the cheapest line item in controlling what an AI agent costs you.

Challenge walls are a separate subject, and Trawl is built for exactly that: Cloudflare Turnstile, reCAPTCHA, hCaptcha and GeeTest. A challenge wall is a site refusing automated traffic in plain terms. Working around it puts you against the site's terms of use, and in some places against the law, so this guide covers fetching infrastructure and stops there. The same techniques that pass a wall are the ones site owners watch for and block, which makes any pipeline built on them fragile as well as rude. When a source matters that much, look for its RSS feed, its public API, or a bulk export. Each is cheaper to run and none of them breaks the week the wall changes.

Wire it to an agent over MCP

MCP (model context protocol) is the interface an agent uses to call a tool. Draco carries an MCP server in the same binary, over stdio:

{ "mcpServers": { "draco": { "command": "draco", "args": ["mcp"] } } }

The tools then appear to the agent as draco_scrape, draco_search and the draco_interact_* set. Stdio only works when the agent process and the binary sit on the same machine, because the transport is that process's standard input. For an agent on another host, Hound serves MCP over HTTP instead: hound --http --host 127.0.0.1 --port 8765 publishes an endpoint at http://127.0.0.1:8765/mcp, which you reach across the tunnel. Transport choices and what to expose are in running MCP servers on a VPS.

Fetching pairs with search. An agent that can only fetch waits for you to supply URLs. Add a self-hosted SearXNG search instance and it can find them itself, the same shape as the browser search skill built on SearXNG. Once the daemon is up, it is one shared service for whichever of the self-hosted AI agents you run.

FAQ

Do I need a headless browser to fetch pages for an AI agent?

Not for most pages. Server-rendered documentation, blogs and news articles come back complete from a plain HTTP fetch plus an HTML-to-markdown step, which is what Draco does in its lower tiers at roughly a browser-free 300 ms per page by the project's own figures. A browser earns its memory on client-rendered applications, where the delivered HTML is an empty shell. Draco's V8 isolate covers much of that middle ground with no browser process, and it exits with code 3, needs_browser, when it cannot.

How much RAM does self-hosted Firecrawl need on a VPS?

Its compose file sets a 8 GB ceiling on the api container and 4 GB on the Playwright container, and the same stack also starts Redis, RabbitMQ, PostgreSQL and FoundationDB. Plan for 8 GB. On a 2 GB box the kernel out-of-memory killer removes containers under load, and the first sign is a restarted container in docker compose ps with nothing useful in the application log, so confirm it with dmesg -T | tail.

Is Draco a drop-in replacement for the Firecrawl API?

For the v1 endpoints it is close. It serves /v1/scrape, /v1/map, /v1/crawl, /v1/batch/scrape and /v1/search, and it ignores request fields it does not know, so a client written against Firecrawl v1 usually needs only a new base URL. It is not the hosted product: there is no managed proxy pool behind it, and Firecrawl's newer v2 routes are not part of the surface. Verify each call your agent makes with curl first.

Does self-hosting a scraper mean I can ignore robots.txt?

No. Where the code runs changes nothing about what a site published or what its terms permit. Both Draco and Firecrawl respect robots.txt by default, and the override flag exists for sites you own or have written permission to crawl. Rate limits are enforced at the far end regardless, so a polite --delay with low concurrency keeps your IP address working. A stack that only functions by defeating a challenge wall is a stack that breaks without warning.