SSD Nodes Learn Hosting plans →
How to do am Matt ConnorBy Matt Connor

How to self-host Langfuse for AI agent tracing

Learn how to self-host Langfuse on your VPS. We cover pinned image tags, TLS setup, ClickHouse disk retention settings, and reliable backup steps to keep your data safe.

Why you need trace AI agent at all

You self-host Langfuse so you fit see wetin your agent actually do for one run. Langfuse na open source LLM (large language model) observability tool. E dey record every prompt, every model response, every tool call, and every token, then e group dem under one trace wey you fit open and read. If you run am for your own VPS, those prompts no go ever leave server wey you control.

The reason to do am clear. You no fit fix cost problem or quality problem wey you no fit see. Provider invoice fit tell you say Tuesday cost four times wetin Monday cost. Trace go tell you which agent run cause am, which prompt grow reach 40,000 tokens, and which retry loop run nine times before e give up. Invoice give you the number. Trace give you the code wey produce am.

Three terms dey show throughout this guide. A trace na one complete run of your agent from beginning to end. An observation na one step inside that run: a span for ordinary code, or a generation for call to model. A score na number wey human review or automated evaluator attach to a trace. Langfuse dey use OpenTelemetry (OTel), the vendor-neutral standard for distributed tracing, so instrumentation wey you already get fit point to am.

Wetin self-hosting Langfuse actually dey run

Langfuse v4 no be one container. Na two application containers and four storage services, and for one VPS all six go run for your box.

  • langfuse-web dey serve the web interface and the ingestion API.
  • langfuse-worker dey drain the queue for background. E dey parse ingestion batches, calculate cost, and run the nightly retention job.
  • Postgres dey hold transactional data like users, organisations, projects, API keys and prompts.
  • ClickHouse dey hold the trace data itself, meaning observations and scores. Na column store wey analytical queries dey use, na why dashboard over hundred million rows still dey answer quickly.
  • Redis na the queue and cache wey dey between web and worker.
  • MinIO dey give you S3 compatible object storage for the box. E dey hold every raw incoming event plus any media wey you attach.

Langfuse dey publish minimum resources for each component wey e dey run. The chart dey show the three wey dey do the work.

ChartLangfuse published minimum resources per component
Di data wey dey behind dis chart
[
  {
    "label": "ClickHouse",
    "cpu_cores": 2,
    "memory_gib": 8
  },
  {
    "label": "Langfuse web",
    "cpu_cores": 2,
    "memory_gib": 4
  },
  {
    "label": "Langfuse worker",
    "cpu_cores": 2,
    "memory_gib": 4
  }
]

ClickHouse alone dey ask for 8 GiB of memory. The web container and the worker each dey ask for 4 GiB. Those na the published minimums for the 3 components for the chart. The same page list Postgres with 4 GiB, Redis or Valkey with 1.5 GiB, and MinIO with 4 GiB, so the minimums for all six total 25.5 GiB. The project's own Docker Compose guide recommend at least 4 cores, 16 GiB of memory and around 100 GiB of storage for one machine. That one less than the sum of the component minimums, so treat 16 GiB as the least wey a shared Compose box need, no be extra capacity.

No try this for 2 GiB plan. ClickHouse go start, accept writes for some time, then e go die during background merge, because merge dey load large parts of table into memory. You go see docker compose ps dey report the clickhouse container as restarting, dmesg dey carry line like Out of memory: Killed process 1234 (clickhouse-serv), and every Langfuse dashboard dey return 500. When pressure no too much, ClickHouse fit refuse the query instead and log DB::Exception: Memory limit (total) exceeded. Sixteen na the number wey you need plan for. If the same VPS suppose carry anything else, budget for that separately, because even comparatively light stack like self-hosted AFFiNE workspace want its own gigabytes and ClickHouse no go give any back.

Deploy Langfuse with Docker Compose

Clone the repository. The stack, the wiring, and the default environment dey inside its docker-compose.yml.

git clone https://github.com/langfuse/langfuse.git
cd langfuse

Every value wey you must change get # CHANGEME mark for that file. Generate the three application secrets first.

openssl rand -base64 32   # NEXTAUTH_SECRET
openssl rand -base64 32   # SALT
openssl rand -hex 32      # ENCRYPTION_KEY

ENCRYPTION_KEY must be 256 bits, written as 64 hex characters. Na exactly wetin openssl rand -hex 32 prints be that. E dey encrypt sensitive values at rest, including any LLM provider keys wey you store for the instance. If you change am after data don exist, those rows no go fit decrypt again. Treat am as permanent from the first boot. SALT dey hash your Langfuse API keys, so changing am invalidates every key wey your agents dey already use.

Then set POSTGRES_PASSWORD, CLICKHOUSE_PASSWORD, REDIS_AUTH and MINIO_ROOT_PASSWORD. The MinIO password dey appear for four places: first as MINIO_ROOT_PASSWORD, then again as LANGFUSE_S3_EVENT_UPLOAD_SECRET_ACCESS_KEY, LANGFUSE_S3_MEDIA_UPLOAD_SECRET_ACCESS_KEY and LANGFUSE_S3_BATCH_EXPORT_SECRET_ACCESS_KEY. If you miss one, MinIO rejects that client with SignatureDoesNotMatch. The error go enter the worker log while the web interface still looks healthy. Keeping these values for an env file instead of the tracked compose file na the pattern wey Docker Compose env files and secrets cover.

Pin the image tags before you start

The shipped file dey use docker.langfuse.com/langfuse/langfuse:4 and docker.langfuse.com/langfuse/langfuse-worker:4. Those tags dey change. Langfuse runs its Postgres and ClickHouse migrations automatically when e starts. So ordinary docker compose pull months later fit become an unplanned schema migration for a database wey you no back up that morning. Pin both to one release inside a docker-compose.override.yml. Compose merges that file on top of the shipped file, so later git pull no go fight your edits. Docker Hub publishes the same images, and na there the override below dey pull dem from.

services:
  langfuse-web:
    image: docker.io/langfuse/langfuse:4.72.0
  langfuse-worker:
    image: docker.io/langfuse/langfuse-worker:4.72.0

Version 4.3.1 na the current 4.3 release as of August 2026. Releases don dey come quickly since then, and v4.56.0 na the latest for GitHub as of October 11, 2026. The override above dey use v4.72.0, wey na the latest for GitHub as of October 31, 2026. Check the project GitHub releases page, pin the version wey current on the day you deploy, then move that number deliberately. Some storage images for the shipped file need the same treatment, because their tags dey change too. postgres:17, wey be the default of a POSTGRES_VERSION variable, and redis:7 follow a major version. The shipped file now dey use an exact version tag for docker.io/clickhouse/clickhouse-server, so e dey pin an exact ClickHouse version. This rule no be only for Langfuse: a self-hosted openGym workout tracker dey run a smaller part of this stack and still need a named git tag, because anything wey migrates its own database when e starts fit turn ordinary pull into schema change.

Bring am up.

docker compose up -d
docker compose ps
docker compose logs -f langfuse-worker

First boot runs the migrations, so allow two or three minutes before anything answers. The web container logs Ready once e don come up. docker compose ps should list six services for state running. If container dey restart in a loop, its log go hold the reason. ClickHouse migrations apply when the web container starts, so bad CLICKHOUSE_MIGRATION_URL go show for docker compose logs langfuse-web, not for the worker log wey you dey tail. That variable dey use the ClickHouse native protocol for port 9000, not the HTTP port 8123. If you point am at 8123, the web container startup migrations go fail. The worker runs the slower background migrations, so na its log you go check for those reports.

Check health from the box itself.

curl -s "http://localhost:3000/api/public/health?failIfDatabaseUnavailable=true"
curl -s -o /dev/null -w '%{http_code}\n' http://localhost:3000/api/public/ready

Plain /api/public/health call only proves say the API process dey alive, because by default e no dey check the database. The failIfDatabaseUnavailable=true form na the one wey make sense to point monitor at, and e returns 503 when database no reachable. /api/public/ready returns 200 when the web container ready to receive traffic, and 500 once e don receive instruction to shut down. Both na ordinary HTTP checks, so an Uptime Kuma status page fit monitor dem and tell you say the stack don go down before your agents notice am.

Put TLS for front and close the extra ports

The compose file wey ship publish 3000:3000 for the web container and 9090:9000 for MinIO. Both dey bind for every interface. For public IP, anybody wey scan port 3000 fit reach your sign up page, and anybody wey scan 9090 dey talk to the bucket wey hold your raw prompts.

Firewall rule by itself no close dem. Docker dey write im own DNAT rules inside the nat table, and dem rules dey evaluate before ufw filter rules ever see the packet, so ufw deny 3000 leave the published port open. This issue common enough to get im own guide: why Docker published ports dey bypass ufw. Bind for loopback inside your override file instead. Use !override with Docker Compose 2.24.4 or later to replace the published ports; plain list dey merge with the original bindings.

services:
  langfuse-web:
    ports: !override
      - "127.0.0.1:3000:3000"
    environment:
      NEXTAUTH_URL: https://langfuse.example.com
  minio:
    ports: !override
      - "127.0.0.1:9090:9000"
      - "127.0.0.1:9091:9001"

NEXTAUTH_URL must be the exact public address, including the scheme, because the login flow dey build im callback URL from that value. If you leave am as http://localhost:3000 behind an HTTPS proxy, the sign in round trip go send the browser somewhere e no fit reach.

Now point a reverse proxy to 127.0.0.1:3000 and make am hold the certificate. For Traefik inside the same Compose project, route to langfuse-web:3000 through the Compose network with the labels wey running several apps behind one Traefik reverse proxy cover. Caddy fit do the same work with two lines if Langfuse na the only thing for the server. Verify with curl -sI https://langfuse.example.com/api/public/ready, then confirm from a second machine say curl http://YOUR_IP:3000 no fit connect.

One thing to note about MinIO. Langfuse dey give presigned URLs wey point to that S3 endpoint, both to SDKs wey upload media and to your browser wey load am. So if you use multi-modal traces wey carry images or audio, MinIO wey only bind to loopback no go upload those attachments from other machines or load dem for your browser. Read the blob storage configuration page before you proxy am, because the endpoint wey dey written inside the presigned URL, LANGFUSE_S3_MEDIA_UPLOAD_ENDPOINT, must match wetin you publish. Plain text traces no dey affected.

Create your account on the first visit, then make the instance remain your own. Set AUTH_DISABLE_SIGNUP=true after your account don exist, so stranger wey reach the page no fit register at all. Langfuse get LANGFUSE_ALLOWED_ORGANIZATION_CREATORS too, wey limit who fit create organisations, but na Enterprise Edition feature and e only take effect with a license key. If you already dey run Authentik as your own identity provider, Langfuse accept standard OIDC connection, so accounts dey come and go with your other apps instead of living inside password list wey only this server know about.

Send your first trace

Create one project for web interface, then copy the public and secret keys from project settings. The Python SDK dey read three environment variables.

export LANGFUSE_PUBLIC_KEY="pk-lf-..."
export LANGFUSE_SECRET_KEY="sk-lf-..."
export LANGFUSE_BASE_URL="https://langfuse.example.com"

LANGFUSE_BASE_URL na the variable name for SDK v4, wey release for March 2026. Older code and older guides dey use LANGFUSE_HOST. If your traces dey enter Langfuse Cloud instead of your server, na because base URL no dey set; the default one point to the hosted instance.

The current Python SDK v5 need Python 3.11 or later and self-hosted Langfuse v4.59.0 or later. The pinned server wey dey above meet this requirement.

pip install langfuse opentelemetry-instrumentation-anthropic anthropic
import os
from anthropic import Anthropic
from langfuse import get_client, observe
from opentelemetry.instrumentation.anthropic import AnthropicInstrumentor

AnthropicInstrumentor().instrument()
langfuse = get_client()
client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])

@observe(as_type="tool")
def lookup_order(order_id: str) -> str:
    return f"order {order_id}: shipped"

@observe()
def handle_request(question: str) -> str:
    context = lookup_order("A-1042")
    message = client.messages.create(
        model="claude-haiku-4-5",
        max_tokens=512,
        messages=[{"role": "user", "content": f"{context}\n\n{question}"}],
    )
    return message.content[0].text

if __name__ == "__main__":
    assert langfuse.auth_check()
    print(handle_request("Where is my order?"))
    langfuse.flush()

The @observe decorator open observation around the function, capture its arguments and return value, then nest am under any observation wey already dey active. AnthropicInstrumentor na the OpenTelemetry instrumentation for Anthropic client. E turn every messages.create call into one generation wey carry the model name, token usage, and latency, without changing the call site.

Two calls go do the checks for you. langfuse.auth_check() return False when keys bad or base URL wrong. This one faster than wondering why dashboard empty. langfuse.flush() block until queued spans don send. Short-lived processes need am because the SDK batch for background, and script wey exit immediately go carry the unsent batch comot. If na whole team dey send traces instead of one script, set those three variables once for the gateway wey everybody agents already dey pass through, instead of setting dem for each person shell. Na so self-hosted OneCLI harness dey keep every colleague runs instrumented while the keys remain for one place.

Why ClickHouse dey keep growing?

Traces na the data wey dey grow fastest for most things people self-host. Every agent run dey write one row for each step. E dey store inputs and outputs complete. So agent wey dey talk plenty and get long prompts fit produce much more bytes every day than the application wey e dey monitor. If you leave am, ClickHouse go fill the disk. Full disk go stop ingestion instead of just making am slow.

Two different things dey grow here, and each one need separate fix.

The first one na your own trace data. The fix na the retention setting. For self-hosted instance, data retention na Enterprise Edition feature and e need license key. If you no get license, Langfuse go keep traces, observations, scores and media forever. Size the disk for this, and depend on the disk alert for the end of this section. With license, project owners and admins go open project settings and set data retention period for days. Langfuse accepts minimum of 3 days. Every night, one job go select traces, observations, scores and media assets wey older than that window, then delete dem from ClickHouse and blob storage. E no go delete media wey connect to saved dataset items. The job need DeleteObject permission for the bucket. The MinIO root credentials for the default compose file already get this permission. Deletion permanent, so configure blob storage export first if you need long-term history. No write TTL clauses by hand for Langfuse own tables. The retention job na wetin keep ClickHouse and the bucket in sync. Manual TTL go delete only one side.

Choose the window based on wetin you actually use. Cost and quality review dey happen on data wey be some days old, not data wey be months old. Thirty days na reasonable starting point for small team. 14 days fit work if you only open trace when something break.

The second one na ClickHouse own system log tables. This one dey surprise people because the disk still dey grow after you configure retention. ClickHouse dey write trace_log, text_log, opentelemetry_span_log, metric_log, asynchronous_metric_log and latency_log for its own diagnostics. Dem ship without TTL, and Langfuse never reads dem. First find where the disk space actually go.

SELECT table, formatReadableSize(size) AS size, rows FROM (
    SELECT table, database, sum(bytes) AS size, sum(rows) AS rows
    FROM system.parts
    WHERE active
    GROUP BY table, database
    ORDER BY size DESC
)

Run am with docker compose exec clickhouse clickhouse-client --password "$CLICKHOUSE_PASSWORD". If system tables dey near the top, disable dem with config overlay, because ClickHouse merges every file inside /etc/clickhouse-server/config.d/ over its main config when e starts.

<clickhouse>
    <trace_log remove="1"/>
    <text_log remove="1"/>
    <opentelemetry_span_log remove="1"/>
    <asynchronous_metric_log remove="1"/>
    <metric_log remove="1"/>
    <latency_log remove="1"/>
</clickhouse>

Mount am and restart ClickHouse.

services:
  clickhouse:
    volumes:
      - ./clickhouse-config.d/system-logs.xml:/etc/clickhouse-server/config.d/system-logs.xml:ro

This one go stop new writes. Rows wey already dey on disk go remain there. So reclaim the space explicitly with DROP TABLE IF EXISTS system.trace_log, and do the same for every table wey you remove. If you prefer to keep the diagnostics, use aggressive TTL on each table instead of remove="1". The Langfuse scaling docs explain this process. Those TTL directives only apply when system table first gets created. For existing install, the same docs also show how to retrofit the tables.

One more table dey worth knowing. blob_storage_file_log dey track event files wey upload to your bucket. If you also set lifecycle policy for the bucket, give the table matching TTL so both no go drift apart.

ALTER TABLE blob_storage_file_log MODIFY TTL created_at + INTERVAL 30 DAY DELETE;

Set plain df -h alert on the data disk too. Traces no dey grow smoothly. Dem grow on the day you release new agent. The first sign of that no suppose be ingestion failing.

Back up Postgres and ClickHouse

Langfuse backup get three parts. Postgres dey hold your users, organisations, projects and API keys. ClickHouse dey hold the traces. MinIO dey hold the raw events. If you restore only Postgres, login go work but history no go dey. If you restore only ClickHouse, history go dey but nobody fit log in to see am. This split no be only Langfuse matter. self-hosted Chatwoot support desk get the same arrangement. If you take Postgres dump without the uploads directory, conversations go restore but all their attachments go miss.

Postgres na plain pg_dump, and na wetin Langfuse backup docs use for Docker install.

docker compose exec -T postgres pg_dump -U postgres postgres \
  | gzip > langfuse-pg-$(date +%F).sql.gz

ClickHouse need more care, because if you copy live data directory while merges dey run, the backup no go consistent. For one server, the simple method na to stop the container and archive the volume.

docker compose stop clickhouse
docker volume ls | grep clickhouse
docker run --rm -v langfuse_langfuse_clickhouse_data:/data -v "$PWD":/backup alpine \
  tar czf /backup/langfuse-ch-$(date +%F).tar.gz -C /data .
docker compose start clickhouse

Use the volume name wey docker volume ls print, no be the one wey YAML write. The file declare langfuse_clickhouse_data, and Compose add the project name as prefix. So clone for directory wey dem call langfuse go produce langfuse_langfuse_clickhouse_data. If you use the wrong one, docker run go create new empty volume without any error, and your archive go contain nothing.

Langfuse upload raw event data go the bucket for every ingestion request, and e dey read am again to retry failed ingestion jobs. So short ClickHouse stop mostly mean say the worker go retry afterwards. Do am for quiet hour and keep the stop short. For busier instance, ClickHouse own BACKUP DATABASE default TO S3(...) statement fit write consistent backup without stopping the server. MinIO na the third piece, and mc mirror or MinIO replication to off-box bucket fit cover am. Anything wey you produce, move am comot from the server. Na wetin encrypted restic backups on a VPS dey help you do.

Redis dey hold the queue and cache. If you lose am, queued jobs go disappear, even when their raw event files still dey the bucket.

The consistency warning dey important and make we talk am plainly. Postgres and ClickHouse dumps happen for different times. Because of that, restore fit leave project row without traces, or traces wey belong to project wey no dey again. Langfuse fit tolerate this, but take both dumps close together and for low-traffic window. The event bucket na the real safety net, because Langfuse upload raw event data there for every ingestion request.

Restore am into scratch stack at least once. Na so you go discover wrong volume name now, instead of during outage.

Wetin to look at first

Four things deserve make you check dem for the first week.

  • Cost per trace. Langfuse dey calculate cost from model name and token usage, so sort traces by cost and read the most expensive one from beginning to end. Most times, na prompt wey don grow too much cause am: person paste whole document inside context, or conversation history wey nobody dey trim. Once you fit see am, how to control wetin AI agent dey cost you go become engineering task instead of guesswork.
  • Token usage split by input and output. Input tokens plenty and cheap, output tokens few and expensive, while cached input cheaper pass again. how Claude Code token usage dey counted explain the same accounting in detail, and e apply to any agent wey you write yourself.
  • Latency percentiles. Median dey hide the problem. Na p95 and p99 be where timeouts dey happen, and inside agent loop, slow tool call for p95 go multiply by the number of iterations.
  • Failed tool calls. Filter observations by level ERROR. Tool wey dey fail 5% of the time fit no show for aggregate success rate, but e go show clearly for traces, where you go monitor as model retry am and then use more tokens to work around the failure.

Set the retention window and choose the dashboard wey you go check every week on the same day wey you deploy. Observability tool wey nobody dey open na database wey go eventually fill disk.

FAQ

How much memory does a self-hosted Langfuse need?

Plan for 4 CPU cores and 16 GiB memory, because na wetin Langfuse Docker Compose guide recommend for one virtual machine. Add around 100 GiB storage too. The published component minimums na 8 GiB for ClickHouse and 4 GiB each for the web and worker containers. The same page list 4 GiB for Postgres, 1.5 GiB for Redis and 4 GiB for MinIO. So all six add up to 25.5 GiB, wey pass the 16 GiB Compose recommendation.

Why does my ClickHouse disk keep filling up after I set data retention?

The retention setting cover Langfuse own data only. ClickHouse dey write diagnostic tables trace_log, text_log, opentelemetry_span_log, metric_log, asynchronous_metric_log and latency_log separately, and dem come without TTL. Query system.parts grouped by table to see which one big pass. Then disable the unused tables with a remove="1" entry inside a file under /etc/clickhouse-server/config.d/. Restart ClickHouse, then drop the existing tables to recover the space wey dem don already use.

What is the minimum data retention period in Langfuse?

Three days. You set retention per project inside project settings, or through the Organization Management API. Every night, a job deletes traces, observations, scores and media assets wey older than the configured window from both ClickHouse and blob storage. E no delete media wey connect to saved dataset items. For self-hosted instance, na Enterprise Edition feature wey need license key. Without license key, data dey remain indefinitely. You no fit undo deletion, so configure blob storage export first if you need history beyond that window.

Do I have to back up both Postgres and ClickHouse?

Yes, because dem hold different things. Postgres hold users, organisations, projects and API keys, while ClickHouse hold the trace data itself. If you restore Postgres only, you go get instance wey you fit log into but nothing dey inside. Back up the MinIO bucket too, because e hold the raw events wey Langfuse persist as dem arrive. Na this be the closest thing to a source of truth for the stack.

Can I point an existing OpenTelemetry setup at self-hosted Langfuse?

Yes. Langfuse v4 and the current SDKs dey built on OpenTelemetry, and the Anthropic and OpenAI OTel instrumentations export directly to am. For Python, run pip install langfuse opentelemetry-instrumentation-anthropic, call AnthropicInstrumentor().instrument() once when the application start, and set LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY and LANGFUSE_BASE_URL to your own host. Confirm with langfuse.auth_check() before you start looking for a missing dashboard.

Dis article help you?