How to self-host Langfuse for AI agent tracing

Why you need trace AI agent at all

You self-host Langfuse so you fit see wetin your agent actually do for one run. Langfuse na open source LLM (large language model) observability tool. E dey record every prompt, every model response, every tool call, and every token, then e group dem under one trace wey you fit open and read. If you run am for your own VPS, those prompts no go ever leave server wey you control.

The reason to do am clear. You no fit fix cost problem or quality problem wey you no fit see. Provider invoice fit tell you say Tuesday cost four times wetin Monday cost. Trace go tell you which agent run cause am, which prompt grow reach 40,000 tokens, and which retry loop run nine times before e give up. Invoice give you the number. Trace give you the code wey produce am.

Three terms dey show throughout this guide. A trace na one complete run of your agent from beginning to end. An observation na one step inside that run: a span for ordinary code, or a generation for call to model. A score na number wey human review or automated evaluator attach to a trace. Langfuse dey use OpenTelemetry (OTel), the vendor-neutral standard for distributed tracing, so instrumentation wey you already get fit point to am.

Langfuse self-hosting really dey run

Langfuse v4 no be one container. Na two application containers and four storage services, and for one VPS all six go dey run for your box.

• langfuse-web dey serve web interface and ingestion API.
• langfuse-worker dey drain queue for background. E dey parse ingestion batches, calculate cost, and run nightly retention job.
• Postgres dey hold transactional data like users, organisations, projects, API keys and prompts.
• ClickHouse dey hold the trace data itself, meaning observations and scores. Na column store wey analytical queries dey use, na why dashboard over hundred million rows still dey answer fast.
• Redis na the queue and cache wey dey between web and worker.
• MinIO dey give you S3 compatible object storage for the box. E dey hold every raw incoming event plus any media wey you attach.

Langfuse dey publish minimum resources for the three components wey dey do the main work.

[
  {
    "label": "ClickHouse",
    "cpu_cores": 2,
    "memory_gib": 8
  },
  {
    "label": "Langfuse web",
    "cpu_cores": 2,
    "memory_gib": 4
  },
  {
    "label": "Langfuse worker",
    "cpu_cores": 2,
    "memory_gib": 4
  }
]

ClickHouse alone dey ask for 8 GiB memory. The web container and worker dey ask for 4 GiB each. Na the published minimum for the 3 components wey Langfuse size, and Postgres, Redis and MinIO still need extra memory on top. The project own Docker Compose guide recommend machine wey get 4 cores and 16 GiB memory plus around 100 GiB storage. This match the calculation instead of adding extra padding.

No try this for 2 GiB plan. ClickHouse go start, accept writes for some time, then die during background merge, because merge dey load large table parts into memory. You go see docker compose ps reporting say clickhouse container dey be restarting, dmesg carrying line like Out of memory: Killed process 1234 (clickhouse-serv), and every Langfuse dashboard dey return 500. When pressure no too high, ClickHouse fit refuse the query instead and log DB::Exception: Memory limit (total) exceeded. Eight GiB fit work for one developer wey dey send few thousand traces every day. Sixteen na the number to plan for.

Langfuse deploy with Docker Compose

Clone the repository. The stack, the wiring and the default environment dey inside its docker-compose.yml.

git clone https://github.com/langfuse/langfuse.git
cd langfuse

Every value wey you must change get # CHANGEME mark for that file. Generate the three application secrets first.

openssl rand -base64 32   # NEXTAUTH_SECRET
openssl rand -base64 32   # SALT
openssl rand -hex 32      # ENCRYPTION_KEY

ENCRYPTION_KEY must be 256 bits, written as 64 hex characters. Na exactly wetin openssl rand -hex 32 go print. E dey encrypt sensitive values at rest, including any LLM provider keys wey you store for the instance. If you change am after data don exist, those rows no fit decrypt again. So treat am as permanent from the first boot. SALT dey hash your Langfuse API keys. If you change am, every key wey your agents dey use already go stop working.

Then set POSTGRES_PASSWORD, CLICKHOUSE_PASSWORD, REDIS_AUTH and MINIO_ROOT_PASSWORD. The MinIO password dey four places: first as MINIO_ROOT_PASSWORD, then again as LANGFUSE_S3_EVENT_UPLOAD_SECRET_ACCESS_KEY, LANGFUSE_S3_MEDIA_UPLOAD_SECRET_ACCESS_KEY and LANGFUSE_S3_BATCH_EXPORT_SECRET_ACCESS_KEY. If you miss one, MinIO go reject that client with SignatureDoesNotMatch. The error go enter the worker log while the web interface still dey look healthy. Keeping these values for an env file instead of the tracked compose file na the pattern wey Docker Compose env files and secrets cover.

Pin the image tags before you start

The shipped file dey use langfuse/langfuse:4 and langfuse/langfuse-worker:4. Those tags dey change. Langfuse dey run its Postgres and ClickHouse migrations automatically when e start. So one routine docker compose pull months later fit become an unplanned schema migration for a database wey you no back up that morning. Pin both to one release inside a docker-compose.override.yml. Compose go merge am on top of the shipped file, so later git pull no go fight your edits.

services:
  langfuse-web:
    image: docker.io/langfuse/langfuse:4.3.1
  langfuse-worker:
    image: docker.io/langfuse/langfuse-worker:4.3.1

Version 4.3.1 na the current 4.3 release as of August 2026 (4.4.0 don release since then). Check the project's GitHub releases page. Pin whichever version dey current on the day you deploy, then change that number deliberately. The storage images for the shipped file don already pin to major versions: postgres:17, clickhouse-server:25.12 and redis:7. Dem deserve the same treatment.

Bring am up.

docker compose up -d
docker compose ps
docker compose logs -f langfuse-worker

First boot dey run the migrations, so allow one or two minutes before anything respond. docker compose ps suppose list six services for state running. If the worker dey restart in a loop, its log go show the reason: CLICKHOUSE_MIGRATION_URL dey use the ClickHouse native protocol on port 9000, not the HTTP port 8123. If you point am to 8123, e go fail there while the web container still dey look fine.

Check health from the box itself.

curl -s "http://localhost:3000/api/public/health?failIfDatabaseUnavailable=true"
curl -s -o /dev/null -w '%{http_code}\n' http://localhost:3000/api/public/ready

A plain /api/public/health call only prove say the API process dey alive, because e deliberately skip the database so the service fit continue serving when Postgres blips. The failIfDatabaseUnavailable=true form na the one wey make sense to point monitor at, and e dey return 503 when the database no reachable. /api/public/ready dey return 200 after migrations finish and the container fit accept traffic. Both na ordinary HTTP checks, so an Uptime Kuma status page fit monitor dem and tell you say the stack don go down before your agents notice.

Put TLS for front and close the extra ports

The compose file wey ship publish 3000:3000 for the web container and 9090:9000 for MinIO. Both dey bind for every interface. For public IP, this mean anybody wey scan port 3000 fit reach your sign-up page, and anybody wey scan 9090 dey talk to the bucket wey hold your raw prompts.

Firewall rule by itself no close dem. Docker dey write im own DNAT rules inside the nat table, and dem rules dey process before ufw filter rules ever see the packet, so ufw deny 3000 leave the published port open. This issue common enough to get im own guide: why Docker published ports dey bypass ufw. Bind am to loopback inside your override file instead.

services:
  langfuse-web:
    ports:
      - "127.0.0.1:3000:3000"
    environment:
      NEXTAUTH_URL: https://langfuse.example.com
  minio:
    ports:
      - "127.0.0.1:9090:9000"
      - "127.0.0.1:9091:9001"

NEXTAUTH_URL must be the exact public address, including the scheme, because the login flow dey build im callback URL from that value. If you leave am as http://localhost:3000 behind an HTTPS proxy, the sign-in round trip go send the browser go somewhere e no fit reach.

Now point a reverse proxy to 127.0.0.1:3000 and make am hold the certificate. Traefik for the same Compose project na the usual choice, and the routing labels na the ones wey running several apps behind one Traefik reverse proxy cover. Caddy fit do the same work with two lines if Langfuse na the only thing for the server. Verify with curl -sI https://langfuse.example.com/api/public/ready, then confirm from another machine say curl http://YOUR_IP:3000 now dey time out.

One thing to note about MinIO. Langfuse dey serve attached media to your browser through presigned URLs wey point to that S3 endpoint. So, if you use multi-modal traces wey carry images or audio, MinIO wey only dey listen on loopback mean the attachments no go load. Read the blob storage configuration page before you proxy am, because the endpoint wey dey inside the presigned URL must match the endpoint wey you publish. Plain text traces no dey affected.

Create your account the first time you visit, then keep the instance under your control. Set LANGFUSE_ALLOWED_ORGANIZATION_CREATORS to your own email address, so stranger wey reach the page no fit create organisation for your server. If you already dey run Authentik as your own identity provider, Langfuse accept standard OIDC connection, so accounts fit come and go with your other apps instead of living inside password list wey only this server know.

Send your first trace

Create project for web interface and copy the public and secret keys from project settings. The Python SDK dey read three environment variables.

export LANGFUSE_PUBLIC_KEY="pk-lf-..."
export LANGFUSE_SECRET_KEY="sk-lf-..."
export LANGFUSE_BASE_URL="https://langfuse.example.com"

LANGFUSE_BASE_URL na the variable name for SDK v4, wey release for March 2026. Older code and older guides dey use LANGFUSE_HOST. If your traces dey land for Langfuse Cloud instead of your server, na because base URL no set; the default dey point to the hosted instance.

pip install langfuse opentelemetry-instrumentation-anthropic anthropic

import os
from anthropic import Anthropic
from langfuse import get_client, observe
from opentelemetry.instrumentation.anthropic import AnthropicInstrumentor

AnthropicInstrumentor().instrument()
langfuse = get_client()
client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])

@observe(as_type="tool")
def lookup_order(order_id: str) -> str:
    return f"order {order_id}: shipped"

@observe()
def handle_request(question: str) -> str:
    context = lookup_order("A-1042")
    message = client.messages.create(
        model="claude-haiku-4-5",
        max_tokens=512,
        messages=[{"role": "user", "content": f"{context}\n\n{question}"}],
    )
    return message.content[0].text

if __name__ == "__main__":
    assert langfuse.auth_check()
    print(handle_request("Where is my order?"))
    langfuse.flush()

The @observe decorator dey open observation around the function, capture its arguments and return value, then nest am under any observation wey already dey active. AnthropicInstrumentor na the OpenTelemetry instrumentation for Anthropic client, and e dey turn each messages.create call into generation wey carry model name, token usage, and latency, without changing anything for the call site.

Two calls dey do the checking for you. langfuse.auth_check() dey return False for bad keys or wrong base URL, and this faster pass to dey wonder why dashboard empty. langfuse.flush() dey block until e send queued spans, and short-lived processes need am, because the SDK dey batch for background and script wey exit immediately go carry the unsent batch comot with am.

ClickHouse why e dey keep growing?

Traces na the data wey dey grow fastest for most things wey people self-host. Every agent run dey write one row for each step, and e dey store inputs and outputs complete. So agent wey dey talk plenty and use long prompts fit produce much more bytes per day than the application wey e dey monitor. If you leave am like that, ClickHouse go fill the disk, and full disk go stop ingestion instead of just making am slow.

Two different things dey grow here, and each one need different fix.

The first one na your own trace data, and retention setting na the fix. Open project settings for the web interface and set data retention period in days. Langfuse accepts minimum of 3 days. Every night, one job selects traces, observations, scores and media assets wey older than that period, then e deletes dem from ClickHouse and blob storage. The job need DeleteObject permission for the bucket. The MinIO root credentials for the default compose file already get this permission. Deletion permanent, so configure blob storage export first if you need long-term history. No write TTL clauses by hand for Langfuse own tables. The retention job na wetin keep ClickHouse and the bucket synchronized. Manual TTL go delete only one side.

Choose the period based on wetin you actually use. Cost and quality review dey happen with data wey old by days, not months. Thirty days na reasonable starting point for small team, and 14 days enough if you only open trace when something break.

The second one na ClickHouse own system log tables. This one dey surprise people because disk dey continue to grow after dem configure retention. ClickHouse dey write trace_log, text_log, opentelemetry_span_log, metric_log and asynchronous_metric_log for its own diagnostics. Dem come without TTL, and Langfuse never reads dem. First find where the disk space actually go.

SELECT table, formatReadableSize(size) AS size, rows FROM (
    SELECT table, database, sum(bytes) AS size, sum(rows) AS rows
    FROM system.parts
    WHERE active
    GROUP BY table, database
    ORDER BY size DESC
)

Run am with docker compose exec clickhouse clickhouse-client --password "$CLICKHOUSE_PASSWORD". If system tables dey near the top, disable dem with config overlay, because ClickHouse dey merge every file for /etc/clickhouse-server/config.d/ over its main config when e start.

<clickhouse>
    <trace_log remove="1"/>
    <text_log remove="1"/>
    <opentelemetry_span_log remove="1"/>
    <asynchronous_metric_log remove="1"/>
    <metric_log remove="1"/>
</clickhouse>

Mount am and restart ClickHouse.

services:
  clickhouse:
    volumes:
      - ./clickhouse-config.d/system-logs.xml:/etc/clickhouse-server/config.d/system-logs.xml:ro

That go stop new writes. Rows wey already dey for disk go remain there, so reclaim the space explicitly with DROP TABLE IF EXISTS system.trace_log and do the same for each table wey you remove. If you prefer keep the diagnostics, the alternative na to put aggressive TTL for each table instead of remove="1". Langfuse scaling docs explain this option.

One more table dey worth knowing. blob_storage_file_log dey track the event files wey upload to your bucket. If you also set lifecycle policy for the bucket, give the table matching TTL so the two no go become different.

ALTER TABLE blob_storage_file_log MODIFY TTL created_at + INTERVAL 30 DAY DELETE;

Put plain df -h alert for the data disk too. Traces no dey grow smoothly. Dem grow on the day you release new agent, and ingestion failing no suppose be the first sign.

Back up Postgres and ClickHouse

Langfuse backup get three parts. Postgres dey hold your users, organisations, projects and API keys. ClickHouse dey hold the traces. MinIO dey hold the raw events. If you restore only Postgres, you go get working login but no history. If you restore only ClickHouse, you go get history wey nobody fit log in to see.

Postgres na plain pg_dump, and na wetin Langfuse backup docs recommend.

docker compose exec -T postgres pg_dump -U postgres postgres \
  | gzip > langfuse-pg-$(date +%F).sql.gz

ClickHouse need more care, because if you copy live data directory while merges dey run, the backup no go consistent. For one box, the simple way na to stop the container and archive the volume.

docker compose stop clickhouse
docker volume ls | grep clickhouse
docker run --rm -v langfuse_langfuse_clickhouse_data:/data -v "$PWD":/backup alpine \
  tar czf /backup/langfuse-ch-$(date +%F).tar.gz -C /data .
docker compose start clickhouse

Use the volume name wey docker volume ls prints, no be the one wey dem write for the YAML. The file declare langfuse_clickhouse_data, and Compose add the project name as prefix, so clone for directory wey dem call langfuse go produce langfuse_langfuse_clickhouse_data. If you get am wrong, docker run go create new empty volume without any complaint, and your archive go contain nothing.

The web container dey write every incoming event to the bucket before the worker process am, so short ClickHouse stop mostly mean say the worker go retry afterwards. Do am for quiet hour and make e short. For busier instance, ClickHouse own BACKUP DATABASE default TO S3(...) statement fit write consistent backup without stopping the server. MinIO na the third piece, and mc mirror or MinIO replication go an off-box bucket cover am. Anything wey you produce, move am comot from the server. Na wetin encrypted restic backups for VPS dey do.

Redis no need backup. E dey hold the queue and cache, so if e loss, na only events wey dey process at that time you lose; older ones no go loss.

The consistency warning na real, and e good make we talk am plain. Postgres and ClickHouse dumps happen for different times, so restore fit leave project row without traces, or traces wey belong to project wey no longer exist. Langfuse fit tolerate this, but take both dumps close together and for low-traffic period. The event bucket na the real safety net, because Langfuse dey save every incoming event there before processing am.

Restore into scratch stack at least once. Na so you go discover wrong volume name now, instead of during outage.

Wetin to look at first

Four things deserve make you check dem for the first week.

• Cost per trace. Langfuse dey calculate cost from model name and token usage, so sort traces by cost and read the most expensive one from beginning to end. Most times, na prompt wey don grow too much cause am: person paste whole document inside context, or conversation history wey nobody dey trim. Once you fit see am, how to control wetin AI agent dey cost you go become engineering task instead of guesswork.
• Token usage split by input and output. Input tokens plenty and cheap, output tokens few and expensive, while cached input cheaper pass again. how Claude Code token usage dey counted explain the same accounting in detail, and e apply to any agent wey you write yourself.
• Latency percentiles. Median dey hide the problem. Na p95 and p99 be where timeouts dey happen, and inside agent loop, slow tool call for p95 go multiply by the number of iterations.
• Failed tool calls. Filter observations by level ERROR. Tool wey dey fail 5% of the time fit no show for aggregate success rate, but e go show clearly for traces, where you go monitor as model retry am and then use more tokens to work around the failure.

Set the retention window and choose the dashboard wey you go check every week on the same day wey you deploy. Observability tool wey nobody dey open na database wey go eventually fill disk.

FAQ

How much memory does a self-hosted Langfuse need?

Plan for 4 CPU cores and 16 GiB memory, because na wetin the Langfuse Docker Compose guide recommend for one virtual machine, plus around 100 GiB storage. The published component minimums na 8 GiB for ClickHouse and 4 GiB each for the web and worker containers. Postgres, Redis and MinIO still need extra memory on top of those. Eight GiB fit run one developer instance. Two GiB no fit: kernel dey kill ClickHouse during background merges, and dmesg dey show Out of memory: Killed process.

Why does my ClickHouse disk keep filling up after I set data retention?

The retention setting cover Langfuse own data only. ClickHouse separately dey write diagnostic tables trace_log, text_log, opentelemetry_span_log, metric_log and asynchronous_metric_log, and dem come without TTL. Query system.parts and group am by table to see which one big pass. Then disable the unused tables with a remove="1" entry inside a file under /etc/clickhouse-server/config.d/. Restart ClickHouse, then drop the existing tables to reclaim the space wey dem don already use.

What is the minimum data retention period in Langfuse?

Three days. You fit set retention for each project inside project settings, or through the projects API. A nightly job dey delete traces, observations, scores and media assets wey older pass the window from both ClickHouse and blob storage. You no fit undo the deletion, so configure blob storage export first if you need history beyond that window.

Do I have to back up both Postgres and ClickHouse?

Yes, because dem dey hold different things. Postgres dey hold users, organisations, projects and API keys, while ClickHouse dey hold the trace data itself. If na Postgres-only restore, you go get instance wey you fit log into but nothing dey inside. Back up the MinIO bucket too, because e dey hold the raw events wey Langfuse persist when dem arrive. Na this be the closest thing to source of truth for the stack.

Can I point an existing OpenTelemetry setup at self-hosted Langfuse?

Yes. Langfuse v4 and its v4 SDKs dey built on OpenTelemetry, and the Anthropic and OpenAI OTel instrumentations export directly to am. For Python, run pip install langfuse opentelemetry-instrumentation-anthropic, call AnthropicInstrumentor().instrument() once when startup happen, and set LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY and LANGFUSE_BASE_URL to your own host. Confirm with langfuse.auth_check() before you start looking for missing dashboard.