Self-Hosted Log Management on One VPS

What self-hosted log management on one VPS really costs

Self-hosted log management on one VPS (virtual private server) comes down to a single question: do you need a search cluster, or do you need rotation and a grep? Most vendor guides answer that by starting at three nodes and 12 GB of RAM before a single log line is shipped. On one server that answer is useless, so the comparison below is by what each option demands from a small box before it holds anything.

If you run one or two servers and you want to know what happened last Tuesday, systemd-journald and logrotate already do that job, and you can stop after the next section. If several machines must land their logs in one place with search over weeks, Grafana Loki fits a small box, because it indexes labels and not the text of the lines. Elasticsearch and OpenSearch give you real full text search, and they charge for it in memory, because the JVM (Java virtual machine) heap has a floor you cannot go under.

Start with journald, because most people stop here

systemd-journald is already running on any current Ubuntu or Debian server. It captures the standard output of every service unit, the kernel messages, and anything sent to syslog. Four commands cover most incidents.

journalctl -u nginx.service --since "2026-08-14 09:00" --until "2026-08-14 10:00"
journalctl -p err -b
journalctl -f -u ssh.service
journalctl --disk-usage

The last one prints a line like Archived and active journals take up 1.1G in the file system. That is the number that decides whether you need anything else. If it reads a few hundred megabytes and you can find what you need with -u and --since, you are done.

Whether the journal survives a reboot depends on Storage= and on whether /var/log/journal exists. With the common Storage=auto setting, journald writes to /var/log/journal when that directory is present, and to /run/log/journal when it is not. /run is memory backed, so on a box without that directory every log is erased at reboot, which is the exact moment you want to read them. Ubuntu images ship the directory. Minimal and container based images often do not.

ls -d /var/log/journal
sudo mkdir -p /var/log/journal
sudo systemctl restart systemd-journald
journalctl --disk-usage

After the restart, journalctl --disk-usage should report a size under /var/log/journal rather than /run. The defaults are already bounded, which is the main reason journald is a serious answer and not a fallback. The journald.conf man page sets SystemMaxUse= to 10% of the file system size and SystemKeepFree= to 15%, and caps each calculated default at 4G. SystemMaxFileSize= defaults to one eighth of SystemMaxUse=, capped at 128M, so you normally keep seven rotated files. MaxRetentionSec= defaults to 0, which turns age based deletion off. Read that last default again: out of the box the journal is limited by size only, never by age.

[Journal]
Storage=persistent
SystemMaxUse=2G
MaxRetentionSec=30day

Write that to /etc/systemd/journald.conf.d/99-size.conf, restart journald, then check that journalctl --disk-usage moved toward your new ceiling. To reclaim space now rather than waiting for the next rotation, run sudo journalctl --vacuum-size=500M or sudo journalctl --vacuum-time=14d. Both print every file they remove, so a silent run means there was nothing to delete.

Everything outside the journal, such as /var/log/nginx/access.log, is logrotate's job, and it runs daily from a systemd timer. One failure is worth knowing because it looks like a bug in df. After a rotation the old file is gone from the directory listing while the daemon still holds it open, so df -h reports the disk full while du -sh /var/log reports far less. The space comes back only when the process reopens its log, which is what the postrotate reload line in the config is for. sudo lsof -nP +L1 lists deleted files that are still held open, and names the process holding each one. Test a rule without touching anything using sudo logrotate -d /etc/logrotate.d/nginx.

Sending logs from several servers to one collector

Once there is more than one box, managing several Linux servers at once gets easier when their logs arrive in one place. rsyslog is installed on most distributions already, so the cheapest central collector is one file on each sender.

*.* action(type="omfwd" target="logs.example.com" port="514" protocol="tcp")

Save that as /etc/rsyslog.d/50-forward.conf, check it with sudo rsyslogd -N1, which validates the configuration and exits without starting anything, then restart rsyslog. On the collector, enable the TCP input.

module(load="imtcp")
input(type="imtcp" port="514")

Two warnings, both mechanical. Plain syslog carries no encryption and no authentication, so anything that can reach port 514 can inject log lines that look exactly like yours. Bind it to a private network or a VPN and firewall the port. Second, the default action queue lives in memory, so when the collector is unreachable the queue fills and messages are dropped with no copy kept. rsyslog documents a disk assisted queue for that case in its reliable forwarding tutorial.

Why the ELK stack does not fit on a small VPS

ELK means Elasticsearch for storage and search, Logstash for the ingestion pipeline, and Kibana for the interface. The floor is the JVM heap, and it is set before any log arrives.

Elastic's documentation says to set the heap to no more than 50% of the total memory available to each Elasticsearch node, because the process also uses off heap buffers and depends on the operating system file cache to read index files quickly. So a 2 GB heap implies a 4 GB machine, before Kibana, and before whatever the server was actually bought to run. Elastic also states that Elasticsearch sizes the heap automatically from the node's roles and total memory, which means a small box gets a small heap and then spends its life garbage collecting.

Logstash is the part that breaks a small budget outright. Elastic's own JVM settings page recommends a heap of no less than 4GB and no more than 8GB for typical ingestion. That is the whole of a 4 GB VPS, for one process in the middle of the pipeline.

[
  {
    "label": "Loki plus Alloy (no JVM)",
    "documented_heap_mb": 0
  },
  {
    "label": "OpenSearch demo compose",
    "documented_heap_mb": 512
  },
  {
    "label": "OpenSearch production example",
    "documented_heap_mb": 2048
  },
  {
    "label": "Logstash recommended minimum",
    "documented_heap_mb": 4096
  }
]

Those are the values each project publishes in its own documentation. They are not measurements taken on a test box, and your workload will move them. OpenSearch's sample compose file sets 512 MB per node for a demo and 2048 MB in its production example, while Logstash's recommended lower bound is 4096 MB. The heap column reads 0 for Loki and Alloy because they are Go programs with no JVM heap to reserve. That is the whole difference in one number: a JVM component takes its reservation whether or not any logs arrive.

If you want the Elastic stack on one small server anyway, drop Logstash and ship straight into Elasticsearch with a light collector. Logstash exists to parse and transform at volume, and on one box you can do that work at the edge or skip it.

Both Elasticsearch and OpenSearch also need vm.max_map_count raised to 262144, because they memory map index files and the Linux default limit is too low for them. A container that exits seconds after startup on a fresh box is usually this and nothing else.

OpenSearch or Elasticsearch: which one can you deploy?

The short licence history, because it decides what you are allowed to run. In January 2021 Elastic moved Elasticsearch and Kibana off Apache 2.0 to a dual SSPL (server side public license) and Elastic License 2.0 model. AWS forked the last Apache 2.0 code as OpenSearch, which stays Apache 2.0. In September 2024 Elastic added AGPLv3 (GNU Affero General Public License version 3) as another option for the free source code. For one person self hosting on one VPS, every one of those permits what you are doing. The licences bite when you offer the software to other people as a managed service.

The practical difference on a small box is smaller than the history suggests, because both are the same engine underneath. The names differ: index lifecycle is ISM (index state management) in OpenSearch and ILM (index lifecycle management) in Elasticsearch. As of August 2026, OpenSearch 2.12 and later refuse to start without an admin password set at first run.

sudo sysctl -w vm.max_map_count=262144
printf 'vm.max_map_count = 262144\n' | sudo tee /etc/sysctl.d/99-opensearch.conf
docker run -d -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" \
  -e "OPENSEARCH_INITIAL_ADMIN_PASSWORD=<custom-admin-password>" \
  opensearchproject/opensearch:latest

The sysctl -w line applies the setting now and the file in /etc/sysctl.d/ is the half that survives a reboot. Check the container came up with curl -k -u admin:<password> https://localhost:9200. It answers over https using a demo certificate, so -k skips verification, and a healthy reply is a small JSON block naming the cluster and the version. The OpenSearch install page also tells Docker Desktop users to allow the host at least 4 GB of memory, which is a fair signal of what the process expects to have.

How Loki stays small: labels instead of a full text index

Loki keeps one index over labels and stores the log lines as compressed chunks. A query selects streams first and filters text second. {unit="ssh.service"} |= "Failed password" picks the stream by its label, then scans those chunks for the string. Nothing indexes the body of a line, so ingestion stays cheap and there is no inverted index to keep in memory. The cost moves to query time, and that is a good exchange when you usually know which service you are looking at.

Grafana's documentation puts monolithic mode, meaning all of Loki in one process with -target=all, at small read and write volumes of up to approximately 20GB per day. One VPS sits well inside that.

The trap is label cardinality. Every distinct combination of label values is one stream, and stream count drives Loki's memory and index size. A label holding a client IP address or a request identifier creates a stream per value, so a busy web server can produce tens of thousands of streams in a day and the process grows until the kernel stops it. Keep labels to values you could count on paper: unit, host, job, level. Put the variable detail in the line itself, where a filter expression finds it at query time.

Install Loki and Alloy on one VPS

Two processes do the work. Loki stores and answers queries. Grafana Alloy reads logs and pushes them. Promtail used to be the shipper, and it reached end of life on 2 March 2026, so new installs use Alloy and Loki's own Docker example now ships an Alloy config.

wget https://raw.githubusercontent.com/grafana/loki/v3.7.0/cmd/loki/loki-local-config.yaml -O loki-config.yaml

Read that file before using it. It sets path_prefix: /tmp/loki with chunks under /tmp/loki/chunks, which is correct for a demo and wrong for a server: nothing under the container's /tmp survives the container being recreated, so your history disappears at the next image update. Point it at a path you mount.

common:
  instance_addr: 127.0.0.1
  path_prefix: /loki
  storage:
    filesystem:
      chunks_directory: /loki/chunks
      rules_directory: /loki/rules
  replication_factor: 1
  ring:
    kvstore:
      store: inmemory

docker volume create loki-data
docker run --name loki -d \
  -v $(pwd):/mnt/config -v loki-data:/loki \
  -p 127.0.0.1:3100:3100 \
  grafana/loki:3.7.0 -config.file=/mnt/config/loki-config.yaml
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:3100/ready

That last command should print 200, because /ready returns HTTP 200 once Loki is ready to accept traffic. Anything else means the process is still starting or the config was rejected, and docker logs loki says which. Two details in the run command are deliberate. The port is published on 127.0.0.1 only, because the sample config carries auth_enabled: false and Loki ships no user authentication of its own, so anything able to reach port 3100 can read every log and write fake ones. Keep it on loopback, or behind a VPN or an authenticating reverse proxy. The named volume matters because the image runs as the user loki with UID 10001, so a bind mounted host directory owned by root is not writable by the container.

sudo apt-get update && sudo apt-get install -y gpg wget
sudo mkdir -p /etc/apt/keyrings/
sudo wget -O /etc/apt/keyrings/grafana.asc https://apt.grafana.com/gpg-full.key
echo "deb [signed-by=/etc/apt/keyrings/grafana.asc] https://apt.grafana.com stable main" \
  | sudo tee /etc/apt/sources.list.d/grafana.list
sudo apt-get update
sudo apt-get install -y alloy

Alloy reads /etc/alloy/config.alloy. This one takes the system journal and one set of files, and pushes both to the local Loki.

loki.write "local" {
  endpoint {
    url = "http://127.0.0.1:3100/loki/api/v1/push"
  }
}

loki.relabel "journal" {
  forward_to = []
  rule {
    source_labels = ["__journal__systemd_unit"]
    target_label  = "unit"
  }
}

loki.source.journal "read" {
  forward_to    = [loki.write.local.receiver]
  relabel_rules = loki.relabel.journal.rules
  labels        = {job = "systemd-journal", host = "app-01"}
}

local.file_match "nginx" {
  path_targets = [{"__path__" = "/var/log/nginx/*.log", "job" = "nginx", "host" = "app-01"}]
}

loki.source.file "nginx" {
  targets    = local.file_match.nginx.targets
  forward_to = [loki.write.local.receiver]
}

The relabel rule copies the journal field __journal__systemd_unit into a label named unit, which is what makes {unit="ssh.service"} work later. Without that rule the unit name is inside the entry rather than in a label, so you cannot select on it and every query has to scan everything.

sudo systemctl reload alloy
systemctl show -p User alloy
sudo journalctl -n 5
sudo -u alloy journalctl -n 5

This is where most setups stall. Alloy runs as its own service account, not as root, and reading the system journal requires membership of the systemd-journal group, while files under /var/log/nginx belong to group adm on Debian and Ubuntu. Substitute the account that systemctl show printed into the last command. If it returns far fewer entries than the root run, that account cannot read the system journal, and Loki stays empty however correct your config is. Add the groups and restart with sudo usermod -aG systemd-journal,adm alloy followed by sudo systemctl restart alloy.

curl -G -s "http://127.0.0.1:3100/loki/api/v1/query_range" \
  --data-urlencode 'query={job="systemd-journal"}' \
  --data-urlencode 'limit=5' | jq '.data.result | length'

A number above 0 means streams exist with that label and hold entries. A 0 means nothing has arrived under that label yet. One default explains a common false alarm: loki.source.journal sets max_age to 7h, so a fresh start reads the last seven hours of the journal and nothing older. For a human interface, run Grafana on the same box and point a Loki data source at http://127.0.0.1:3100. Container logs need a different source: Alloy discovers running Docker containers and tails them, which is what Loki's own getting started example does, and on a single node k3s cluster on a VPS that job moves to the pod log directory the kubelet writes.

Retention: pick the day your logs die

Almost nobody picks a retention period until the disk fills, and then they pick it at 3 a.m. with the service down. Decide it on day one from two questions: how far back do you actually look, and what must you still have during an incident review next month. For a single server, 14 to 30 days answers both.

Loki deletes nothing at all until you enable the compactor. Retention is off by default, which surprises people whose volume filled while retention_period sat in the config doing nothing.

limits_config:
  retention_period: 744h

compactor:
  working_directory: /loki/retention
  compaction_interval: 10m
  retention_enabled: true
  retention_delete_delay: 2h
  retention_delete_worker_count: 150
  delete_request_store: filesystem

744h is 31 days. Four documented rules govern that block:

• Retention is applied by the compactor, and Grafana's documentation says to run the compactor as a single instance. On one VPS that happens by itself.
• The minimum retention period is 24h, and retention works only when the index period is 24h. The sample schema_config already uses period: 24h, so leave it alone.
• delete_request_store is required once retention_enabled is true. It names the store holding delete requests, so on a filesystem backed single node it matches the object_store: filesystem already in the schema.
• Chunks are marked first and removed after retention_delete_delay, which is 2h here, so free space returns later than the policy implies. Do not judge the setting by df five minutes after a reload.

OpenSearch deletes whole indexes rather than individual lines, which is why log indexes are created per day. An ISM policy walks an index through states and deletes it once it is old enough, and an ism_template attaches the policy to new indexes so you never have to remember.

{
  "policy": {
    "description": "delete log indexes after 14 days",
    "default_state": "hot",
    "states": [
      {
        "name": "hot",
        "actions": [],
        "transitions": [
          { "state_name": "delete", "conditions": { "min_index_age": "14d" } }
        ]
      },
      {
        "name": "delete",
        "actions": [ { "delete": {} } ],
        "transitions": []
      }
    ],
    "ism_template": { "index_patterns": ["logs-*"], "priority": 100 }
  }
}

Whichever system you run, a retention number is only as good as the free space check behind it. Deleting at 14 days does not save you if 10 days of logs already fill the volume, so pair the policy with disk health monitoring on a VPS and an alert at 80% used.

How much disk per GB of logs

The honest answer depends on your lines and your fields, so measure it on your own data rather than trusting any published ratio. The mechanisms are different enough to predict the direction. OpenSearch and Elasticsearch write an inverted index over every indexed field alongside the stored document, so what lands on disk is larger than the raw text, and every replica multiplies it. On a single node set the replica count to 0, because a replica shard on the same node cannot survive that node failing: leaving it at 1 doubles the disk and pins cluster health at yellow forever. Loki writes compressed chunks plus a small label index, so its footprint tracks the compressed size of the lines.

sudo du -sh /var/lib/docker/volumes/loki-data/_data
curl -k -u admin:<password> "https://localhost:9200/_cat/indices?v&h=index,docs.count,store.size"

Run whichever applies on two consecutive days. The difference is your daily growth. Multiply by your retention days, add about 30% headroom for compaction and merges, then compare that against the volume. If it does not fit, cut retention before you buy disk, because a bigger volume only moves the same problem out by a few weeks.

What breaks first on a small box

Memory goes first. The kernel OOM (out of memory) killer picks a large process, and the largest process on a logging box is the JVM. journalctl -k | grep -i "killed process" shows the kill with the process name in brackets. The victim is not always the log stack: sshd or your database can be chosen instead, which is how a logging experiment takes down the application you wanted logs from. Give containers explicit ceilings so the failure lands where you chose it to, which is what memory limits in Docker Compose are for.

Disk goes second, and search engines fail in a specific and recognisable way. Elasticsearch and OpenSearch watch disk usage at several levels. The low watermark sits at 85% and the high watermark at 90%. At the flood stage of 95%, every index with a shard on that node receives the block index.blocks.read_only_allow_delete, and writes then fail with blocked by: [FORBIDDEN/12/index read-only / allow delete (api)]. The block is released once usage falls back below the high watermark. Free space first, then clear the block by hand only if it lingers.

curl -k -u admin:<password> -X PUT "https://localhost:9200/_all/_settings" \
  -H 'Content-Type: application/json' \
  -d '{"index.blocks.read_only_allow_delete": null}'

Loki fails more quietly. It has no read-only mode to fall into, so a full volume shows up as failed pushes at the sender and gaps in query results, and the cardinality problem arrives as a slow climb in memory rather than an error. Watch the size of the chunks directory on a schedule, not after an incident.

The last failure is putting the wrong data in. A log system is not a metrics system: CPU load sampled every 10 seconds and stored as text is expensive to keep and awkward to graph, and that job belongs to something like a Zabbix monitoring server on Ubuntu 24.04. Application exceptions want grouping, deduplication and a stack trace view, which is the job of a self-hosted error tracker. Knowing the site is down at all is a separate job again, answered by an uptime and status page such as Uptime Kuma. Keep the log system for lines of text a person will read.

FAQ

Do I need Elasticsearch to search my server logs?

Not for one or two servers. journalctl already filters by unit, priority, boot and time range, and rotated files answer to grep and zgrep. A search cluster earns its memory when you have many machines, when you need free text search across all of them at once, or when several people need a shared interface. Below that, journald with a size limit and a retention time does the same job for no extra RAM.

How much RAM do I need for self-hosted log management?

Use each project's published figures rather than a rule of thumb. Loki and Alloy are Go programs with no heap to reserve up front, and Grafana documents monolithic Loki at up to approximately 20GB per day. OpenSearch's sample compose sets 512 MB of heap for a demo and 2 GB in its production example, and Elastic says heap must stay at or below 50% of total memory, so a 2 GB heap means a 4 GB machine before Kibana. Logstash's documentation recommends no less than 4GB of heap on its own. Those are documented settings, not benchmarks, so measure your own load before sizing a plan.

What is the real difference between Loki and OpenSearch for logs?

The index model. Loki indexes labels only and keeps the log body as compressed chunks that are scanned at query time, so writes are cheap and queries cost more when they are broad. OpenSearch indexes the content of the fields, so arbitrary full text search is fast and both memory and disk pay for the index. Choose Loki when you know which service and time window you want. Choose OpenSearch when you need to search text you cannot predict in advance.

How long should I keep logs on a VPS?

Pick the number before the disk picks it for you. Set it in exactly one place per system: MaxRetentionSec= and SystemMaxUse= for journald, retention_period with the compactor enabled for Loki, and an ISM policy with min_index_age for OpenSearch. For most single server setups, 14 to 30 days covers debugging and incident review. Anything you must keep longer belongs in a copy stored off the box, because a log kept only on the server that failed is not a record.

Is Promtail still the way to ship logs to Loki?

No. Promtail reached end of life on 2 March 2026, and Grafana Alloy replaces it. Loki's own Docker install example now ships an Alloy configuration, and Grafana provides a converter that turns an existing Promtail config into Alloy syntax. An existing Promtail install keeps running, but it receives no fixes, so treat migration as maintenance rather than an upgrade you can postpone indefinitely.