Self-host a RustDesk relay server

What a self-hosted RustDesk relay server is

A self-hosted RustDesk relay server is two daemons on one VPS. hbbs is the ID and rendezvous server: it registers every client ID and introduces two clients to each other. hbbr is the relay server: it carries the session bytes, but only for the sessions that could not be made to talk directly. Most guides install both, get you paired, and stop there. What follows is the rest of the job: the key that is your access control, the ports, the upgrade, and the bandwidth.

Both daemons ship in the same image, rustdesk/rustdesk-server, and both read the same Ed25519 key pair out of the same directory. Ed25519 is a public key signature scheme. That key pair decides which clients your server will speak to, and there is no user database behind it.

hbbs and hbbr: which daemon costs you bandwidth

hbbs traffic is small and constant: ID registration and heartbeats, plus the short exchange that introduces two peers. It runs all day and costs almost nothing.

hbbr traffic is the session itself. Screen frames go one way, keyboard and mouse go the other, and every relayed byte arrives at your VPS and then leaves it again. If your provider meters egress only, a relayed session costs you about the session rate. If it meters total transfer, it costs about double that.

The relay is a fallback, not the normal path. hbbs first tries to connect the two clients directly, using hole punching through whatever NAT (network address translation) sits in front of each of them. When that works, the session never touches hbbr and your transfer allowance is untouched. When one side is behind a NAT that assigns a new port per destination, or a firewall that drops the punched path, the session falls back to hbbr and every frame crosses your VPS.

One environment variable removes the choice. ALWAYS_USE_RELAY=Y on hbbs forces every session through hbbr. The RustDesk documentation shows it in one of its Compose examples, so it gets copied a lot. It makes connections more predictable and it makes your egress real. Set it because you decided to, not because you pasted it.

Which ports does a self-hosted RustDesk server need

The port numbers below were checked against the RustDesk server documentation and the rustdesk-server repository on 17 August 2026.

• TCP 21115, on hbbs: the NAT type test.
• UDP 21116, on hbbs: ID registration and heartbeat. Without this the client never comes online, whatever else is open.
• TCP 21116, on hbbs: TCP hole punching and the connection service.
• TCP 21117, on hbbr: the relay. This is the port that moves session data, so it is the port that costs you money.
• TCP 21118 on hbbs and TCP 21119 on hbbr: WebSocket, used by the browser client. Leave both closed if you do not use it.
• TCP 21114 is the web console in RustDesk Server Pro. The open source build does not listen on it.

Install hbbs and hbbr with the image tag pinned

services:
  hbbs:
    container_name: hbbs
    image: rustdesk/rustdesk-server:1.1.16
    command: hbbs
    volumes:
      - ./data:/root
    network_mode: "host"
    depends_on:
      - hbbr
    restart: unless-stopped

  hbbr:
    container_name: hbbr
    image: rustdesk/rustdesk-server:1.1.16
    command: hbbr -k _
    volumes:
      - ./data:/root
    network_mode: "host"
    restart: unless-stopped

mkdir -p ~/rustdesk
cd ~/rustdesk
# save the block above as ~/rustdesk/compose.yml before this line
sudo docker compose up -d
sudo docker compose ps

Both services should read running. Confirm the listeners exist before you touch the firewall.

sudo ss -tlnp | grep -E ':2111[5-7]'
sudo ss -ulnp | grep ':21116'

You should see TCP listeners on 21115, 21116 and 21117, and a UDP listener on 21116. A missing UDP line means hbbs is not running, because that listener is the one clients register against.

Four things in that file are deliberate. The tag is 1.1.16, the current release as of August 2026, published on 20 July 2026, rather than latest, because latest means whatever was pushed most recently and a docker compose pull six months from now can hand you a server you never tested. network_mode: "host" binds the host interfaces directly, which is what the RustDesk documentation recommends and which decides how your firewall behaves. ./data:/root maps the image working directory onto the host, so the key pair lands somewhere you can back up. And hbbr -k _ is the one change from the upstream example, because the default leaves your relay open to anyone. If Compose itself is new to you, running Docker Compose on a VPS covers the file format and the lifecycle commands.

If hbbr ever moves to a second server, hbbs has to be told where it went: pass -r relay.example.com:21117, or set the RELAY-SERVERS environment variable. On a single box you do not need it.

The Ed25519 key pair is the access control

On first start hbbs generates id_ed25519 and id_ed25519.pub in its working directory. With the mount above, both files appear on the host.

ls -l ~/rustdesk/data/
sudo cat ~/rustdesk/data/id_ed25519.pub

id_ed25519.pub holds one base64 string. That string goes into the Key field on every client. id_ed25519 is the private half and never leaves the server. The public key is not a secret, because it is copied into every client config anyway. The private key is a secret: anyone holding it can stand up a server your clients would trust.

Back both files up now, before you have twenty clients configured.

sudo tar czf ~/rustdesk-keys.tgz -C ~/rustdesk/data id_ed25519 id_ed25519.pub
sudo chmod 600 ~/rustdesk-keys.tgz

Copy that archive off the box. Here is why it matters more than any other step. Delete ~/rustdesk/data, or rebuild on a new VPS without copying it, and hbbs generates a fresh key pair on the next start. Every client still carries the old public key, so hbbs refuses it and the client drops offline. Run sudo cat ~/rustdesk/data/id_ed25519.pub and compare it against the Key field on any client: the two strings no longer match, and that mismatch is the entire failure. Fixing it means editing the settings on every machine by hand, including the machines you were relying on RustDesk to reach.

The key is not the session password, and confusing the two leads people to skip one. The key decides which clients your server will talk to. The permanent password or one time code on the controlled machine decides who may open a session on that machine. You need both, and having one does not make up for a weak other.

Why an unauthenticated relay is a problem

By default hbbr checks nothing. The RustDesk configuration documentation states it directly: an empty key allows clients without a matching key to use the relay. The empty default exists so that new users do not hit key mismatch failures on the first run. The cost is that anyone who finds your address on TCP 21117 can push their session traffic through your VPS, on your transfer allowance, from your IP address.

command: hbbr -k _ closes that. The _ argument tells hbbr to load a key pair from its working directory, and because both containers mount the same ./data, that is the pair hbbs already generated. Nothing gets copied by hand, so nothing can drift.

The shared volume is the part people get wrong. Give hbbr its own directory and it generates a different key pair. hbbs and hbbr then disagree, every relayed session fails, and direct sessions keep working. The symptom is confusing: RustDesk reaches some peers and not others, depending on whether hole punching happened to succeed. One ls -l ~/rustdesk/data/ showing a single id_ed25519 pair rules it out.

Point the clients at your server

On each machine, open RustDesk, then Settings, then Network, then ID/Relay Server.

• ID Server: your hostname, for example rustdesk.example.com. The client uses port 21116 unless you write a different one.
• Relay Server: leave empty when hbbr runs on the same host as hbbs.
• API Server: leave empty. The open source server does not provide one.
• Key: the base64 string from id_ed25519.pub, pasted exactly, with no trailing space.

The main window should then report that the client is ready. If it does not, UDP 21116 is not reaching hbbs, because registration and heartbeat run over UDP and nothing else brings the ID online.

Restrict the ports so the relay is not an open service

Because the containers use host networking, there is no Docker NAT rule in front of them, so ufw rules apply the way you expect.

sudo ufw allow 22/tcp
sudo ufw allow 21115:21117/tcp
sudo ufw allow 21116/udp
sudo ufw enable
sudo ufw status numbered

Add 21118:21119/tcp only if you run the browser client. Keep a second SSH session open while you enable ufw, so a mistake in the SSH rule does not lock you out of your own server. The ufw firewall basics for a VPS covers the default policies and the rule ordering.

Now the trap. If you switch to publishing ports with a ports: block, which the alternative RustDesk supervisor image example uses, Docker writes its own DNAT rules and packets reach the container without passing through the chain your ufw rules live in. A ufw deny on 21117 then has no effect, and the relay is open to the internet while ufw status claims otherwise. Docker published ports bypass ufw explains the chain order. Host networking avoids the problem entirely. If you do publish a port, bind it to one address, as in "127.0.0.1:21118:21118" behind a reverse proxy.

Restricting by source address works only when your clients have stable addresses.

sudo ufw allow from 203.0.113.0/24 to any port 21115:21117 proto tcp
sudo ufw allow from 203.0.113.0/24 to any port 21116 proto udp

Laptops on hotel networks do not have stable addresses, which is exactly why the key on hbbr does more real work here than the firewall does.

Upgrading a stack that holds your key

The key lives in the bind mount, not inside the container, so an upgrade is safe as long as you leave ./data alone.

1. Back up the data directory first: sudo tar czf ~/rustdesk-data-backup.tgz -C ~/rustdesk data.
2. Read the release notes for the new tag on the rustdesk-server releases page.
3. Edit compose.yml and change both image: lines to the new tag.
4. Run sudo docker compose pull, then sudo docker compose up -d.
5. Run sudo cat ~/rustdesk/data/id_ed25519.pub and confirm the string is the one your clients already hold.

Step 5 is the check that matters, because a changed key is silent on the server and breaks every client at the same moment. Rolling back is putting the old tag back and running up -d again, and that only works because you pinned it: with latest, docker compose pull moved the name onto the new image, so there is no tag left that names the old one.

The usual way to lose the key is not docker compose down, which leaves a bind mount alone. It is migrating to a new VPS and copying only compose.yml. Copy ./data with it.

Monitor egress on a plan with a transfer allowance

hbbr is the only part of this stack that can consume a transfer allowance. RustDesk's FAQ puts one relayed connection on a 1920x1080 screen between 30 KB/s and 3 MB/s, and puts plain office work at around 100 KB/s. Those are published figures for a single session, not a measurement of your setup. Multiplied out over sixty hours in a month, two hours a day, they look like this.

[
  {
    "label": "Low end, 30 KB/s",
    "gb_per_month": 6.5
  },
  {
    "label": "Office work, 100 KB/s",
    "gb_per_month": 21.6
  },
  {
    "label": "High end, 3 MB/s",
    "gb_per_month": 648
  }
]

At the office work rate, one session costs about 21.6 GB a month, which no plan will notice. At the top of the published range the same sixty hours cost 648 GB, and two concurrent sessions at that rate clear a 1 TB allowance inside the month. The low end is 6.5 GB. Gigabytes here are 1000 MB, which is how transfer allowances are normally counted.

docker stats will not break this down for you, because a container using host networking shares the host network namespace, so its counters are the host's counters. Two other tools do work. vnstat measures the whole box:

sudo apt install -y vnstat
sudo systemctl enable --now vnstat
vnstat -m

An nftables counter measures the relay specifically:

sudo nft add table inet relaymeter
sudo nft add chain inet relaymeter out '{ type filter hook output priority 0 ; }'
sudo nft add rule inet relaymeter out tcp sport 21117 counter
sudo nft list table inet relaymeter

That rule carries no verdict, so it counts packets and bytes without changing what is allowed, and it sits in a table of its own so it does not disturb ufw. It is not persistent: put the same lines in /etc/nftables.conf if you want it back after a reboot. The counter grows only while a session is actually being relayed. A counter that keeps climbing while none of your own machines are connected means somebody else found your relay, which is the case that hbbr -k _ exists to prevent.

hbbr also has rate limits you can lower. SINGLE_BANDWIDTH defaults to 128 Mb/s per relay connection and TOTAL_BANDWIDTH to 1024 Mb/s across all of them. Setting SINGLE_BANDWIDTH=8 caps one session near 1 MB/s. That limits the speed, not the monthly total, so treat it as a way to stop one session saturating the link rather than as a budget control.

When you do not need a relay at all

For a personal setup, the honest answer is that you may not need any of this. Put both machines on a mesh VPN and connect straight to the tunnel address. There is no hbbs, no hbbr, no relay egress, and no container on a VPS to upgrade.

On the machine you want to control, enable direct IP access in RustDesk's security settings. The port field defaults to 21118. Check that it is listening before you try to connect:

ss -tlnp | grep 21118

Then connect to that peer's VPN address instead of an ID. RustDesk's FAQ says of this mode that the connection is unencrypted, so run it inside the tunnel and never across the open internet. The tunnel is what supplies the encryption.

Pick by who owns the machines. A self-hosted hbbs and hbbr is right when you support machines that are not yours, or people who will never install a VPN client, because their side of the setup is an ID and a password. A mesh VPN with direct IP access is right when every machine belongs to you and can carry a key. WireGuard compared with Tailscale covers the two usual ways to build that mesh, and running a remote desktop on a Linux VPS covers the other case, where the machine you want a screen on is the server itself.

FAQ

Does every RustDesk session go through my relay?

No. hbbs tries to connect the two clients directly first, using hole punching through the NAT in front of each of them. Only the sessions where that fails drop back to hbbr, and only those cost you bandwidth. The exception is ALWAYS_USE_RELAY=Y on hbbs, which forces every session through hbbr regardless of whether a direct path was available. If that variable is set in your Compose file, every byte of every session lands on your transfer bill.

Where is the RustDesk server key stored, and what if I lose it?

hbbs generates id_ed25519 and id_ed25519.pub in its working directory on first start. That directory is /root inside the official image, so with the volume mount shown above the files appear in ./data on the host. Back up both files off the server. If they are lost, hbbs generates a new pair on the next start, and every client still holding the old public key is refused. There is no recovery beyond editing the Key field on every client by hand.

Which ports do I need to open for a self-hosted RustDesk server?

TCP 21115, 21116 and 21117, plus UDP 21116. hbbs uses 21115 for the NAT type test, and 21116 for ID registration and heartbeat over UDP and for hole punching over TCP. hbbr uses 21117 for the relay. TCP 21118 and 21119 are the WebSocket ports for the browser client, so leave them closed if you do not use it. TCP 21114 belongs to the Pro web console and the open source build does not need it.

Can strangers use my self-hosted RustDesk relay?

Yes, if you run hbbr with its default configuration. The RustDesk documentation states that an empty key allows clients without a matching key to use the relay, so anyone who learns your hostname and port 21117 can push traffic through your server. Run hbbr with -k _ so that it loads the same key pair hbbs generated in the shared ./data volume. After that, only clients configured with your public key can relay through you.