How to Self-Host RustDesk Relay Server for VPS

Wetin self-hosted RustDesk relay server be

Self-hosted RustDesk relay server na two daemons for one VPS. hbbs na the ID and rendezvous server: e dey register every client ID and connect two clients to each other. hbbr na the relay server: e dey carry the session bytes, but na only for sessions wey no fit connect directly. Most guides install both, make dem pair, then stop there. The remaining work na the key wey control access, the ports, the upgrade, and the bandwidth.

Both daemons dey ship for the same image, rustdesk/rustdesk-server, and both dey read the same Ed25519 key pair from the same directory. Ed25519 na public key signature scheme. That key pair decide which clients your server go talk to, and no user database dey behind am.

hbbs and hbbr: which daemon dey chop your bandwidth

hbbs traffic small and steady: ID registration and heartbeats, plus the short exchange wey dey introduce two peers. E dey run all day and e cost almost nothing.

hbbr traffic na the session itself. Screen frames dey go one side, keyboard and mouse dey go the other side, and every relayed byte first dey enter your VPS, then e leave am again. If your provider dey measure only egress, one relayed session go cost you around the session rate. If e dey measure total transfer, e go cost around double that.

The relay na fallback, e no be the normal path. hbbs first dey try connect the two clients directly, using hole punching through any NAT (network address translation) wey dey in front of each one. When this one work, the session no go touch hbbr and your transfer allowance remain intact. When one side dey behind NAT wey dey assign new port for every destination, or firewall wey dey drop the punched path, the session go fall back to hbbr and every frame go cross your VPS.

One environment variable dey remove the choice. ALWAYS_USE_RELAY=Y for hbbs go force every session pass through hbbr. RustDesk documentation show am for one of its Compose examples, so people dey copy am plenty. E make connections more predictable and e make your egress real. Set am because na you decide, no be because you paste am.

Self-hosted RustDesk server need which ports

We check the port numbers below against RustDesk server documentation and the rustdesk-server repository on 17 August 2026.

• TCP 21115, for hbbs: e dey do NAT type test.
• UDP 21116, for hbbs: e dey handle ID registration and heartbeat. If this one no open, client no go ever come online, no matter wetin else dey open.
• TCP 21116, for hbbs: e dey handle TCP hole punching and connection service.
• TCP 21117, for hbbr: na relay port. Na this port dey move session data, so na this port fit cost you money.
• TCP 21118 for hbbs and TCP 21119 for hbbr: na WebSocket, wey browser client dey use. Leave both ports closed if you no dey use am.
• TCP 21114 na web console for RustDesk Server Pro. Open source build no dey listen on this port.

hbbs and hbbr install with image tag pin

services:
  hbbs:
    container_name: hbbs
    image: rustdesk/rustdesk-server:1.1.16
    command: hbbs
    volumes:
      - ./data:/root
    network_mode: "host"
    depends_on:
      - hbbr
    restart: unless-stopped

  hbbr:
    container_name: hbbr
    image: rustdesk/rustdesk-server:1.1.16
    command: hbbr -k _
    volumes:
      - ./data:/root
    network_mode: "host"
    restart: unless-stopped

mkdir -p ~/rustdesk
cd ~/rustdesk
# save the block above as ~/rustdesk/compose.yml before this line
sudo docker compose up -d
sudo docker compose ps

Both services suppose read running. Confirm say the listeners dey exist before you touch firewall.

sudo ss -tlnp | grep -E ':2111[5-7]'
sudo ss -ulnp | grep ':21116'

You suppose see TCP listeners for 21115, 21116 and 21117, plus UDP listener for 21116. If UDP line no dey, e mean hbbs no dey run, because na that listener clients dey register against.

Four things for that file na deliberate. The tag na 1.1.16, the current release as of August 2026, wey dem publish on 20 July 2026, instead of latest, because latest mean anything wey dem push most recently, and docker compose pull six months from now fit give you server wey you never test. network_mode: "host" bind the host interfaces directly, na wetin RustDesk documentation recommend, and na this dey decide how your firewall go behave. ./data:/root map the image working directory onto the host, so the key pair go land for place wey you fit back up. And hbbr -k _ na the only change from the upstream example, because the default leave your relay open to anybody. If Compose still new to you, run Docker Compose for VPS explain the file format and lifecycle commands.

If hbbr ever move go second server, you must tell hbbs where e go: pass -r relay.example.com:21117, or set the RELAY-SERVERS environment variable. For one server, you no need am.

Ed25519 key pair na the access control

When hbbs start for the first time, e go generate id_ed25519 and id_ed25519.pub inside im working directory. With the mount wey dey above, both files go show for the host.

ls -l ~/rustdesk/data/
sudo cat ~/rustdesk/data/id_ed25519.pub

id_ed25519.pub hold one base64 string. Na this string you go put for the Key field on every client. id_ed25519 na the private half, and e no suppose ever comot from the server. Public key no be secret, because you go copy am into every client config anyway. Private key na secret: anybody wey get am fit set up server wey your clients go trust.

Back up both files now, before you configure twenty clients.

sudo tar czf ~/rustdesk-keys.tgz -C ~/rustdesk/data id_ed25519 id_ed25519.pub
sudo chmod 600 ~/rustdesk-keys.tgz

Copy that archive comot from the box. Make you know why this one matter pass every other step. If you delete ~/rustdesk/data, or rebuild for new VPS without copying am, hbbs go generate fresh key pair when e start next time. Every client still get the old public key, so hbbs go reject am and the client go go offline. Run sudo cat ~/rustdesk/data/id_ed25519.pub and compare am with the Key field for any client: the two strings no go match again, and na this mismatch cause the whole failure. To fix am, you go edit settings by hand for every machine, including the machines wey you dey depend on RustDesk to reach.

The key no be session password, and if you mix the two up, you fit skip one. The key decide which clients your server go talk to. Permanent password or one time code for the controlled machine decide who fit open session for that machine. You need both, and having one no go compensate for the other one wey weak.

Why unauthenticated relay na problem

By default, hbbr no dey check anything. RustDesk configuration documentation talk am directly: empty key allow clients wey no get matching key to use the relay. This empty default dey there so new users no go hit key mismatch failures for first run. The cost na say anybody wey find your address for TCP 21117 fit push their session traffic through your VPS, use your transfer allowance, and send am from your IP address.

command: hbbr -k _ close this gap. The _ argument tell hbbr to load key pair from its working directory. Since both containers mount the same ./data, na the pair wey hbbs don already generate be that. Nobody need copy anything by hand, so nothing fit drift.

The shared volume na the part wey people dey get wrong. If you give hbbr its own directory, e go generate a different key pair. hbbs and hbbr go then disagree, every relayed session go fail, while direct sessions go continue to work. The symptom dey confusing: RustDesk fit reach some peers but no fit reach others, depending on whether hole punching manage to succeed. One ls -l ~/rustdesk/data/ wey show one id_ed25519 pair rule out this problem.

Point clients dem to your server

For each machine, open RustDesk, then go Settings, then Network, then ID/Relay Server.

• ID Server: your hostname, for example rustdesk.example.com. Client dey use port 21116 unless you write another one.
• Relay Server: leave am empty when hbbr dey run for the same host as hbbs.
• API Server: leave am empty. The open source server no provide one.
• Key: the base64 string from id_ed25519.pub. Paste am exactly, with no trailing space.

The main window suppose report say client ready. If e no, UDP 21116 no dey reach hbbs, because registration and heartbeat dey run over UDP, and nothing else dey bring ID online.

Restrict the ports make relay no be open service

Because containers dey use host networking, Docker no put any NAT rule for front of dem, so ufw rules go apply the way you expect.

sudo ufw allow 22/tcp
sudo ufw allow 21115:21117/tcp
sudo ufw allow 21116/udp
sudo ufw enable
sudo ufw status numbered

Add 21118:21119/tcp only if you dey run browser client. Keep second SSH session open while you enable ufw, so mistake for SSH rule no go lock you out of your own server. The ufw firewall basics for a VPS explain the default policies and how rule ordering dey work.

Now the trap. If you switch to publishing ports with ports: block, wey the alternative RustDesk supervisor image example dey use, Docker go write im own DNAT rules and packets go reach container without passing through the chain wey your ufw rules dey inside. A ufw deny on 21117 no go get any effect, and relay go open to internet while ufw status dey claim otherwise. Docker published ports bypass ufw explain the chain order. Host networking avoid the problem completely. If you publish port, bind am to one address, as e dey for "127.0.0.1:21118:21118" behind a reverse proxy.

Restricting by source address go work only when your clients get stable addresses.

sudo ufw allow from 203.0.113.0/24 to any port 21115:21117 proto tcp
sudo ufw allow from 203.0.113.0/24 to any port 21116 proto udp

Laptops for hotel networks no get stable addresses. Na exactly why the key for hbbr dey do more real work here than the firewall.

How to upgrade stack wey hold your key

The key dey inside bind mount, e no dey inside container. So upgrade safe as long as you leave ./data untouched.

1. Back up the data directory first: sudo tar czf ~/rustdesk-data-backup.tgz -C ~/rustdesk data.
2. Read the release notes for the new tag for rustdesk-server releases page.
3. Edit compose.yml and change both image: lines to the new tag.
4. Run sudo docker compose pull, then sudo docker compose up -d.
5. Run sudo cat ~/rustdesk/data/id_ed25519.pub and confirm say the string na the same one wey your clients already get.

Step 5 na the important check, because if key change, server no go show any clear sign and e go break every client at the same time. To roll back, put the old tag back and run up -d again. This one only work because you pin am. With latest, docker compose pull move the name go the new image, so no tag remain wey name the old one.

The normal way to lose the key no be docker compose down, because e leave bind mount untouched. Na when you migrate go new VPS and copy only compose.yml. Copy ./data together with am.

Monitor egress for plan wey get transfer allowance

hbbr na the only part of this stack wey fit use transfer allowance. RustDesk FAQ talk say one relayed connection for 1920x1080 screen fit use between 30 KB/s and 3 MB/s, and plain office work dey around 100 KB/s. These na published figures for one session, no be measurement from your own setup. If you multiply am for sixty hours each month, two hours every day, e go look like this.

[
  {
    "label": "Low end, 30 KB/s",
    "gb_per_month": 6.5
  },
  {
    "label": "Office work, 100 KB/s",
    "gb_per_month": 21.6
  },
  {
    "label": "High end, 3 MB/s",
    "gb_per_month": 648
  }
]

For office work rate, one session go use about 21.6 GB each month, and no plan go notice am. For the highest published rate, the same sixty hours go use 648 GB, and two sessions wey dey run together for that rate go pass 1 TB allowance inside the month. The low end na 6.5 GB. GB for here mean 1000 MB, na so dem normally count transfer allowance.

docker stats no go break this down for you, because container wey dey use host networking dey share the host network namespace, so na host counters e dey use. Two other tools fit work. vnstat go measure the whole box:

sudo apt install -y vnstat
sudo systemctl enable --now vnstat
vnstat -m

Whole box mean the whole box: if this VPS dey run another thing wey dey move real data, like one of the self-hosted photo servers wey dey pull phone libraries every night, the uploads go enter the same monthly count with your relay traffic. Media server na the same matter from the other side, because something like Halcyon, wey dey turn Jellyfin library to video store wey people fit browse like 90s store dey stream out to whoever dey watch, and that egress dey share the allowance wey your relay dey use.

nftables counter go measure the relay specifically:

sudo nft add table inet relaymeter
sudo nft add chain inet relaymeter out '{ type filter hook output priority 0 ; }'
sudo nft add rule inet relaymeter out tcp sport 21117 counter
sudo nft list table inet relaymeter

That rule no get verdict, so e dey count packets and bytes without changing wetin dem allow, and e dey inside table wey belong to itself so e no go disturb ufw. E no persistent: put the same lines for /etc/nftables.conf if you want am back after reboot. The counter go increase only while session dey actually relay. If counter dey continue climb while none of your own machines dey connected, e mean say somebody else don find your relay. Na the case wey hbbr -k _ dey prevent. Since you no go dey read nft list every morning, put cron job on top am wey go compare byte count with threshold and send push alert when e cross am. your own ntfy server go handle that job.

hbbr get rate limits too, and you fit reduce dem. SINGLE_BANDWIDTH default na 128 Mb/s for each relay connection, while TOTAL_BANDWIDTH default na 1024 Mb/s across all connections. If you set SINGLE_BANDWIDTH=8, one session go get limit near 1 MB/s. This one limit speed, no be monthly total, so use am to stop one session from saturating the link, not as budget control.

When you no need relay at all

For personal setup, honest answer be say you fit no need any of these things. Put both machines for mesh VPN and connect direct to the tunnel address. No hbbs, no hbbr, no relay egress, and no container for VPS wey you need upgrade.

For the machine wey you want control, enable direct IP access for RustDesk security settings. Port field dey default to 21118. Check say e dey listen before you try connect:

ss -tlnp | grep 21118

Then connect to that peer VPN address instead of ID. RustDesk FAQ talk say connection for this mode no get encryption, so run am inside the tunnel and never across open internet. Na the tunnel dey provide the encryption.

Choose based on who own the machines. Self-hosted hbbs and hbbr make sense when you support machines wey no be your own, or people wey no go ever install VPN client, because their side of the setup na ID and password. Mesh VPN with direct IP access make sense when every machine belong to you and fit carry key. WireGuard compared with Tailscale cover the two usual ways to build that mesh, and running remote desktop for Linux VPS cover the other case, where the machine wey you want display on na the server itself.

FAQ

Every RustDesk session dey pass through my relay?

No. hbbs first dey try connect the two clients directly, by using hole punching through the NAT wey dey in front of each client. Na only sessions wey this one fail go fall back to hbbr, and na only dem go use your bandwidth. The exception na ALWAYS_USE_RELAY=Y for hbbs, wey dey force every session pass through hbbr, whether direct path dey available or not. If you set that variable for your Compose file, every byte for every session go enter your transfer bill.

Where RustDesk server key dey stored, and wetin go happen if I lose am?

hbbs dey generate id_ed25519 and id_ed25519.pub for its working directory when e start for the first time. That directory na /root inside the official image, so with the volume mount wey show above, the files go appear for ./data on the host. Back up both files outside the server. If dem lost, hbbs go generate new pair the next time e start, and e go refuse every client wey still get the old public key. No recovery dey apart from editing the Key field by hand for every client.

Which ports I need open for self-hosted RustDesk server?

TCP 21115, 21116 and 21117, plus UDP 21116. hbbs dey use 21115 for the NAT type test, and 21116 for ID registration and heartbeat over UDP, plus hole punching over TCP. hbbr dey use 21117 for the relay. TCP 21118 and 21119 na the WebSocket ports for browser client, so leave dem closed if you no dey use am. TCP 21114 belong to the Pro web console, and the open source build no need am.

Strangers fit use my self-hosted RustDesk relay?

Yes, if you run hbbr with its default configuration. RustDesk documentation talk say empty key allows clients wey no get matching key to use the relay, so anybody wey learn your hostname and port 21117 fit push traffic through your server. Run hbbr with -k _ so e go load the same key pair wey hbbs generate inside the shared ./data volume. After that, na only clients configured with your public key fit relay through your server.