ZFS for VPS: How Much RAM ARC Go Chop?

Wetin ZFS dey give you, and wetin e need

ZFS for FreeBSD and Linux now dey use one codebase, OpenZFS, so the features dey the same for both systems. Server wey dey run ZFS gets checksummed data, snapshots wey no cost anything until data change, replication with zfs send, and compression wey you fit enable with one property. Wetin e need na memory: ARC (adaptive replacement cache) dey claim large part of RAM by default, and for VPS (virtual private server) wey get 2 GB or 4 GB, na that same memory your application need.

This guide dey assess ZFS from rented VPS wey get one or two virtual disks, no be from storage box wey get forty drive bays. The features wey still work for that kind setup na the ones wey worth your time. The parts wey no work for there na the ones you suppose know before you build pool.

OpenZFS for FreeBSD and Linux: one codebase, two packaging stories

FreeBSD don carry ZFS for the base system since FreeBSD 7.0 for 2008, first as experimental feature. Since OpenZFS 2.0 for December 2020, FreeBSD and Linux dey build from the same source tree, so zfs and zpool dey behave the same way for both, and pool wey you create for one fit import for the other.

The reason ZFS na package for Linux and part of the base system for FreeBSD na licensing. OpenZFS dey under CDDL (common development and distribution license). Linux kernel dey under GPL (general public license) version 2. The kernel project see the two licenses as incompatible, so dem no merge ZFS code into mainline Linux, and each distribution decide how e go ship am. FreeBSD no get this conflict, so ZFS just dey there. Na the whole practical story be this: one packaging difference, and nothing wey you need choose side on.

SSD Nodes no offer FreeBSD images, so for server wey you rent here, na the Linux part of this guide apply. If you dey run FreeBSD for another place, a FreeBSD server get ZFS without module wey you need build and without kernel upgrade wey you need survive.

Install ZFS and create a pool

For Ubuntu, the module dey inside the kernel packages, so na the commands you go install.

sudo apt update
sudo apt install -y zfsutils-linux
zfs version

zfs version dey print two lines: the userland version and the kernel module version. If e print only one line, module no load. The package dey inside universe component, wey Ubuntu server images enable by default. If apt no fit find am, run sudo add-apt-repository universe first.

For Debian, the packages dey inside contrib component, and DKMS (dynamic kernel module support) go build the module for your machine. Add contrib to the Components: line inside /etc/apt/sources.list.d/debian.sources, run sudo apt update, then:

sudo apt install -y linux-headers-$(dpkg --print-architecture) zfs-dkms zfsutils-linux

The install go compile the module and print Building initial module for 6.12.0-.... This one fit take some minutes. Remember wetin this mean: every kernel upgrade go rebuild am, and if the build fail, your pool no go import until you fix the problem.

For FreeBSD, nothing dey installed. Enable the service and start am.

sysrc zfs_enable=YES
service zfs start

Now make the pool. First check the stable device paths, because /dev/vdb dey assign names according to detection order, and the names fit change when you attach another volume.

ls -l /dev/disk/by-id/
sudo zpool create -o ashift=12 tank /dev/disk/by-id/virtio-abc123def456
zpool status tank

zpool status suppose print state: ONLINE with your device listed under tank. ashift=12 set the pool smallest block size to 4 KiB. This one match current SSDs and you no fit change am after creation.

Most rented images dey boot from an ext4 root, so ZFS for here na data pool on a second volume, not the root filesystem. Confirm say the device na the one you expect before you build on top of am, because confirming the NVMe disk wey dem sell you takes one minute, while rebuild fit take the whole afternoon.

Checksum dey only repair when pool get redundancy

Every block wey ZFS write get checksum, and every read dey verify am. Detection always dey work. Repair need second copy.

For single-disk pool, ZFS go tell you the truth and stop there. zpool status -v go report am like this:

status: One or more devices has experienced an error resulting in data
        corruption.
action: Restore the file in question if possible.  Otherwise restore the
        entire pool from backup.
errors: Permanent errors have been detected in the following files:

        /tank/data/archive.tar

E name the bad file. ext4 for return those bytes without any comment, so this one already get value. ZFS still no fit fix am, because pool no get second copy wey e fit use repair am.

With mirror, the same read go come from the good side, the bad block go write again, and the event go show for CKSUM column of zpool status. This na self-healing, and e need two devices.

sudo zpool create -o ashift=12 tank mirror /dev/disk/by-id/DISK1 /dev/disk/by-id/DISK2

For VPS, host storage usually get redundancy already, often RAID 10 under the hypervisor. This protect you when drive die. E no tell you when block return wrong, because array no get way to know which copy correct. ZFS know, because e compare the data with checksum wey e write by itself.

If you get one virtual disk and you want some repair ability, sudo zfs set copies=2 tank/important dey store two copies of every block for that dataset on the same disk. E double the space wey that dataset use, e fit survive bad block, and e no do anything when the whole volume disappear.

Scrub go read everything for the pool and verify am.

sudo zpool scrub tank
zpool status tank

Healthy pool dey end with line like scan: scrub repaired 0B in 00:04:11 with 0 errors. Put am for schedule; monthly dey okay for small pool.

systemctl list-unit-files 'zfs-scrub*'
sudo systemctl enable --now zfs-scrub-monthly@tank.timer

Dataset na policy unit

Dataset na filesystem wey dey inside pool, and creating one no cost much, so make you create one for each job. Properties dey inherit down from pool. This mean say you fit set default once, then override am where e matter. For FreeBSD, na so dem normally dey run jails: one dataset for each jail. This allow you snapshot and roll back one jail by itself, and na part of wetin separate jail from Docker container.

sudo zfs create tank/data
sudo zfs create tank/pg
sudo zfs set compression=lz4 tank
sudo zfs set atime=off tank
sudo zfs set quota=20G tank/data
sudo zfs set recordsize=16K tank/pg
zfs get -r compression,compressratio,quota tank

Compression na property wey people dey leave off because dem dey cautious, but that approach no correct. lz4 dey use small CPU and reduce the bytes wey need reach disk, so for data wey fit compress, e normally make reads and writes faster. zstd dey compress more, but e use more CPU, so e fit logs and archives wey you rarely read again. Use zfs get compressratio tank check wetin you dey actually get, and remember say the ratio only count data wey dem write after you set the property.

recordsize na the biggest block wey dataset fit write, and default na 128K. Database wey dey write 8 KiB pages inside 128 KiB records go turn one small write into read of the whole record, change, then write am back. Set recordsize=16K for the database dataset before you load the data, because the property only apply to newly written blocks.

quota na how you go stop one dataset from filling the pool. ZFS pool wey near 100% full dey slow, and cleanup dey become difficult, so leave headroom on purpose.

Snapshot no cost anything until data change

ZFS no dey overwrite live block. E dey write new block and update the pointers, and na this one be copy-on-write. Snapshot na note wey talk say “keep the blocks wey this dataset point to now”, so to take one dey happen immediately and e free.

sudo zfs snapshot tank/data@2026-08-11
zfs list -t snapshot -o name,used,refer -r tank/data

The USED column for snapshot na the space wey na only that snapshot dey hold. E dey start near zero and e dey grow as you change or delete data, because dem no fit release the old blocks again.

To bring file back no need restore step.

ls /tank/data/.zfs/snapshot/
cp /tank/data/.zfs/snapshot/2026-08-11/notes.txt /tank/data/notes.txt

The .zfs directory dey hidden, even from ls -a, until you run sudo zfs set snapdir=visible tank/data. Take the snapshot before you need am, because if you no get one, stray rm -rf go put you for the ext4 recovery path, wey start by unmounting the disk and things go worse from there.

Rollback go throw away everything wey you write after the snapshot.

sudo zfs rollback tank/data@2026-08-11

E go refuse if newer snapshots dey, and -r go destroy those newer snapshots so e fit continue. Read the dataset name two times before you press enter.

Snapshot no be backup. E dey the same pool, for the same volume, on the same server. Failed volume or one zpool destroy go carry the snapshots and the data together. Snapshots protect you from your own rm and from bad upgrade. This one cover plenty real incidents, but snapshots no protect you from anything wey happen to the pool itself. The complete explanation dey here: why VPS snapshot no be backup.

Send and receive: replication for one command

zfs send dey turn snapshot into byte stream for standard output, while zfs receive dey turn that stream back into dataset. The first copy na full send.

sudo zfs snapshot tank/data@daily-2026-08-11
sudo zfs send tank/data@daily-2026-08-11 | ssh backup.example.com "sudo zfs recv -F backup/data"

After that, send only wetin change between two snapshots.

sudo zfs snapshot tank/data@daily-2026-08-12
sudo zfs send -i tank/data@daily-2026-08-11 tank/data@daily-2026-08-12 | ssh backup.example.com "sudo zfs recv backup/data"

The receiving side still need hold the snapshot wey you dey send from. If e no hold am, receive go stop with cannot receive incremental stream: most recent snapshot of backup/data does not match incremental source, because ZFS no get base to apply the difference to. Send from snapshot wey both sides hold, or start again with full send.

Give rights for target instead of using remote root: sudo zfs allow -u backupuser create,mount,receive backup/data.

This na real off-site backup with one condition. The far end must be ZFS pool, because object storage no fit receive stream. If your target na S3-compatible storage or plain Linux host, use tool wey dey talk to am, and restic backups from VPS cover that path.

Why ZFS dey use plenty RAM? The ARC

The ARC (adaptive replacement cache) na ZFS read cache. E dey inside kernel memory instead of normal Linux page cache, so free -h no dey report am under buff/cache. E go show as memory wey dey use. ZFS box wey look almost full usually get warm cache, and na this dey explain most reports say “ZFS chop my RAM”.

The default limit dey high on purpose. OpenZFS 2.3 set maximum ARC size to the bigger value between RAM minus 1 GiB and 5/8 of RAM. OpenZFS 2.2 and older versions use half of RAM for Linux, while FreeBSD already use the newer rule. Run zfs version to see which rule apply to your system.

[
  {
    "label": "2 GB VPS",
    "openzfs_2_2_linux_gib": 1,
    "openzfs_2_3_gib": 1.25
  },
  {
    "label": "4 GB VPS",
    "openzfs_2_2_linux_gib": 2,
    "openzfs_2_3_gib": 3
  },
  {
    "label": "8 GB VPS",
    "openzfs_2_2_linux_gib": 4,
    "openzfs_2_3_gib": 7
  },
  {
    "label": "16 GB VPS",
    "openzfs_2_2_linux_gib": 8,
    "openzfs_2_3_gib": 15
  }
]

Those figures na the documented default rule wey dem apply to common instance sizes. Dem no be measurements from running server. For 4 GB instance, the 2.3 rule allow ARC of 3 GiB. The same box for 2.2 stop at 2 GiB. 2 GB instance under 2.3 rule still allow 1.25 GiB. Your application go get wetin remain.

Read the real numbers from your own server instead of relying on the table:

grep -E '^(size|c_max) ' /proc/spl/kstat/zfs/arcstats
arc_summary | head -n 20

The third column na bytes. c_max na the ceiling wey dey active now, and size na the amount wey ARC dey hold currently.

ARC dey return memory. Kernel dey signal memory pressure, then ARC dey reduce. The problem na timing, because that pressure dey drive the reduction. So, process wey request several hundred MiB at once fit meet the OOM (out of memory) killer while ARC still dey release memory. For 2 GB box wey dey run database and web server, this no be rare event. OpenZFS manual talk the same thing about manual changes: lowering the limit “will not cause the ARC to shrink without memory pressure to induce shrinking”.

How to limit ARC for small VPS

First decide how much memory the workload need. Add the memory wey the database and application need together. Keep some margin for the operating system. Give the remaining memory to ARC. For 4 GB instance wey dey run Postgres and one web application, 512 MiB to 1 GiB ARC na sensible starting point.

Set am live, in bytes. This one na 1 GiB.

echo 1073741824 | sudo tee /sys/module/zfs/parameters/zfs_arc_max

Make the setting survive reboot.

echo 'options zfs zfs_arc_max=1073741824' | sudo tee /etc/modprobe.d/zfs.conf
sudo update-initramfs -u

The initramfs step matter because the module fit load from initramfs before the root filesystem mount. If that happen, e no go read the file wey you just write. After reboot, confirm the value with the c_max line from arcstats.

Two cautions dey inside the manual itself. You no fit set the value back to 0 while the system dey run. To undo am, edit the file and reboot. Also, reducing the number no go shrink a large ARC immediately.

For FreeBSD, the same limit dey as a sysctl under vfs.zfs.arc. Run sysctl vfs.zfs.arc to see the current values and the exact name wey your version use. Then write the maximum value into /boot/loader.conf.

Two more memory rules dey for small server. Leave deduplication off because the dedup table dey inside memory. The commonly published rule of thumb na 1 to 3 GB of RAM per TB of unique data. Also, no put swap on a zvol (a block device carved out of the pool). Swapping through the filesystem wey dey try free memory fit deadlock the machine. Keep swap on a plain partition or swap file outside the pool.

When ext4 or XFS plus restic na the better answer

ZFS dey worth am for server wey get spare memory and second volume. Outside that one, plain filesystem plus real backup tool dey better. Choose ext4 or XFS when:

• The instance get 2 GB or 4 GB RAM and the workload need all of am.
• Na one virtual disk dey, and no second copy dey. So ZFS fit detect problem but e no fit repair am.
• Your backup target na object storage or plain Linux host, so nothing there fit receive zfs send stream.
• You dey run Debian with DKMS and you no fit afford kernel upgrade wey go leave the module unbuilt.
• You need ZFS for root filesystem, and the provider images only offer ext4.

Keep ZFS when you get separate data volume, enough spare RAM (8 GB and up dey comfortable), and a plan wey use snapshots and zfs send instead of just enabling dem. For everything else, ext4 with restic dey write encrypted, deduplicated backups to storage wey the server no control, and e cover most of the same work without using all that memory.

Failure modes, and the strings wey you go see

The pool don disappear after reboot. zpool status dey print no pools available. The import service dey read /etc/zfs/zpool.cache, so pool wey no dey inside that file no go ever import during boot. sudo zpool import dey list wetin fit import, sudo zpool import tank go bring am back, and sudo zpool set cachefile=/etc/zfs/zpool.cache tank go make am remain. Pool wey another system no export cleanly go report cannot import 'tank': pool may be in use from other system, and sudo zpool import -f tank go override am once you don confirm say no other host get am.

modprobe: FATAL: Module zfs not found in directory /lib/modules/6.12.0-... for Debian after kernel upgrade. DKMS no build for the new kernel, usually because matching headers no dey installed. dkms status go show wetin don build for each kernel. sudo apt install -y linux-headers-$(uname -r) go then sudo dkms autoinstall rebuild am, and sudo zpool import tank go bring the pool back.

The pool full but you don delete the files. Deleted data still dey for disk while snapshot still dey reference am, so du and df no go agree. zfs list -o space -r tank go divide usage into USEDDS and USEDSNAP, and large USEDSNAP na your answer. Destroy the old snapshots with sudo zfs destroy tank/data@2026-06-01 and the space go return.

CKSUM dey increase for zpool status. Something under ZFS return bad data. For mirror, the count na warning and the block don repair. For single-disk pool, the file don lost, zpool status -v go name am, and you go restore that one file from backup wey no dey inside this pool.

The server slow and e dey swap. Limit the ARC as described above, then run arc_summary and check the hit ratio. If ARC too small to hold the working set, every read go disk. Na that point plain filesystem wey dey use page cache for serve you better.

FAQ

How much RAM ZFS need for VPS?

ZFS fit run for 2 GB instance. The real question na how much remain for your application. If you no tune am, OpenZFS 2.3 go let ARC grow reach the bigger value between RAM minus 1 GiB and 5/8 of RAM, so 4 GB box fit give cache 3 GiB. Set zfs_arc_max to the number wey your workload fit spare, then confirm am by reading the c_max line from /proc/spl/kstat/zfs/arcstats.

ZFS snapshot na backup?

No. Snapshot dey inside the same pool with the data. E go survive bad rm and failed upgrade, but e go disappear if the pool or instance fail. Turn am to backup by sending am go another machine with zfs send, or run backup tool wey dey write to storage wey this server no control.

ZFS dey work the same way for FreeBSD and Linux?

Dem dey use the same codebase since OpenZFS 2.0 for December 2020. Dem get the same commands and same on-disk format, and pools fit move between dem. The difference na packaging. FreeBSD dey ship ZFS inside the base system. For Linux, each distribution decide how to package am: Ubuntu builds the module inside its kernel packages, while Debian builds am for your machine with DKMS. So kernel upgrade fit leave you without module until the rebuild succeed.

ZFS fit repair corruption on VPS wey get one disk?

E fit detect the corruption and name the file, but e no fit repair am because repair need second copy of the block. zfs set copies=2 for a dataset gives you that second copy with double the space. E fit handle bad block, but e no fit handle volume wey get lost. Mirror across two volumes na the answer wey fit actually heal the data.

Compression dey slow server down?

lz4 usually dey make am faster. Compressed blocks mean say fewer bytes dey write and fewer bytes dey read. The CPU cost for each block small compared with the disk work wey e save. Set compression=lz4 for the pool root so every dataset inherit am, then check zfs get compressratio tank after real data don write.