ZFS for FreeBSD and Linux: How Much RAM E Go Chop?

Wetin ZFS dey give you, and wetin e need

ZFS for FreeBSD and Linux don dey use one codebase now, OpenZFS, so the features dey the same for both systems. Server wey dey run ZFS get checksummed data, snapshots wey no cost anything until data change, replication with zfs send, and compression wey you fit enable with one property. Wetin e need na memory: ARC (adaptive replacement cache) dey take large part of RAM by default, and for VPS (virtual private server) wey get 2 GB or 4 GB, na that same memory your application need.

This guide dey assess ZFS from rented VPS wey get one or two virtual disks, no be from storage box wey get forty drive bays. The features wey still work after this change na the ones wey worth your time. You need know the parts wey no still work before you build pool.

OpenZFS for FreeBSD and Linux: one codebase, two packaging stories

FreeBSD don carry ZFS for base system since FreeBSD 7.0 for 2008, first as experimental feature. Since OpenZFS 2.0 for December 2020, FreeBSD and Linux dey build from the same source tree, so zfs and zpool dey behave the same way for both, and pool wey you create for one fit import for the other.

The reason ZFS na package for Linux but part of base system for FreeBSD na licensing. OpenZFS dey under CDDL (common development and distribution license). Linux kernel dey under GPL (general public license) version 2. Kernel project treat both licenses as incompatible, so dem no merge ZFS code into mainline Linux, and each distribution decide how e go ship am. FreeBSD no get this conflict, so ZFS just dey there. Na the whole practical story be this: na one packaging difference, and you no need choose side.

SSD Nodes no dey offer FreeBSD images, so for server wey you rent here, na the Linux part of this guide apply. If you dey run FreeBSD for another place, FreeBSD server go get ZFS without module wey you need build and without kernel upgrade wey you need survive.

Install ZFS and create a pool

For Ubuntu, the module dey inside the kernel packages, so na only the commands you need install.

sudo apt update
sudo apt install -y zfsutils-linux
zfs version

zfs version dey print two lines: the userland version and the kernel module version. If e print only one line, the module no load. The package dey inside universe component, wey Ubuntu server images enable by default. If apt no fit find am, run sudo add-apt-repository universe first.

For Debian, the packages dey inside contrib component, and DKMS (dynamic kernel module support) go build the module for your machine. Add contrib to the Components: line for /etc/apt/sources.list.d/debian.sources, run sudo apt update, then:

sudo apt install -y linux-headers-$(dpkg --print-architecture) zfs-dkms zfsutils-linux

The installation go compile the module and print Building initial module for 6.12.0-.... This fit take some minutes. Remember wetin this mean: every kernel upgrade go rebuild the module, and if the build fail, your pool no go import until you fix am.

For FreeBSD, nothing dey installed. Enable the service and start am.

sysrc zfs_enable=YES
service zfs start

Now make the pool. First check the stable device paths, because /dev/vdb dey assign names according to detection order, and the names fit change when you attach another volume.

ls -l /dev/disk/by-id/
sudo zpool create -o ashift=12 tank /dev/disk/by-id/virtio-abc123def456
zpool status tank

zpool status suppose print state: ONLINE, with your device listed under tank. ashift=12 set the pool smallest block size to 4 KiB. This match current SSDs, and you no fit change am after creation.

Most rented images boot from an ext4 root filesystem, so ZFS for here na data pool on a second volume, not the root filesystem. Confirm say the device be the one you expect before you build on am, because confirming the NVMe disk wey dem sell you dey take one minute, while rebuilding dey take one afternoon.

Checksums go fit repair only when pool get redundancy

Every block wey ZFS write get checksum, and every read dey verify am. Detection dey always work. Repair need another copy.

For single-disk pool, ZFS go tell you the truth and stop there. zpool status -v go report am like this:

status: One or more devices has experienced an error resulting in data
        corruption.
action: Restore the file in question if possible.  Otherwise restore the
        entire pool from backup.
errors: Permanent errors have been detected in the following files:

        /tank/data/archive.tar

E name the bad file. ext4 for don return those bytes without any comment, so this one don already get value. ZFS still no fit repair am, because pool no get another copy wey e fit use repair am.

With mirror, the same read go come from the good side, ZFS go rewrite the bad block, and the event go show for CKSUM column of zpool status. Na self-healing be this, and e need two devices.

sudo zpool create -o ashift=12 tank mirror /dev/disk/by-id/DISK1 /dev/disk/by-id/DISK2

For VPS, host storage usually get redundancy already, often RAID 10 under the hypervisor. This one protect you if drive die. But e no tell you when block return wrong, because array no get way to know which copy correct. ZFS know am because e compare the data with checksum wey e write by itself.

If you get one virtual disk and want small repair ability, sudo zfs set copies=2 tank/important go store two copies of every block for that dataset on the same disk. E go double the space wey that dataset use, e fit survive bad block, but e no go help when the whole volume disappear.

Scrub go read everything for pool and verify am.

sudo zpool scrub tank
zpool status tank

Healthy pool go end with line like scan: scrub repaired 0B in 00:04:11 with 0 errors. Put am for schedule; monthly dey okay for small pool.

systemctl list-unit-files 'zfs-scrub*'
sudo systemctl enable --now zfs-scrub-monthly@tank.timer

Dataset na unit wey policy dey use

Dataset na filesystem inside pool, and to create one no cost much, so make one for each job. Properties dey inherit from pool go down, so set default once and override am where e matter.

sudo zfs create tank/data
sudo zfs create tank/pg
sudo zfs set compression=lz4 tank
sudo zfs set atime=off tank
sudo zfs set quota=20G tank/data
sudo zfs set recordsize=16K tank/pg
zfs get -r compression,compressratio,quota tank

Compression na property wey people dey leave off because dem dey fear say e fit cause wahala, but na opposite dey happen. lz4 dey use small CPU and reduce bytes wey need reach disk, so for data wey fit compress, e usually make reads and writes faster. zstd dey compress harder with more CPU, and e fit logs and archives wey you rarely read again. Use zfs get compressratio tank check wetin you dey really get, and remember say ratio only count data wey you write after you set the property.

recordsize na the biggest block wey dataset dey write, and 128K na the default. Database wey dey write 8 KiB pages inside 128 KiB records go turn one small write into read of the whole record, change, then write am back. Set recordsize=16K for the database dataset before you load the data, because the property only apply to newly written blocks.

quota na how you fit stop one dataset from filling the pool. ZFS pool wey near 100% full dey slow and e hard to clean up, so leave free space on purpose.

Snapshots no dey cost anything until data change

ZFS no dey overwrite live block. E dey write new block and update the pointers, and na this one dem dey call copy-on-write. Snapshot na note wey talk say “keep the blocks wey this dataset point to now”, so to take one dey happen instantly and e free.

sudo zfs snapshot tank/data@2026-08-11
zfs list -t snapshot -o name,used,refer -r tank/data

The USED column for snapshot na the space wey na only that snapshot dey hold. E dey start near zero and e dey grow as you change or delete data, because old blocks no fit release again.

To bring file back no need any restore step.

ls /tank/data/.zfs/snapshot/
cp /tank/data/.zfs/snapshot/2026-08-11/notes.txt /tank/data/notes.txt

The .zfs directory dey hidden even from ls -a until you run sudo zfs set snapdir=visible tank/data. Take the snapshot before you need am, because if you no get one, stray rm -rf go put you for the ext4 recovery path, wey start by unmounting the disk and the matter go worse from there.

Rollback dey throw away everything wey you write since the snapshot.

sudo zfs rollback tank/data@2026-08-11

E no go work when newer snapshots dey exist, and -r dey destroy those newer snapshots so e fit continue. Read the dataset name two times before you press enter.

Snapshot no be backup. E dey the same pool, for the same volume, and on the same server. Failed volume or one zpool destroy go carry the snapshots along with the data. Snapshots protect you from your own rm and from bad upgrade, and this one cover plenty real incidents. But dem no protect you from anything wey happen to the pool itself. You fit see the full explanation here: why VPS snapshot no be backup.

Send and receive: replication with one command

zfs send dey turn snapshot into byte stream for standard output, while zfs receive dey turn that stream back into dataset. The first copy na full send.

sudo zfs snapshot tank/data@daily-2026-08-11
sudo zfs send tank/data@daily-2026-08-11 | ssh backup.example.com "sudo zfs recv -F backup/data"

After that, send only wetin change between two snapshots.

sudo zfs snapshot tank/data@daily-2026-08-12
sudo zfs send -i tank/data@daily-2026-08-11 tank/data@daily-2026-08-12 | ssh backup.example.com "sudo zfs recv backup/data"

The receiving side still must get the snapshot wey you dey send from. If e no get am, receive go stop with cannot receive incremental stream: most recent snapshot of backup/data does not match incremental source, because ZFS no get base to apply the difference to. Send from snapshot wey both sides get, or start again with full send.

Grant rights for the target instead of using remote root: sudo zfs allow -u backupuser create,mount,receive backup/data.

This na real off-site backup, but e get one condition. The far end must be ZFS pool, because object storage no fit receive stream. If your target na S3-compatible storage or plain Linux host, use tool wey dey communicate with am, and restic backups from a VPS cover that path.

Why ZFS dey use so much RAM? The ARC

The ARC (adaptive replacement cache) na ZFS read cache. E dey inside kernel memory instead of normal Linux page cache, so free -h no dey report am under buff/cache. E go show as memory wey dey use. ZFS box wey look nearly full usually na box wey cache don warm, and na this dey explain most "ZFS ate my RAM" reports.

The default limit dey generous on purpose. OpenZFS 2.3 set the maximum ARC size to the bigger value between RAM minus 1 GiB and 5/8 of RAM. OpenZFS 2.2 and earlier use half of RAM for Linux, while FreeBSD don already dey use the newer rule. Run zfs version to see which one apply to you.

[
  {
    "label": "2 GB VPS",
    "openzfs_2_2_linux_gib": 1,
    "openzfs_2_3_gib": 1.25
  },
  {
    "label": "4 GB VPS",
    "openzfs_2_2_linux_gib": 2,
    "openzfs_2_3_gib": 3
  },
  {
    "label": "8 GB VPS",
    "openzfs_2_2_linux_gib": 4,
    "openzfs_2_3_gib": 7
  },
  {
    "label": "16 GB VPS",
    "openzfs_2_2_linux_gib": 8,
    "openzfs_2_3_gib": 15
  }
]

Those figures na documented default rule wey dem apply to common instance sizes, no be measurements from running box. For 4 GB instance, the 2.3 rule allow ARC of 3 GiB. The same box for 2.2 stop at 2 GiB. 2 GB instance under the 2.3 rule still allow 1.25 GiB. Your application go get wetin remain.

Read the real numbers from your own server instead of trusting the table:

grep -E '^(size|c_max) ' /proc/spl/kstat/zfs/arcstats
arc_summary | head -n 20

The third column na bytes. c_max na the ceiling wey dey apply now, and size na wetin the ARC hold currently.

The ARC dey return memory. Kernel go signal pressure and ARC go shrink. The problem na timing, because na that pressure dey drive the shrinking, so process wey ask for several hundred MiB at once fit meet OOM (out of memory) killer while ARC still dey release memory. For 2 GB box wey dey run database and web server, this no be rare event. OpenZFS manual talk the same thing about manual changes: lowering the limit "will not cause the ARC to shrink without memory pressure to induce shrinking".

How to limit ARC for small VPS

First decide how much memory the workload need. Add the memory wey the database and application need together. Keep some margin for the operating system. Give the remaining memory to ARC. For a 4 GB instance wey dey run Postgres and one web application, 512 MiB to 1 GiB ARC na sensible starting point.

Set am live, in bytes. This one na 1 GiB.

echo 1073741824 | sudo tee /sys/module/zfs/parameters/zfs_arc_max

Make am remain after reboot.

echo 'options zfs zfs_arc_max=1073741824' | sudo tee /etc/modprobe.d/zfs.conf
sudo update-initramfs -u

The initramfs step important because the module fit load from initramfs before the root filesystem mount. If that happen, e no go read the file wey you just write. After reboot, confirm am with the c_max line from arcstats.

Two caveats dey inside the manual itself. You no fit set the value back to 0 while the system dey run. To undo am, edit the file and reboot. Also, lowering the number no go reduce a large ARC immediately.

For FreeBSD, the same limit dey as a sysctl under vfs.zfs.arc. Run sysctl vfs.zfs.arc to see the current values and the exact name wey your version dey use. Then write the maximum value into /boot/loader.conf.

Two more memory rules dey for small server. Leave deduplication off, because the dedup table dey inside memory. The commonly published rule of thumb na 1 to 3 GB RAM for every TB of unique data. Also, no put swap on a zvol, wey be a block device carved out of the pool. Swapping through the filesystem wey dey try free memory fit deadlock the machine. Keep swap on a plain partition or a swap file outside the pool.

When ext4 or XFS plus restic dey be the better answer

ZFS dey make sense for server wey get spare memory and second volume. Outside that one, plain filesystem plus real backup tool dey win. Choose ext4 or XFS when:

• The instance get 2 GB or 4 GB RAM and the workload need all of am.
• One virtual disk dey, and no second copy dey, so ZFS fit only detect problem without repairing am.
• Your backup target na object storage or plain Linux host, so nothing for there fit receive a zfs send stream.
• You dey run Debian with DKMS and you no fit afford kernel upgrade wey go leave the module unbuilt.
• You need ZFS for root filesystem, and the provider images only offer ext4.

Keep ZFS when you get separate data volume, enough spare RAM (8 GB and up dey comfortable), and plan wey dey use snapshots and zfs send instead of just enabling dem. For everything else, ext4 with restic writing encrypted, deduplicated backups to storage wey the server no control covers almost the same work without using the memory.

Failure modes, plus the strings wey you go see

The pool don disappear after reboot. zpool status go print no pools available. The import service dey read /etc/zfs/zpool.cache, so e no go ever import pool wey no dey inside that file when system boot. sudo zpool import go list wetin fit import, sudo zpool import tank go bring am back, and sudo zpool set cachefile=/etc/zfs/zpool.cache tank go make am stay after reboot. Pool wey dem no export cleanly from another system go report cannot import 'tank': pool may be in use from other system, and sudo zpool import -f tank go override am after you don confirm say no other host get am.

modprobe: FATAL: Module zfs not found in directory /lib/modules/6.12.0-... for Debian after kernel upgrade. DKMS no build for the new kernel, usually because matching headers no dey installed. dkms status go show wetin don build for each kernel. sudo apt install -y linux-headers-$(uname -r) then sudo dkms autoinstall go rebuild am, and sudo zpool import tank go bring the pool back.

The pool full but you don delete the files. Deleted data still dey for disk while snapshot still dey reference am, so du and df no go agree. zfs list -o space -r tank go split usage into USEDDS and USEDSNAP, and large USEDSNAP na the answer. Destroy the old snapshots with sudo zfs destroy tank/data@2026-06-01 and the space go come back.

CKSUM counts dey increase for zpool status. Something underneath ZFS return bad data. For mirror, the count na warning and the block don repair. For single-disk pool, the file don loss, zpool status -v go name am, and you go restore that one file from backup wey no dey inside this pool.

The server slow and dey swap. Limit the ARC as described above, then run arc_summary and check the hit ratio. If ARC too small to hold the working set, every read go disk. Na for this point plain filesystem wey dey use page cache for serve you better.

FAQ

How much RAM ZFS dey need for VPS?

ZFS fit run for instance wey get 2 GB. The real question na how much remain for your application. If you no tune am, OpenZFS 2.3 go let ARC grow reach the bigger value between RAM minus 1 GiB and 5/8 of RAM, so 4 GB box fit give cache 3 GiB. Set zfs_arc_max to number wey your workload fit spare, then confirm am by reading the c_max line from /proc/spl/kstat/zfs/arcstats.

Snapshot for ZFS na backup?

No. Snapshot dey inside the same pool with the data. E go survive bad rm and failed upgrade, but e go disappear if the pool or instance fail. Turn am to backup by sending am go another machine with zfs send, or run backup tool wey dey write to storage wey this server no control.

ZFS dey work the same way for FreeBSD and Linux?

Same codebase since OpenZFS 2.0 for December 2020, same commands, same on-disk format, and pools fit move between dem. The difference na packaging. FreeBSD ships ZFS inside the base system. For Linux, each distribution dey decide: Ubuntu builds the module inside its kernel packages, while Debian builds am for your machine with DKMS. So kernel upgrade fit leave you without module until the rebuild succeed.

ZFS fit repair corruption for VPS wey get one disk?

E dey detect the corruption and name the file, but e no fit repair am because repair need second copy of the block. zfs set copies=2 for dataset gives you that second copy with double the space. This handles bad block, but e no handle lost volume. Mirror across two volumes na the answer wey actually heals the data.

Compression dey slow server down?

lz4 usually dey make am faster. Compressed blocks mean say fewer bytes dey write and fewer bytes dey read, and the CPU cost for each block small compared with the disk work wey e save. Set compression=lz4 for pool root so every dataset inherit am, then check zfs get compressratio tank after real data don write.