RAID 10 for VPS Storage: Wetin E Mean

RAID 10 wetin be, and why VPS hosts dey run am

RAID 10 na storage layout wey most VPS hosts dey use under virtualised NVMe (non-volatile memory express) drives. E mirror every drive go one partner, then e stripe data across those mirrored pairs. One drive fit die without the array stopping, and repair na simple copy from the surviving partner instead of recalculation wey go read every other drive for the set.

RAID mean redundant array of independent disks. E get one job: make machine continue serving while disk dey dead or dem dey replace am. That job na availability, and availability no be safety.

RAID dey replicate your writes. rm -rf /srv na write. Both halves of the mirror fit drop the directory for the same millisecond, and the array still report say e clean afterwards.

Keep that sentence. The rest of this page explain wetin each level fit survive and wetin each one cost you for every write. The last sections na the commands to read array status for machine wey you own, and the failure wey RAID never cover.

The levels wey hosting buyer dey really meet: 1, 5, 6 and 10

Plan page go name one number and stop there. The number answer two questions: how many drives fit fail, and wetin every write dey cost.

RAID 1 na mirror. Two drives dey hold identical blocks. Every write dey go both drives. Either drive fit serve read. One drive fit fail without data loss, and half of the raw capacity dey usable. No parity dey to calculate, so the write path short.

RAID 5 na striping with one parity block for every stripe. With n drives, you get capacity equal to n-1 drives, and the array fit survive exactly one failure. Parity no dey one dedicated drive. E dey rotate across all of dem, so every drive dey carry both data and parity.

RAID 6 add second, independent parity block to every stripe, usually dem dey write am as P and Q. E fit survive any two drives failing at the same time. This matter pass as e sound, because the second failure most times dey happen while dem dey repair the first one.

RAID 10 na stripe of mirrors. Dem dey mirror drives in pairs, and data dey spread across the pairs. Usable capacity na half of the raw total, same as RAID 1, with the parallelism of striping on top.

You go also see am written as RAID 1+0, and na the honest description be that: mirror first, then stripe across the mirrors. RAID 0+1 na the other order, stripe first and then mirror the two stripes. E worse, because one drive failure go take one whole stripe out of service, and the repair go need copy the entire other side.

Linux na special case wey you suppose know. The kernel's raid10 na one personality instead of two stacked layers, so e fit run on odd number of drives and e get layouts (near, far, offset) wey nested setup no fit express. Na why the status line for Linux box dey show 2 near-copies instead of naming two arrays.

[
  {
    "label": "RAID 1 (four mirrored pairs)",
    "usable_tb": 4,
    "worst_case_drives_lost": 1,
    "best_case_drives_lost": 4
  },
  {
    "label": "RAID 5",
    "usable_tb": 7,
    "worst_case_drives_lost": 1,
    "best_case_drives_lost": 1
  },
  {
    "label": "RAID 6",
    "usable_tb": 6,
    "worst_case_drives_lost": 2,
    "best_case_drives_lost": 2
  },
  {
    "label": "RAID 10",
    "usable_tb": 4,
    "worst_case_drives_lost": 1,
    "best_case_drives_lost": 4
  }
]

Eight 1 TB drives give 7 TB of usable space under RAID 5 and 4 TB under RAID 10. That difference na real money, and na why people dey continue to propose parity. RAID 6 fit survive 2 failures in any pattern at all. RAID 10 guarantee only 1, because the dangerous second failure na the one wey land on the partner of the drive wey don already fail. E fit survive up to 4 when no two failures share one pair, but na luck be that, no be design property.

Wetin each level dey cost for every write

Write to a mirror na two writes, wey dem issue at the same time to both members. Write to a parity stripe na more work, because the parity block for that stripe don become wrong and dem need recompute am.

The controller no fit recompute parity from the new block alone. E first need the old data block and the old parity block. So one small random write to RAID 5 become read, read, write, write. RAID 6 get second syndrome to maintain, so the same write become read, read, read, write, write, write.

[
  {
    "label": "RAID 1 (2 drives)",
    "write_ops_per_host_write": 2,
    "drives_read_to_rebuild": 1
  },
  {
    "label": "RAID 5 (8 drives)",
    "write_ops_per_host_write": 4,
    "drives_read_to_rebuild": 7
  },
  {
    "label": "RAID 6 (8 drives)",
    "write_ops_per_host_write": 6,
    "drives_read_to_rebuild": 7
  },
  {
    "label": "RAID 10 (8 drives)",
    "write_ops_per_host_write": 2,
    "drives_read_to_rebuild": 1
  }
]

Small random write dey cost 6 device operations for RAID 6 and 2 for RAID 10. These counts no show the full latency difference. The two mirror writes go out in parallel, so the guest dey wait for the slower one. The parity path get a read wey must finish before dem fit calculate the new parity, so the guest dey wait for read and then write, one after the other. For busy host, that read go queue behind other people I/O.

One important exception dey. Write wey big enough to fill complete stripe no need old data, because every block for the stripe dey replace. Dem calculate parity from wetin dey memory already, and the cost reduce to one extra write. Na why RAID 5 dey look fine for sequential benchmark but dey behave badly under mixed load of small writes from plenty tenants. Test the pattern wey you actually dey run: how to benchmark VPS disk properly mean random I/O for realistic queue depth, no be one large dd.

Why rebuild dey be the dangerous part

Parity rebuild suppose reconstruct the missing drive from every other drive, so e go read 7 surviving drives from first block reach last one. RAID 10 rebuild read 1: the dead drive mirror partner, and nothing else.

Two costs dey follow from this. The first na time, because the slowest surviving drive and the parity calculation on top of am limit how fast rebuild fit run. The second na load. Every drive for the parity set dey busy throughout the whole period, so every guest for that node go see higher latency until e finish. For RAID 10, one pair dey busy while the other pairs continue to serve at normal speed.

Correctness risk dey during that same period. RAID 5 array wey get one dead drive no get redundancy again, so any unreadable sector for any surviving drive no fit recover. Rebuild na the one operation wey read every sector, including sectors wey nobody don touch for one year. Published datasheet figures put consumer hard drive near one unrecoverable read error per 10^14 bits read, and enterprise NVMe drive at one per 10^17 or better. Those figures na vendor specifications, no be measurements, but the ratio explain why the old warning say RAID 5 rebuild go fail concern large spinning disks, and why the warning no strong as much for NVMe. The load argument still apply to any storage medium.

Use scrubbing to find latent errors before rebuild expose dem. Debian and Ubuntu ship periodic scrub for md arrays, but the mechanism dey differ between releases, so check the one wey your system dey use, then trigger one pass by hand.

systemctl list-timers --all | grep -i mdcheck
ls -l /etc/cron.d/mdadm
echo check | sudo tee /sys/block/md0/md/sync_action
cat /sys/block/md0/md/mismatch_cnt

sync_action go back to idle when the pass finish, and mismatch_cnt suppose read 0. Number wey pass zero for a mirror mean say both halves no agree, and the kernel no fit tell which one correct because neither copy carry checksum. Some mismatches no dey harmful, and swap partitions na the usual source: kernel fit write one page wey change underneath am. If the count dey rise for data array, replace the drive.

Why VPS providers dey standardise on RAID 10 for NVMe

Hypervisor node no dey run just one workload. E dey run dozens of unrelated guests, and their I/O dey arrive interleaved as stream of small writes with no locality between dem. Na this exact pattern parity read-modify-write cycle dey cost pass, and na the pattern shared node dey handle all day.

Add the rebuild behaviour, and the choice go clear. Failed drive for parity node dey slow every guest for the box for hours. Failed drive for RAID 10 node dey slow one pair, and the copy dey run sequentially at drive speed. Providers dey sell latency wey no spike, so dem dey pay for am with capacity: half of the raw NVMe dey go to the mirror.

Drive sizes dey push the same way. As drives dey get bigger, rebuild window dey get longer, and for parity, na that window be the period wey everything slow and nothing dey protected. Na the same reason ZFS deployments for virtualisation dey use pools of mirrored vdevs instead of wide raidz: mirror resilver dey copy only the blocks wey dey in use, for one pair.

None of this mean say RAID 10 correct everywhere. Backup target dey write in long sequential runs and dem rarely dey read am, so RAID 6 na the better deal there. E fit survive two failures and return most of the capacity. Na workload dey decide, no be the number. For plan wey you dey choose today, the medium usually matter pass the layout on top of am, and the jump from SATA SSD to NVMe dey bigger than any RAID difference for either one.

How to read /proc/mdstat

Run dem for machine wey you own the array: a dedicated server, box for house, or VPS with two attached volumes wey you assemble yourself. Read your own output. The blocks below na examples, written out so you fit match the shape of wetin you get.

cat /proc/mdstat
sudo mdadm --detail /dev/md0
lsblk -o NAME,SIZE,TYPE,MOUNTPOINTS

Healthy four-drive RAID 10 go print something close to this.

Personalities : [raid1] [raid10]
md0 : active raid10 nvme3n1p3[3] nvme2n1p3[2] nvme1n1p3[1] nvme0n1p3[0]
      3906764800 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]
      bitmap: 0/30 pages [0KB], 65536KB chunk

unused devices: <none>

Every part of this output carry information.

• Personalities lists the md modules wey the running kernel don load. If raid10 dey there, e mean say the code dey available, and nothing more.
• md0 : active raid10 na the array device, its state, and its level.
• The names after that na the members. The number inside square brackets na the device index for the array metadata. E no be the position for the line, and e no always be the slot.
• After you replace drive, the new member usually keeps index wey higher pass the slot wey e fill. So nvme4n1p3[4] fit dey for slot 2. mdadm --detail prints the real slot for its RaidDevice column, so use that one when the difference matter.
• (F) after member mean say the member faulty. (S) mean spare: e dey present, idle, and dey wait for something to fail.
• 3906764800 blocks super 1.2 na the usable size for 1 KiB blocks, followed by the metadata format.
• 512K chunks 2 near-copies na the stripe chunk size and the RAID 10 layout. For this example, e keeps two copies of each block next to each other.
• [4/4] na the number of members wey the array expect, followed by the number wey currently dey in sync.
• [UUUU] na one character for each slot, in slot order. U na slot wey dey up and in sync. _ na slot wey nothing dey work inside.
• bitmap: na the write intent bitmap. E records the regions wey dem dey write, so if member drop out and come back, e go resync those regions instead of the whole drive.

Wetin [4/3] and [UU_U] mean when something dey wrong

Degraded array fit look like this.

md0 : active raid10 nvme3n1p3[3] nvme2n1p3[2](F) nvme1n1p3[1] nvme0n1p3[0]
      3906764800 blocks super 1.2 512K chunks 2 near-copies [4/3] [UU_U]

Read the two brackets together. [4/3] say one out of four slots no dey contribute. [UU_U] show which one, because underscore na the third character and dem number slots from zero, so slot 2 dey down. The (F) flag name the device only while the failed drive still dey attached. Remove am from the machine and the name go disappear from the line, while the underscore go remain.

The array go continue serving through all this. For RAID 10, e often dey serve almost at full speed, so nobody go notice by feel. Something must tell you.

grep -i mailaddr /etc/mdadm/mdadm.conf
sudo mdadm --monitor --scan --oneshot --test
systemctl list-units --all | grep -i md

The mdadm package installs a monitor daemon wey reads MAILADDR from /etc/mdadm/mdadm.conf. The unit name don change between releases, so use the last command find am instead of guessing. The --test run sends one message for each array immediately. If inbox empty after that, the mail path dey broken. This means the message wey you really care about for don get lost the same way.

When replacement dey rebuild, progress line go show under the array.

md0 : active raid10 nvme4n1p3[4] nvme3n1p3[3] nvme1n1p3[1] nvme0n1p3[0]
      3906764800 blocks super 1.2 512K chunks 2 near-copies [4/3] [UU_U]
      [==>..................]  recovery = 12.4% (242012928/1953382400) finish=63.1min speed=452000K/sec

recovery na rebuild go replacement drive. resync na the first consistency pass over newly created array. check na the scrub wey you trigger above. The pair inside parentheses na progress for 1 KiB blocks against the per-device total, while finish na the kernel estimate for the current speed. /proc/sys/dev/raid/speed_limit_min and speed_limit_max cap that speed. The caps dey there so rebuild no go starve production I/O.

/dev/md0:
           Version : 1.2
     Creation Time : Tue Mar 10 09:14:22 2026
        Raid Level : raid10
        Array Size : 3906764800 (3.64 TiB 4.00 TB)
     Used Dev Size : 1953382400 (1.82 TiB 2.00 TB)
      Raid Devices : 4
     Total Devices : 4
       Persistence : Superblock is persistent

       Update Time : Wed Aug  5 11:02:41 2026
             State : clean, degraded, recovering
    Active Devices : 3
   Working Devices : 4
    Failed Devices : 0
     Spare Devices : 1

            Layout : near=2
        Chunk Size : 512K

    Rebuild Status : 12% complete

              Name : storage:0
            Events : 4184

    Number   Major   Minor   RaidDevice State
       0     259        3        0      active sync set-A   /dev/nvme0n1p3
       1     259        7        1      active sync set-B   /dev/nvme1n1p3
       4     259       11        2      spare rebuilding    /dev/nvme4n1p3
       3     259       15        3      active sync set-B   /dev/nvme3n1p3

Replacing drive for array wey you own na four commands, and the last one na the check.

sudo mdadm --manage /dev/md0 --fail /dev/nvme2n1p3
sudo mdadm --manage /dev/md0 --remove /dev/nvme2n1p3
sudo mdadm --manage /dev/md0 --add /dev/nvme4n1p3
cat /proc/mdstat

The recovery line suppose show within one or two seconds. The replacement partition must be at least as large as Used Dev Size from mdadm --detail. If partition even slightly smaller, system go reject am with message of the form not large enough to join array. Partition the new drive to match the old one before you add am.

Wetin you fit and no fit see from inside VPS

Most guests no fit see host RAID, and na so dem design am. Hypervisor dey give you one virtual disk. Whether dem carve that disk from RAID 10 pool of NVMe drives or e dey on one drive na property of the host, and e no go show inside your guest.

systemd-detect-virt
lsblk -d -o NAME,SIZE,ROTA,MODEL
cat /proc/mdstat

systemd-detect-virt dey print kvm for KVM guest, container type like lxc for container, and none for bare metal. For KVM guest, normally you go see one vda or sda inside lsblk, and no array for /proc/mdstat, because no array dey inside the guest.

For container-based VPS, you no fit trust the reading. Containers dey share host kernel, and parts of /proc no dey namespaced. So wetin you read there fit describe the host instead of your own slice. No treat any of am as fact about your own storage. Ask the provider how the layout be, and make dem give you the answer for writing if e matter to you.

Wetin you fit check from inside na how the disk wey dem give you dey behave. Check whether your VPS disk really be NVMe cover the commands wey report real information, while wetin an SSD VPS actually include cover wetin the label for the plan page dey claim.

You suppose run RAID inside your VPS?

Usually no, and the reason na failure domains. If you attach two volumes to one VPS and mirror dem with mdadm, both volumes fit dey on the same physical array, for the same node, behind the same power supply. You go double the cost of every write for redundancy wey you already get, and you still go lose both copies to the one failure wey matter.

E make sense when the provider document say the volumes dey for separate failure domains, or when you dey use dedicated server with drives wey you fit control. Otherwise, make the effort go into copies wey comot from the machine.

Wetin RAID no dey protect you from

RAID dey cover one event: drive wey stop to work correctly. Everything wey dey below na valid write, so the array go apply am to every copy and report say e healthy.

• Deletion. rm -rf for wrong directory, or deploy script wey get unset variable for path. The array see legal write and do am twice.
• Ransomware. Encryption na writing. Healthy array go store the encrypted version for both sides of the mirror.
• Broken application. Bug wey write garbage inside your database go write the same garbage to the redundant drive.
• The whole node. Host wey fail, or account wey dem suspend by mistake. Array fit dey perfect and still no dey reachable at the same time.
• You, one week later. File wey you delete on Monday don disappear from every drive that same Monday. Na only copy wey dem take before then fit bring am back.

Snapshots for the same storage no be the fix too. Dem fit help against deletion, but dem go die with the array wey dem dey live on. The thing wey make backup be backup na say e dey somewhere else. Encrypted off-server backups with restic na the other half of this page: the array keep you serving through dead drive, while restic bring your data back when the damage na write wey the array happy to make.

FAQ

RAID 10 mean say I no need backups?

No. RAID 10 dey protect against drive wey stop working. E dey apply every valid write to both sides of mirror, so deletion or ransomware run go reach the redundant drive for the same instant. The array go report say e clean afterwards, because from im own point of view nothing fail. You still need copies wey dey outside the machine, and you still need restore one sometimes to prove say dem dey work.

Why VPS providers dey choose RAID 10 instead of RAID 5 or RAID 6?

Two reasons, and both concern small random writes. Parity write need make e read the old data and old parity first before e fit calculate the new parity. So, small write cost 4 operations for RAID 5 and 6 for RAID 6, compared with 2 for mirror. Parity rebuild then dey read every surviving drive from beginning reach end. This one dey slow every guest for the node for hours, while RAID 10 rebuild dey copy one drive go another one and leave the other pairs alone. Providers dey pay for this with capacity: half of the raw NVMe.

Wetin [U_] or [UU_U] mean for /proc/mdstat?

Each character na one slot for the array, arranged according to slot order, with one character for each slot. U mean say that slot get member wey dey up and in sync. _ mean say nothing wey dey work dey inside the slot. [U_] for a two-drive mirror mean say the second slot dey down and no redundancy remain. Read am together with the pair wey dey in front, where [4/3] dey show say the array expect four members and get three. The slot order match the RaidDevice column of mdadm --detail, no be the order wey device names appear for the line.

How many drives fit RAID 10 array lose?

One, for any pattern. After that, e depend on where the failures happen. Each mirror pair fit lose one of the two members. So, an eight-drive array fit survive up to four failures if no two of them share one pair. But e go fail after two failures if both ones hit the same pair. Plan with the guaranteed number, wey be one, and treat anything beyond am as luck, no be protection.

I suppose mirror two volumes inside my VPS with mdadm?

Usually no. Two volumes wey attach to one VPS often dey on the same physical array for the same host. So mirroring dem go double the cost of every write and protect against nothing wey the host's own RAID never already cover. E make sense only when the provider document say the volumes dey for separate failure domains. Otherwise, use that effort for backups wey dey leave the machine.