OpenZFS RAID-Z Expansion: A New Era in Storage Flexibility

ZFS has proven itself to be a reliable and trustworthy filesystem over many years of use. Its RAID-Z level, an advanced form of RAID-5, had a major flaw in its initial implementation: users couldn’t add disks to a RAID-Z array. There was no easy way to expand it after initial creation, which meant creating a new array and copying data. Now FreeBSD has implemented the solution with the addition of RAID-Z expansion.

Many users with small pools previously had no reasonable path to expansion, forcing them to replace all drives at once or migrate data manually. With RAID-Z expansion, they can grow their storage capacity incrementally, reducing costs and complexity.

Storage needs often grow over time in environments using FreeBSD-based NAS systems. Before, expanding a RAID-Z vdev required backing up data, recreating the pool with additional drives, and restoring everything, which was time-consuming and carried inherent risk. With RAID-Z expansion, administrators can add new drives seamlessly, increasing storage capacity without disrupting operations or risking data loss.

RAID-Z vdev are commonly used for high-availability systems in enterprise environments. Despite how critical it is to manage growing datasets while maintaining uptime, expanding storage requires complex migrations or system downtime. With RAID-Z expansion, system administrators can scale storage dynamically without interrupting operations, ensuring continuous availability and data integrity.

The Road to RAID-Z Expansion

Expanding storage capacity has long been a challenge for RAID-Z users. Traditionally, increasing the size of a RAID-Z pool required adding an entirely new RAID-Z vdev, often doubling the number of disks—an impractical solution for smaller storage pools with limited expansion options.

To address this, the FreeBSD Foundation funded the development of RAID-Z expansion, making it both practical and easy to implement. Led by Matt Ahrens, a ZFS co-creator, the feature underwent years of rigorous testing and refinement. Although the pandemic caused some delays, the project was feature complete in 2022. Additional integration steps followed, and the feature is now generally available in the OpenZFS.

Thank You for Your Support

After years of development, industry collaboration, infrastructure testing, and nearly $100,000 investment, we are so excited to see RAID-Z expansion in the recent release of OpenZFS 2.3. We’re also grateful to iXsystems for their efforts in finalizing and integrating this feature into OpenZFS.

This marks a significant milestone in the evolution of the ZFS filesystem and reinforces its position as a cutting-edge open source filesystem for modern storage use cases. 

This development work happened because of your donations to the FreeBSD Foundation. We couldn’t have made this financial commitment without your help. Thank you to all our supporters, large and small.

A Technical Look at How to Expand RAID-Z

RAID-Z expansion enables users to increase storage capacity by adding disks one at a time to an existing RAID-Z group, eliminating the need to double the number of drives.

The process works by redistributing existing data across the new disk configuration, creating a contiguous block of free space at the end of the logical RAID-Z group. Because the expansion occurs online, the pool remains accessible throughout. If interrupted—due to a system reboot, for instance—the process resumes from the last known state, ensuring fault tolerance. Importantly, RAID-Z expansion maintains data redundancy, preserving the reliability of stored data.

This feature supports all RAID-Z levels—RAID-Z1, RAID-Z2, and RAID-Z3—without altering their failure tolerance. A RAID-Z1 pool remains a RAID Z1 pool even after multiple expansions.

The benefits of RAID-Z expansion include:

This demonstration of RAID-Z expansion uses a recent FreeBSD 15-CURRENT snapshot: bsd_FreeBSD-15.0-CURRENT-amd64-zfs-20250116-054c5ddf587a-274800.raw from 2025/01/16, which includes all the latest OpenZFS 2.3 goodies. For reference, here is the uname(1) output of the system used in this demonstration.

The demonstration will use plain files on disk – represented in the /dev tree by md(4) FreeBSD driver.

First we will create and prepare five files – the first four will be used for the initial ZFS RAID-Z pool and the 5th will be used as an expansion.

We will now create our RAID-Z ZFS pool from 4 disks.

Our ZFS pool is healthy and empty. We will now populate it with some /usr/lib data (4 times) to make sure we work on the populated ZFS pool.

Now the most important part – we will expand the ZFS pool with the additional disk.

Now we will watch the ZFS pool expansion ‘live’ with zpool(8) and iostat(8) tools.

The zpool(8) command with iostat argument.

Now the iostat(8) command.

After the expansion is complete you will be able to see similar output as the one below.