EXT4 Max Volume Size - newbie question

[xanbo]

I understand EXT4 has a maximum recommended volume size of 16TB, after which it is not considered to be stable. I have 5 drives of 4TB capacity in a RAID 6 array, so that is a total of 20TB but only 16TB accessible for storage due to double parity. Am I correct in my interpretation that my array therefore has a volume size of 12TB (rather than 20TB) and is therefore EXT4 is a viable option? Thanks in advance for any feedback!

 

[stevech]

I understand EXT4 has a maximum recommended volume size of 16TB, after which it is not considered to be stable. I have 5 drives of 4TB capacity in a RAID 6 array, so that is a total of 20TB but only 16TB accessible for storage due to double parity. Am I correct in my interpretation that my array therefore has a volume size of 12TB (rather than 20TB) and is therefore EXT4 is a viable option? Thanks in advance for any feedback!

By design, EXT4 volumes can be exabytes. But the question should go to the NAS vendor re their OS.

http://en.wikipedia.org/wiki/Ext4

 

[xanbo]

OK, thank you very much.

 

[htismaqe]

The max FILE size in ext4 is 16TB. The maximum file SYSTEM size is 1 exabyte.

However, it's entirely up to your NAS vendor, like steve said. Many of them have OS or hardware limitations that restrict the maximum system volume size. I know several QNAP's have a 16TB maximum...

 

[stevech]

A 16TB file. I wonder where such is used?

 

[azazel1024]

Some of our old VSAM databases (which are flat files in the end) are up there. I don't think any is 16TB, but >1TB, yes.

Very different story though.

Maybe someone wanted to upsample every episode of ER to 4k and then splice it in to a single video file?

 

[mdgm]

A 16TB hardware limit for a NAS with some 32-bit CPUs is a limitation of the pagecache. With a 32-bit index into 4k pages, you can only address 16TB in the Pagecache.

You could of course try using a larger Linux page size with EXT4, but that can lead to other problems such as memory fragmentation, which would be a big issue especially on an ARM box as that probably doesn't have much memory in it to start with.

If you want to use volumes >16TB it would be best in my view to get a NAS with a 64-bit Intel CPU which also has 6TB drives on the compatibility list.

Expansion can also become an issue around the 16TB mark with EXT4. Some NAS units cannot expand an existing volume past 16TB even if a larger volume can be created after wiping the disks.

With our new NETGEAR ReadyNAS running OS6 we decided to use BTRFS. With BTRFS on a NAS with an Intel 64-bit CPU you don't have the expansion limitations that can be found on NAS units using EXT4.

With BTRFS there are features that can help with protecting your data: unlimited snapshots, checksums, bitrot protection (bitrot protection is coming in 6.2.0 - currently in public beta) and more.

BTRFS can have a volume capacity of 16EiB and has a filesize limit of 8EiB per file. With BTRFS you are going to run into hardware limitations long before you reach the limits of the filesystem, whereas with EXT4 you can hit the limitations of the filesystem before you run into hardware limitations with current hardware.

 

[stevech]

I can't see how a cpu page size cache limit is applicable to the topic here.

 

[mdgm]

It's relevant as it explains the 16TB hardware limitation for max volume size with some NAS units.

 

[stevech]

It's relevant as it explains the 16TB hardware limitation for max volume size with some NAS units.

I'm still confused: 16TB file size limit versus a different limit on volume's size.

 

[mdgm]

Which is precisely the point that has been mentioned earlier. That hardware used is important for understanding what the limitations are in a particular case. With EXT4, limitations of the filesystem are being hit, not just hardware limitations now.

Generally speaking for any filesystem if you want a huge volume then something using an Intel 64-bit CPU, able to run the latest NAS OS from the manufacturer with a 6TB disk on the compatibility list is the way to go.

There also things like rebuild times, performance etc. to consider. With ever higher capacity disks rebuild times are getting longer and longer.

Say you do want to use e.g. 6TB disks in a NAS you own and create a volume > 16TB it is advisable to find out what others experiences are like with the same hardware where possible.