Article: Why ZFS?: Pools & Deduplication

[eidolon]

From out medium-sized business experience with ZFS, we have run into a few major "gotchas".

Firstly, RAM required for dedupe is ludicrous. We are running about 24GB RAM for around 4TB of space (we started at 12 or 16 and found it insufficient for the dedupe tables).

ZFS appears to have a known-but-rarely-discussed performance problem: once you hit 70-80% capacity utilisation, system performance grinds to a halt.

Anyone seriously considering ZFS for a production environment should do a shedload of testing (especially with an almost full disk set) first.

 

[John Kotches]

From out medium-sized business experience with ZFS, we have run into a few major "gotchas".

Firstly, RAM required for dedupe is ludicrous. We are running about 24GB RAM for around 4TB of space (we started at 12 or 16 and found it insufficient for the dedupe tables).

ZFS appears to have a known-but-rarely-discussed performance problem: once you hit 70-80% capacity utilisation, system performance grinds to a halt.

Anyone seriously considering ZFS for a production environment should do a shedload of testing (especially with an almost full disk set) first.

I have about 70 servers running ZFS in production daily. At some point, every file system slows down as the free blocks become scarce. This is especially true if you are dealing with larger files, as the free blocks become scattered and the disk heads are doing a lot of seeking to the various free blocks available.

I've worked with ZFS, WAFL (NetApp), VxFS, UFS, Ext2-4 etc. All of them slow down to some extent as the file system approaches capacity.

I haven't run dedupe as the environment is not conducive to dedupe.