Saving space with btrfs out-of-band deduplication

Saving space with btrfs out-of-band deduplication

17 August, 2023 1 min read
ZFS, btrfs, deduplication, storage

ZFS is cool and everything, and obviously the filesystem of choice on FreeBSD, but there is (at least? only?) one feature that Linux BTRFS has that ZFS doesn’t: out-of-band (“offline”) deduplication.

…because, on the commodity hardware I use, I’d rather use cheap and plentiful HDD storage rather than expensive and scarce RAM for dedup.

Enable ZSTD compression (that ZFS also features, to be fair) and set 𝚍𝚞𝚙𝚎𝚛𝚎𝚖𝚘𝚟𝚎 to run nightly at 03:30.

Result on a data set of all of TECTRA Ltd’s emails since the era of dial-up networking: 30 GB saved on originally 71 GB.