Saving space with btrfs out-of-band deduplication

Saving space with btrfs out-of-band deduplication

17 August, 2023 • 1 min read

ZFS is cool and everything, and obviously the filesystem of choice on FreeBSD, but there is (at least? only?) one feature that Linux BTRFS has that ZFS doesn’t: out-of-band (“offline”) deduplication.

…because, on the commodity hardware I use, I’d rather use cheap and plentiful HDD storage rather than expensive and scarce RAM for dedup.

Enable ZSTD compression (that ZFS also features, to be fair) and set 𝚍𝚞𝚙𝚎𝚛𝚎𝚖𝚘𝚟𝚎 to run nightly at 03:30.

Result on a data set of all of TECTRA Ltd’s emails since the era of dial-up networking: 30 GB saved on originally 71 GB.

Have something worth building?

A paragraph about what you are working on is enough to start.

Say hello