Re: Data Deduplication with the help of an online filesystem check

Thomas Glanzmann
2009-04-28T17:37:19Z

Hello Chris,

> > Is there a checksum for every block in btrfs?

> Yes, but they are only crc32c.

I see, is it easily possible to exchange that with sha-1 or md5?

> > Is it possible to retrieve these checksums from userland?

> Not today. The sage developers sent a patch to make an ioctl for
> this, but since it was hard coded to crc32c I haven't taken it yet.

I see.

> Yes, btrfs uses extents but for the purposes of dedup, 4k blocksizes
> are fine.

Does that mean that I can dedup 4k blocks even if you use extents?

> Virtual machines are the ideal dedup workload. But, you do get a big
> portion of the dedup benefits by just starting with a common image and
> cloning it instead of doing copies of each vm.

True, the operating system can be almost completely deduped but as soon
as you start patching you loose the benefit.

Thomas