Re: zero out blocks of freed user data for operation a virtual machine environment

Thomas Glanzmann
2009-05-25T05:26:53Z

Hello David,

[ RESEND: CC forgotten ]

> Are you proposing to de-duplicate a live filesystem?

I do, but on the storage appliance / nfs server and not inside the VM.
But inside VM a filesystem could make the deduplication effort much
easier if it reports unused blocks to the outside world by overwriting
them with zero. I have two scenarios in the moment in my head:

- btrfs has already checksums. I'm at the moment evaluating if
the crc32 is good enough to find candidates for deduplication
or if a stronger checksum is required. After that one patch
needs to be adapted and ioctl needs to be implemented in btrfs
which than double checks if the blocks are for real
duplications of each other and deduplicates them

- btrfs will be at some point be able to generate a list of
blocks that have changed between two transactions. This list
can be used to create an (offsite-backup).

See also: http://thread.gmane.org/gmane.comp.file-systems.btrfs/2922

Thomas

PS: And it seems that NetApp has the above already in a product. They
have the ability to dedup blocks on WAFL and they also have a feature
that allows to have an offsite duplication of the filesystem.