Re: Data Deduplication with the help of an online filesystem check

[Chris Mason <chris.mason@oracle.com>]

On Wed, 2009-04-29 at 14:03 +0200, Thomas Glanzmann wrote:
> Hello Chris,
> 
> > You can start with the code documentation section on
> > http://btrfs.wiki.kernel.org
> 
> I read through this and at the moment one questions come in my mind:
> 
> http://btrfs.wiki.kernel.org/images-btrfs/7/72/Chunks-overview.png
> 
> Looking at this picture, when I'm going to implement the dedup code, do I also
> have to take care to spread the blocks over the different devices or is
> there already infrastructure in place that automates that process?

The layering inside of btrfs means that you don't need to worry about
chunks or multiple devices.

But, in your ioctls you want to deal with [file, offset, len], not
directly with block numbers.  COW means that blocks can move around
without you knowing, and some of the btrfs internals will COW files in
order to relocate storage.

So, what you want is a dedup file (or files) where your DB knows a given
offset in the file has a given csum.

-chris