Re: Data Deduplication with the help of an online filesystem check

Tomasz Chmielewski
2009-06-05T12:20:48Z

Chris Mason wrote:
> On Thu, Jun 04, 2009 at 10:49:19AM +0200, Thomas Glanzmann wrote:
>> Hello Chris,
>>
>>>> My question is now, how often can a block in btrfs be refferenced?
>>> The exact answer depends on if we are referencing it from a single
>>> file or from multiple files. But either way it is roughly 2^32.
>> could you please explain to me what underlying datastructure is used to
>> monitor if the block is still referenced or already free? Is a counter
>> used, bitmap (but that can't be if is 2^32) or some sort of list? I
>> assume that a counter is used. If this is the case, I assume when a
>> snapshot for example is deleted the reference counter of every block
>> that was referenced in the snapshot will be decremented by one. Is this
>> correct or am I missing something here?
>
> It is a counter and a back reference. With Yan Zheng's new format work,
> the limit is not 2^64.
>
> When a snapshot is deleted, the btree is walked to efficiently drop the
> references on the blocks it referenced.
>
> From a dedup point of view, we'll want the dedup file to hold a
> reference on the file extents. The kernel ioctl side of things will
> take care of that part.

I wonder how well would deduplication work with defragmentation? One
excludes the other to some extent.

--
Tomasz Chmielewski