Re: [ANNOUNCE] Btrfs: a copy on write, snapshotting FS

Vladislav Bolkhovitin
2007-06-20T08:41:45Z

david@lang.hm wrote:
>>> > 3. De-de-duplicate blocks on disk, i.e. copy them on write
>>> > > I suppose that de-duplication itself would be done by some user
>>> space
>>> > process that would scan files, determine blocks with the same data and
>>> > then de-duplicate them by using syscall or IOCTL (2).
>>> > > That would be very usable feature, which in most cases would
>>> allow to
>>> > shrink occupied disk space on 50-90%.
>>>
>>> Have you references for this number?
>>
>>
>> No, I've seen it somewhere and it well confirms with my own observations.
>>
>>> In my experience one gets a lot of benefit from
>>> the much simpler process of "de-duplication" of files.
>>
>>
>> Yes, sure, de-duplication on files level brings its benefits, but on
>> FS blocks level it would bring ever more benefits, because there are
>> many more or less big files, which are different as a whole, but with
>> a lot of the same blocks. Simple example of such files is UNIX-style
>> mail boxes on a mail server.
>
>
> unix style mail boxes would not be a good example of wins for
> sector-based de-duplication since the duplicate mail is not going to be
> sector aligned.

Yes, I realized that after I sent the e-mail. Handling of the same, but
not aligned, data in different files would need more complex logic.
Maybe too complex.

Vlad