Re: Data Deduplication with the help of an online filesystem check

[Andrey Kuzmin <andrey.v.kuzmin@gmail.com>]

On Tue, Apr 28, 2009 at 2:02 PM, Chris Mason <chris.mason@oracle.com> w=
rote:
> On Tue, 2009-04-28 at 07:22 +0200, Thomas Glanzmann wrote:
>> Hello Chris,
>>
>> > There is a btrfs ioctl to clone individual files, and this could b=
e used
>> > to implement an online dedup. =A0But, since it is happening from u=
serland,
>> > you can't lock out all of the other users of a given file.
>>
>> > So, the dedup application would be responsible for making sure a g=
iven
>> > file was not being changed while the dedup scan was running.
>>
>> I see, does that mean that I can not do ,,dedup'' for files that are
>> currently opened by a userland program?
>
> No, but it does mean the dedup done from userland is racey. =A0Pictur=
e

Race disappears if (background) dedupe is run against snapshot(s).

Regards,
Andrey

> this:
>
> process A:
> =A0 =A0create some_file # some_file matches the contents of another f=
ile
>
> dedup proc:
> =A0 =A0check some_file
> =A0 =A0decide to start block dedup
>
> process A:
> =A0 =A0 modify some_file
>
> dedup proc:
> =A0 =A0 progress through block dedup
>
> So, this will happily replace blocks in some_file with the dedup bloc=
ks.
> But there's no way to atomically swap them.
>
> We could create new ioctls for this, basically a variant of the clone
> file ioctl that makes sure a given set of pages has a given sum (or
> strict memory contents) before doing the swap.
>
> But they don't exist yet.
>
> -chris
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs=
" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html