VFS support for fast copy on deduplicating FSes

VFS support for fast copy on deduplicating FSes

[David Nicol]

unresearched question/suggestion:

Is there general support for a "fast copy" ioctl in the VFS layer,
which would be hooked by file systems that support COW or other forms
of deduplication and can provide copy semantics by manipulating
metadata only?

-- 
"It is merely a matter of persistence." -- Albert Camus

Re: VFS support for fast copy on deduplicating FSes

[Tomasz Torcz]

On Thu, Nov 25, 2010 at 04:19:17AM -0600, David Nicol wrote:
> unresearched question/suggestion:
> 
> Is there general support for a "fast copy" ioctl in the VFS layer,
> which would be hooked by file systems that support COW or other forms
> of deduplication and can provide copy semantics by manipulating
> metadata only?
> 

  Yes, see reflink(2) syscall (http://lwn.net/Articles/331576/).

-- 
Tomasz Torcz                "Funeral in the morning, IDE hacking
xmpp: zdzichubg@chrome.pl    in the afternoon and evening." - Alan Cox

Re: VFS support for fast copy on deduplicating FSes

[Gordan Bobic]

David Nicol wrote:
> unresearched question/suggestion:
> 
> Is there general support for a "fast copy" ioctl in the VFS layer,
> which would be hooked by file systems that support COW or other forms
> of deduplication and can provide copy semantics by manipulating
> metadata only?

What would be nice to have is something that provides functionality like 
this:
http://www.xmailserver.org/flcow.html

My use case is to do with chroot-type virtualization, to save DLL memory 
and disk space, but the author's use case is clearly different.

Vserver does something similar transparently within the combined 
guest-spaces (i.e. once you hashify the files, you no longer have 10 
instances of the identical glibc for 10 guests, and they are no longer 
using 10x the RAM and disk space), and when the file is touched for 
writing it COWs a new copy.

It would be nice to have something like this exist on FS level without 
need for patching by the likes of vserver, by simply applying a flag to 
files we want to be COW-ed (chattr?).

Having this work across snapshots is potentially a bit dubious. I can 
see why it might be a bad idea, so perhaps a better way might be to have 
something that works semantically and syntactically similar to snapshots 
for duplicating directory subtrees into COW copies?

Gordan

Re: VFS support for fast copy on deduplicating FSes

[Pádraig Brady]

On 25/11/10 10:32, Tomasz Torcz wrote:
> On Thu, Nov 25, 2010 at 04:19:17AM -0600, David Nicol wrote:
>> unresearched question/suggestion:
>>
>> Is there general support for a "fast copy" ioctl in the VFS layer,
>> which would be hooked by file systems that support COW or other forms
>> of deduplication and can provide copy semantics by manipulating
>> metadata only?
>>
> 
>   Yes, see reflink(2) syscall (http://lwn.net/Articles/331576/).
> 

Yep and this: http://lwn.net/Articles/335380/
Note that cp --reflink has been supported for a while,
but it's BTRFS specific at the moment.
We'll need to change to the more generic interface.

cheers,
Pádraig.