copy chunk preliminary results

copy chunk preliminary results

[Steve French]

I extended my initial copy chunk kernel client patch to handle files >
1MB (repeating copychunk requests multiple times, 1 chunk at a time, I
realize that this will be even faster when I send more than 1 chunk at
a time)

Preliminary results local loopback mount of current cifs kernel code
(vers=3.0) to Samba (master branch, running on Fedora 19 with btrfs):

File size of test file (vmlinux.o) was 373987044 bytes

Over the network Copychunk (refcopy) was about six times faster than without.
ane even local copy (no reflink) is slower than remote copy with
reflink.  Copychunk would look even better except that the server disk
is SSD.


[sfrench@pc-on-right cifs-2.6]$ time cp vmlinux.o ~/trgt-1-local
real    0m0.769s
user    0m0.002s
sys    0m0.502s

(Local copy with reflink is amazingly fast on btrfs)
[sfrench@pc-on-right cifs-2.6]$ time cp --reflink vmlinux.o
~/trgt-2-local-reflink
real    0m0.004s
user    0m0.001s
sys    0m0.002s

(remote copy with reflink to Samba was six times faster than remote
with no reflink, similar results when repeated)
[sfrench@pc-on-right cifs-2.6]$ time cp --reflink
/mnt/cifs-2.6/vmlinux.o /mnt/trgt-3-remote-reflink
real    0m0.416s
user    0m0.000s
sys    0m0.029s


[sfrench@pc-on-right cifs-2.6]$ time cp  /mnt/cifs-2.6/vmlinux.o
/mnt/trgt-3-no-reflink
real    0m2.596s
user    0m0.007s
sys    0m0.860s



-- 
Thanks,

Steve

Re: copy chunk preliminary results

[Pádraig Brady]

On 11/04/2013 06:30 AM, Steve French wrote:
> I extended my initial copy chunk kernel client patch to handle files >
> 1MB (repeating copychunk requests multiple times, 1 chunk at a time, I
> realize that this will be even faster when I send more than 1 chunk at
> a time)
> 
> Preliminary results local loopback mount of current cifs kernel code
> (vers=3.0) to Samba (master branch, running on Fedora 19 with btrfs):
> 
> File size of test file (vmlinux.o) was 373987044 bytes
> 
> Over the network Copychunk (refcopy) was about six times faster than without.
> ane even local copy (no reflink) is slower than remote copy with
> reflink.  Copychunk would look even better except that the server disk
> is SSD.
> 
> 
> [sfrench@pc-on-right cifs-2.6]$ time cp vmlinux.o ~/trgt-1-local
> real    0m0.769s
> user    0m0.002s
> sys    0m0.502s
> 
> (Local copy with reflink is amazingly fast on btrfs)
> [sfrench@pc-on-right cifs-2.6]$ time cp --reflink vmlinux.o
> ~/trgt-2-local-reflink
> real    0m0.004s
> user    0m0.001s
> sys    0m0.002s
> 
> (remote copy with reflink to Samba was six times faster than remote
> with no reflink, similar results when repeated)
> [sfrench@pc-on-right cifs-2.6]$ time cp --reflink
> /mnt/cifs-2.6/vmlinux.o /mnt/trgt-3-remote-reflink
> real    0m0.416s
> user    0m0.000s
> sys    0m0.029s
> 
> 
> [sfrench@pc-on-right cifs-2.6]$ time cp  /mnt/cifs-2.6/vmlinux.o
> /mnt/trgt-3-no-reflink
> real    0m2.596s
> user    0m0.007s
> sys    0m0.860s

So cp --reflink has CoW semantics and so probably not an
appropriate interface for this.  Unless I'm misunderstanding,
this SMB copy offload does result in a normal copy on the server right?
So that would imply that cp should try to use the "smb_copy_offload"
ioctl unconditionally after a little introspection, or preferentially
this would be encapsulated in the recently mooted copyfile() level
kernel call, which cp etc. could just call to do the operation.

thanks,
Pádraig.

Re: copy chunk preliminary results

[David Disseldorp]

On Mon, 04 Nov 2013 10:07:22 +0000
Pádraig Brady <P@draigBrady.com> wrote:

> So cp --reflink has CoW semantics and so probably not an
> appropriate interface for this.  Unless I'm misunderstanding,
> this SMB copy offload does result in a normal copy on the server right?

It depends on how the SMB server interprets the copy-chunk wire request.
On Btrfs, Samba can translate the request into a BTRFS_IOC_CLONE_RANGE
ioctl, in which case the same CoW semantics are observed[1]. See:

https://wiki.samba.org/index.php/Server-Side_Copy#Btrfs_Enhanced_Server-Side_Copy_Offload

By default however, Samba (and Windows) will perform the copy on the
server-side using regular reads/writes. A generic cp --offload or
similar would probably make more sense on the client side.

Cheers, David

Re: copy chunk preliminary results

[Christoph Hellwig]

On Thu, Nov 21, 2013 at 11:45:41AM +0100, David Disseldorp wrote:
> It depends on how the SMB server interprets the copy-chunk wire request.
> On Btrfs, Samba can translate the request into a BTRFS_IOC_CLONE_RANGE
> ioctl, in which case the same CoW semantics are observed[1]. See:
> 
> https://wiki.samba.org/index.php/Server-Side_Copy#Btrfs_Enhanced_Server-Side_Copy_Offload
> 
> By default however, Samba (and Windows) will perform the copy on the
> server-side using regular reads/writes. A generic cp --offload or
> similar would probably make more sense on the client side.

I don't think it really matters what the optimal case is, it matters
what the worst case is.  Think about it - a reflink really just is
a smart shortcut for copy + dedup, which a filesystem on the server
could do anyway.

On the other hand a user of cp --reflink expects it to be a quick
operation.

So it's time folks finally get the damn copyfile system call in, use
that for CIFS, NFS and co, as well as letting btrfs optimize it.

Re: copy chunk preliminary results

[Volker Lendecke]

On Thu, Nov 21, 2013 at 03:20:44AM -0800, Christoph Hellwig wrote:
> So it's time folks finally get the damn copyfile system call in, use
> that for CIFS, NFS and co, as well as letting btrfs optimize it.

Has the splice discussion gone anywhere? Sorry for not
following fully, but having splice disk->disk work well
0-copy might have the additional effect that eventually we
get TCP->disk 0-copy.

Volker

Re: copy chunk preliminary results

[Steve French]

On Thu, Nov 21, 2013 at 5:20 AM, Christoph Hellwig <hch-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org> wrote:
> On Thu, Nov 21, 2013 at 11:45:41AM +0100, David Disseldorp wrote:
>> It depends on how the SMB server interprets the copy-chunk wire request.
>> On Btrfs, Samba can translate the request into a BTRFS_IOC_CLONE_RANGE
>> ioctl, in which case the same CoW semantics are observed[1]. See:
>>
>> https://wiki.samba.org/index.php/Server-Side_Copy#Btrfs_Enhanced_Server-Side_Copy_Offload
>>
>> By default however, Samba (and Windows) will perform the copy on the
>> server-side using regular reads/writes. A generic cp --offload or
>> similar would probably make more sense on the client side.

On copychunk Windows server is clearly doing the equivalent of reflink
internally,
so this isn't just Samba/btrfs.  I was getting performance more than 100 times
better using copychunk to Windows than doing copy with sending
reads/writes over the network.

> I don't think it really matters what the optimal case is, it matters
> what the worst case is.  Think about it - a reflink really just is
> a smart shortcut for copy + dedup, which a filesystem on the server
> could do anyway.
>
> On the other hand a user of cp --reflink expects it to be a quick
> operation.

My tests show that in every case the copychunk over SMB2/SMB3 mount is
much faster even in the worst case where it is emulated in Samba.

> So it's time folks finally get the damn copyfile system call in, use
> that for CIFS, NFS and co, as well as letting btrfs optimize it.

I agree that the copyfile system call would be a big help.

-- 
Thanks,

Steve