Re: Copy tools on Linux

[Dave Chinner <david@fromorbit.com>]

On Sun, Jul 01, 2018 at 12:44:43PM -0500, Goldwyn Rodrigues wrote:
> On 07-01 10:10, Dave Chinner wrote:
> > On Fri, Jun 29, 2018 at 09:37:27PM -0500, Steve French wrote:
> > > I have been looking at i/o patterns from various copy tools on Linux,
> > > and it is pretty discouraging - I am hoping that I am forgetting an
> > > important one that someone can point me to ...
> > > 
> > > Some general problems:
> > > 1) if source and target on the same file system it would be nice to
> > > call the copy_file_range syscall (AFAIK only test tools call that),
> > > although in some cases at least cp can do it for --reflink
> > 
> > copy_file_range() should be made to do the right thing in as many
> > scnearios as we can document, and then switch userspace over to use
> > it at all times. Aggregate all the knowledge in one place, where we
> > know what the filesystem implementations are and can get hints to do
> > the right thing.
> 
> We have discussed this earlier in
> https://www.spinics.net/lists/linux-fsdevel/msg125401.html
> and Christoph suggested that there is no point adding new flags
> for holes. Do you have a different opinion?

I don't care. All I was suggesting is that we add flags to...

> The same is with coreutils maintainer where he suggested adding
> flags to perform everything with respect to holes in the kernel.

.... get adoption from the userspace tool maintainers who won't use
anything that doesn't exactly replace what they do now.

And that's the real problem - getting userspace to use all the
go-fast bits we provide. How long have we had the splice interfaces
to do efficient data copies without passing data through userspace?
You'd think using these APIs would be a no-brainer, but nothing uses
them at all. I suspect we're going to see the same thing with
copy_file_range() and similar recent advances in AIO+DIO APIs...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com