From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ipmail06.adl6.internode.on.net ([150.101.137.145]:57852 "EHLO ipmail06.adl6.internode.on.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752625AbeGBARI (ORCPT ); Sun, 1 Jul 2018 20:17:08 -0400 Date: Mon, 2 Jul 2018 10:17:05 +1000 From: Dave Chinner To: Goldwyn Rodrigues Cc: Steve French , linux-fsdevel Subject: Re: Copy tools on Linux Message-ID: <20180702001705.GS19934@dastard> References: <20180701001005.GR19934@dastard> <20180701174443.6lrpuurx5stsyvja@merlin> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20180701174443.6lrpuurx5stsyvja@merlin> Sender: linux-fsdevel-owner@vger.kernel.org List-ID: On Sun, Jul 01, 2018 at 12:44:43PM -0500, Goldwyn Rodrigues wrote: > On 07-01 10:10, Dave Chinner wrote: > > On Fri, Jun 29, 2018 at 09:37:27PM -0500, Steve French wrote: > > > I have been looking at i/o patterns from various copy tools on Linux, > > > and it is pretty discouraging - I am hoping that I am forgetting an > > > important one that someone can point me to ... > > > > > > Some general problems: > > > 1) if source and target on the same file system it would be nice to > > > call the copy_file_range syscall (AFAIK only test tools call that), > > > although in some cases at least cp can do it for --reflink > > > > copy_file_range() should be made to do the right thing in as many > > scnearios as we can document, and then switch userspace over to use > > it at all times. Aggregate all the knowledge in one place, where we > > know what the filesystem implementations are and can get hints to do > > the right thing. > > We have discussed this earlier in > https://www.spinics.net/lists/linux-fsdevel/msg125401.html > and Christoph suggested that there is no point adding new flags > for holes. Do you have a different opinion? I don't care. All I was suggesting is that we add flags to... > The same is with coreutils maintainer where he suggested adding > flags to perform everything with respect to holes in the kernel. .... get adoption from the userspace tool maintainers who won't use anything that doesn't exactly replace what they do now. And that's the real problem - getting userspace to use all the go-fast bits we provide. How long have we had the splice interfaces to do efficient data copies without passing data through userspace? You'd think using these APIs would be a no-brainer, but nothing uses them at all. I suspect we're going to see the same thing with copy_file_range() and similar recent advances in AIO+DIO APIs... Cheers, Dave. -- Dave Chinner david@fromorbit.com