From: Trond Myklebust <trondmy@primarydata.com>
To: "bfields@fieldses.org" <bfields@fieldses.org>,
"kolga@netapp.com" <kolga@netapp.com>
Cc: "hch@infradead.org" <hch@infradead.org>,
Trond Myklebust <trondmy@primarydata.com>,
"linux-nfs@vger.kernel.org" <linux-nfs@vger.kernel.org>,
"linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>
Subject: Re: [RFC v1 01/19] fs: Don't copy beyond the end of the file
Date: Wed, 8 Mar 2017 20:18:31 +0000 [thread overview]
Message-ID: <1489004308.3098.10.camel@primarydata.com> (raw)
In-Reply-To: <85310DA6-7270-49AE-A310-76D73678B1B1@netapp.com>
T24gV2VkLCAyMDE3LTAzLTA4IGF0IDE1OjAwIC0wNTAwLCBPbGdhIEtvcm5pZXZza2FpYSB3cm90
ZToNCj4gPiBPbiBNYXIgOCwgMjAxNywgYXQgMjo1MyBQTSwgSi4gQnJ1Y2UgRmllbGRzIDxiZmll
bGRzQGZpZWxkc2VzLm9yZz4NCj4gPiB3cm90ZToNCj4gPiANCj4gPiBPbiBXZWQsIE1hciAwOCwg
MjAxNyBhdCAxMjozMjoxMlBNIC0wNTAwLCBPbGdhIEtvcm5pZXZza2FpYSB3cm90ZToNCj4gPiA+
IA0KPiA+ID4gPiBPbiBNYXIgOCwgMjAxNywgYXQgMTI6MjUgUE0sIENocmlzdG9waCBIZWxsd2ln
IDxoY2hAaW5mcmFkZWFkLm8NCj4gPiA+ID4gcmc+DQo+ID4gPiA+IHdyb3RlOg0KPiA+ID4gPiAN
Cj4gPiA+ID4gT24gV2VkLCBNYXIgMDgsIDIwMTcgYXQgMTI6MDU6MjFQTSAtMDUwMCwgSi4gQnJ1
Y2UgRmllbGRzDQo+ID4gPiA+IHdyb3RlOg0KPiA+ID4gPiA+IFNpbmNlIGNvcHkgaXNuJ3QgYXRv
bWljIHRoYXQgY2hlY2sgaXMgbmV2ZXIgZ29pbmcgdG8gYmUNCj4gPiA+ID4gPiByZWxpYWJsZS4N
Cj4gPiA+ID4gDQo+ID4gPiA+IFRoYXQncyB0cnVlIGZvciBldmVyeXRoaW5nIHRoYXQgQ09QWSBk
b2VzLsKgwqBCeSB0aGF0IGxvZ2ljIHdlDQo+ID4gPiA+IHNob3VsZA0KPiA+ID4gPiBub3QgaW1w
bGVtZW50IGl0IGF0IGFsbCAoYSBsb2dpYyB0aGF0IEknZCBmdWxseSBzdXBwb3J0KQ0KPiA+ID4g
DQo+ID4gPiBJZiB5b3Ugd2VyZSB0byBvbmx5IGtlZXAgQ0xPTkUgdGhlbiB5b3XigJlkIGxvc2Ug
YSBodWdlIHBlcmZvcm1hbmNlDQo+ID4gPiBnYWluDQo+ID4gPiB5b3UgZ2V0IGZyb20gc2VydmVy
LXRvLXNlcnZlciBDT1BZLsKgDQo+ID4gDQo+ID4gWWVzLsKgwqBBbHNvLCBJIHRoaW5rIGNvcHkt
bGlrZSBjb3B5IGltcGxlbWVudGF0aW9ucyBoYXZlIHJlYXNvbmFibGUNCj4gPiBzZW1hbnRpY3Mg
dGhhdCBhcmUgYmFzaWNhbGx5IHRoZSBzYW1lIGFzIHJlYWQ6DQo+ID4gDQo+ID4gCS0gY29weSBj
YW4gcmV0dXJuIHN1Y2Nlc3NmdWxseSB3aXRoIGxlc3MgY29waWVkIHRoYW4gcmVxdWVzdGVkLg0K
PiA+IAktIGl0J3MgZmluZSBmb3IgdGhlIGNvcGllZCByYW5nZSB0byBzdGFydCBhbmQvb3IgZW5k
IHBhc3QgZW5kDQo+ID4gb2YNCj4gPiAJwqDCoGZpbGUsIGl0J2xsIGp1c3QgcmV0dXJuIGEgc2hv
cnQgcmVhZC4NCj4gPiAJLSBBIGNvcHkgb2YgbW9yZSB0aGFuIDAgYnl0ZXMgcmV0dXJuaW5nIDAg
bWVhbnMgeW91J3JlIGF0IGVuZA0KPiA+IG9mDQo+ID4gCcKgwqBmaWxlLg0KPiA+IA0KPiA+IFRo
ZSBwYXJ0aWN1bGFyIHByb2JsZW0gaGVyZSBpcyB0aGF0IHRoYXQgZG9lc24ndCBmaXQgaG93IGNs
b25lDQo+ID4gd29ya3MgYXQNCj4gPiBhbGwuDQo+ID4gDQo+ID4gSXQgZmVlbHMgbGlrZSB3aGF0
IGhhcHBlbmVkIGlzIHRoYXQgY29weV9maWxlX3JhbmdlKCkgd2FzIG1hZGUNCj4gPiBtYWlubHkN
Cj4gPiBmb3IgdGhlIGNsb25lIGNhc2UsIHdpdGggdGhlIGlkZWEgdGhhdCBjb3B5IG1pZ2h0IGJl
IHJlbHVjdGFudGx5DQo+ID4gYWNjZXB0ZWQgYXMgYSBzZWNvbmQtY2xhc3MgaW1wbGVtZW50YXRp
b24uDQoNCkhpc3RvcmljYWxseT8gTm8uLi4gQ2hyaXN0b3BoIGFkZGVkIGNsb25lIGFzIGEgdmFs
aWQgaW1wbGVtZW50YXRpb24gb2YNCmNvcHlfZmlsZV9yYW5nZSgpIGFsbW9zdCBhIHllYXIgYWZ0
ZXIgWmFjaCBhbmQgQW5uYSBkZWZpbmVkIHRoZQ0Kc2VtYW50aWNzIG9mIHZmc19jb3B5X2ZpbGVf
cmFuZ2UoKS4gZ2l0IGJsYW1lIGlzIHlvdXIgZnJpZW5kLi4uDQoNCj4gPiANCj4gPiBCdXQgdGhl
IHBlcmZvcm1hbmNlIGdhaW4gb2YgY29weSBvZmZsb2FkIGlzIHRvbyBiaWcgdG8ganVzdCBpZ25v
cmUsDQo+ID4gYW5kDQo+ID4gaW4gZmFjdCBpdCdzIHdoYXQgY29weV9maWxlX3JhbmdlIGRvZXMg
b24gZXZlcnkgZmlsZXN5c3RlbSBidXQNCj4gPiBidHJmcyBhbmQNCj4gPiBvY2ZzMiAoYW5kIG1h
eWJlIGNpZnM/KSwgc28gSSBkb24ndCB0aGluayB3ZSBjYW4ganVzdCBpZ25vcmUgaXQuDQo+ID4g
DQo+ID4gSWYgd2UgaGFkIHNlcGFyYXRlIGNvcHlfZmlsZV9yYW5nZSBhbmQgY2xvbmVfZmlsZV9y
YW5nZSwgSSAqdGhpbmsqDQo+ID4gaXQNCj4gPiBjb3VsZCBhbGwgYmUgbWFkZSBzZW5zaWJsZS7C
oMKgQW0gSSBtaXNzaW5nIHNvbWV0aGluZz8NCj4gPiANCj4gDQo+IEhvdyB3b3VsZCB0aGUgYXBw
bGljYXRpb24gKGNwKSBrbm93IHdoZW4gdG8gY2FsbCB0aGUgY2xvbmVfZmlsZV9yYW5nZQ0KPiBh
bmQgd2hlbiB0byBjYWxsIGNvcHlfZmlsZV9yYW5nZT8NCg0KY3AgY2FuIHByb2JhYmx5IGNhbGwg
Y29weV9maWxlX3JhbmdlKCksIGJ1dCBhbnkgYXBwbGljYXRpb24gdGhhdCBuZWVkcw0KYXRvbWlj
IHNlbWFudGljcyAoaS5lLiBhIGJpbmFyeSBvcGVyYXRpb24gc3VjY2Vzcy9mYWlsKSBtdXN0IGNh
bGwNCmNsb25lX2ZpbGVfcmFuZ2UoKS4NCg0KLS0gDQpUcm9uZCBNeWtsZWJ1c3QNCkxpbnV4IE5G
UyBjbGllbnQgbWFpbnRhaW5lciwgUHJpbWFyeURhdGENCnRyb25kLm15a2xlYnVzdEBwcmltYXJ5
ZGF0YS5jb20NCg==
WARNING: multiple messages have this Message-ID (diff)
From: Trond Myklebust <trondmy@primarydata.com>
To: "bfields@fieldses.org" <bfields@fieldses.org>,
"kolga@netapp.com" <kolga@netapp.com>
Cc: "hch@infradead.org" <hch@infradead.org>,
Trond Myklebust <trondmy@primarydata.com>,
"linux-nfs@vger.kernel.org" <linux-nfs@vger.kernel.org>,
"linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>
Subject: Re: [RFC v1 01/19] fs: Don't copy beyond the end of the file
Date: Wed, 8 Mar 2017 20:18:31 +0000 [thread overview]
Message-ID: <1489004308.3098.10.camel@primarydata.com> (raw)
In-Reply-To: <85310DA6-7270-49AE-A310-76D73678B1B1@netapp.com>
On Wed, 2017-03-08 at 15:00 -0500, Olga Kornievskaia wrote:
> > On Mar 8, 2017, at 2:53 PM, J. Bruce Fields <bfields@fieldses.org>
> > wrote:
> >
> > On Wed, Mar 08, 2017 at 12:32:12PM -0500, Olga Kornievskaia wrote:
> > >
> > > > On Mar 8, 2017, at 12:25 PM, Christoph Hellwig <hch@infradead.o
> > > > rg>
> > > > wrote:
> > > >
> > > > On Wed, Mar 08, 2017 at 12:05:21PM -0500, J. Bruce Fields
> > > > wrote:
> > > > > Since copy isn't atomic that check is never going to be
> > > > > reliable.
> > > >
> > > > That's true for everything that COPY does. By that logic we
> > > > should
> > > > not implement it at all (a logic that I'd fully support)
> > >
> > > If you were to only keep CLONE then you’d lose a huge performance
> > > gain
> > > you get from server-to-server COPY.
> >
> > Yes. Also, I think copy-like copy implementations have reasonable
> > semantics that are basically the same as read:
> >
> > - copy can return successfully with less copied than requested.
> > - it's fine for the copied range to start and/or end past end
> > of
> > file, it'll just return a short read.
> > - A copy of more than 0 bytes returning 0 means you're at end
> > of
> > file.
> >
> > The particular problem here is that that doesn't fit how clone
> > works at
> > all.
> >
> > It feels like what happened is that copy_file_range() was made
> > mainly
> > for the clone case, with the idea that copy might be reluctantly
> > accepted as a second-class implementation.
Historically? No... Christoph added clone as a valid implementation of
copy_file_range() almost a year after Zach and Anna defined the
semantics of vfs_copy_file_range(). git blame is your friend...
> >
> > But the performance gain of copy offload is too big to just ignore,
> > and
> > in fact it's what copy_file_range does on every filesystem but
> > btrfs and
> > ocfs2 (and maybe cifs?), so I don't think we can just ignore it.
> >
> > If we had separate copy_file_range and clone_file_range, I *think*
> > it
> > could all be made sensible. Am I missing something?
> >
>
> How would the application (cp) know when to call the clone_file_range
> and when to call copy_file_range?
cp can probably call copy_file_range(), but any application that needs
atomic semantics (i.e. a binary operation success/fail) must call
clone_file_range().
--
Trond Myklebust
Linux NFS client maintainer, PrimaryData
trond.myklebust@primarydata.com
next prev parent reply other threads:[~2017-03-08 20:28 UTC|newest]
Thread overview: 56+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-03-02 16:01 [RFC v1 00/19] NFS support for inter and async COPY Olga Kornievskaia
2017-03-02 16:01 ` [RFC v1 01/19] fs: Don't copy beyond the end of the file Olga Kornievskaia
2017-03-02 16:22 ` Christoph Hellwig
2017-03-02 16:34 ` Olga Kornievskaia
2017-03-03 20:47 ` J. Bruce Fields
2017-03-03 21:08 ` Olga Kornievskaia
2017-03-03 21:32 ` J. Bruce Fields
[not found] ` <B3F80DA0-B4F8-4628-88C5-E5C047620F17@netapp.com>
2017-03-04 2:10 ` J. Bruce Fields
2017-03-06 16:27 ` [RFC v1 01/19] " Olga Kornievskaia
2017-03-06 19:09 ` J. Bruce Fields
[not found] ` <924FF7A2-27CD-4848-BD61-748758C2533F-HgOvQuBEEgTQT0dZR+AlfA@public.gmane.org>
2017-03-06 19:23 ` J. Bruce Fields
2017-03-06 19:23 ` J. Bruce Fields
[not found] ` <20170306192301.GB2294-uC3wQj2KruNg9hUCZPvPmw@public.gmane.org>
2017-03-07 14:18 ` Olga Kornievskaia
2017-03-07 14:18 ` Olga Kornievskaia
2017-03-07 14:18 ` Olga Kornievskaia
2017-03-07 23:40 ` [RFC v1 01/19] fs: " Christoph Hellwig
2017-03-08 17:05 ` J. Bruce Fields
2017-03-08 17:25 ` Christoph Hellwig
2017-03-08 17:32 ` Olga Kornievskaia
2017-03-08 19:53 ` J. Bruce Fields
2017-03-08 20:00 ` Olga Kornievskaia
2017-03-08 20:00 ` Olga Kornievskaia
2017-03-08 20:18 ` J. Bruce Fields
2017-03-08 20:18 ` Trond Myklebust [this message]
2017-03-08 20:18 ` Trond Myklebust
2017-03-08 20:32 ` bfields
2017-03-08 20:49 ` Trond Myklebust
2017-03-08 20:49 ` Trond Myklebust
2017-03-09 15:29 ` bfields
2017-03-09 15:35 ` hch
2017-03-09 16:16 ` bfields
2017-03-09 16:17 ` hch
2017-03-09 17:28 ` Olga Kornievskaia
2017-03-09 17:28 ` Olga Kornievskaia
2017-03-09 18:40 ` bfields
2017-03-09 21:55 ` hch
2017-03-09 17:35 ` Olga Kornievskaia
2017-03-09 17:35 ` Olga Kornievskaia
2017-03-02 16:01 ` [RFC v1 02/19] VFS permit cross device vfs_copy_file_range Olga Kornievskaia
2017-03-02 16:01 ` [RFC v1 03/19] VFS don't try clone if superblocks are different Olga Kornievskaia
2017-03-02 16:01 ` [RFC v1 04/19] NFS inter ssc open Olga Kornievskaia
2017-03-02 16:01 ` [RFC v1 05/19] NFS add COPY_NOTIFY operation Olga Kornievskaia
2017-03-02 16:01 ` [RFC v1 06/19] NFS add ca_source_server<> to COPY Olga Kornievskaia
2017-03-02 16:01 ` [RFC v1 07/19] NFS CB_OFFLOAD xdr Olga Kornievskaia
2017-03-02 16:01 ` [RFC v1 08/19] NFS OFFLOAD_STATUS xdr Olga Kornievskaia
2017-03-02 16:01 ` [RFC v1 09/19] NFS OFFLOAD_STATUS op Olga Kornievskaia
2017-03-02 16:01 ` [RFC v1 10/19] NFS OFFLOAD_CANCEL xdr Olga Kornievskaia
2017-03-02 16:01 ` [RFC v1 11/19] NFS COPY xdr handle async reply Olga Kornievskaia
2017-03-02 16:01 ` [RFC v1 12/19] NFS add support for asynchronous COPY Olga Kornievskaia
2017-03-02 16:01 ` [RFC v1 13/19] NFS handle COPY reply CB_OFFLOAD call race Olga Kornievskaia
2017-03-02 16:01 ` [RFC v1 14/19] NFS send OFFLOAD_CANCEL when COPY killed Olga Kornievskaia
2017-03-02 16:01 ` [RFC v1 15/19] NFS make COPY synchronous xdr configurable Olga Kornievskaia
2017-03-02 16:01 ` [RFC v1 16/19] NFS handle COPY ERR_OFFLOAD_NO_REQS Olga Kornievskaia
2017-03-02 16:01 ` [RFC v1 17/19] NFS skip recovery of copy open on dest server Olga Kornievskaia
2017-03-02 16:01 ` [RFC v1 18/19] NFS recover from destination server reboot for copies Olga Kornievskaia
2017-03-02 16:01 ` [RFC v1 19/19] NFS if we got partial copy ignore errors Olga Kornievskaia
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1489004308.3098.10.camel@primarydata.com \
--to=trondmy@primarydata.com \
--cc=bfields@fieldses.org \
--cc=hch@infradead.org \
--cc=kolga@netapp.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-nfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.