NFSv4.2 server replies to Copy with length == 0

NFSv4.2 server replies to Copy with length == 0

[Rick Macklem]

[-- Attachment #1: Type: text/plain, Size: 558 bytes --]

During interop testing (FreeBSD client->Linux server) of NFSv4.2, my
client got into a loop. It was because the reply to Copy was NFS_OK,
but the length in the reply is 0.
(I'll fix the client to fail for this case so it doesn't loop, but...)

The server is Fedora 30 (5.2.18-200 kernel version).
It you think this might be fixed in a newer kernel or you'd like me to do
something with it to get more info, just let me know.

I've attached a snippet of the packet trace. (If the list strips it off, just email
me and I'll send it to you.)

rick

[-- Attachment #2: yyy.pcap --]
[-- Type: application/octet-stream, Size: 10000 bytes --]

Re: NFSv4.2 server replies to Copy with length == 0

[Rick Macklem]

It seems that the Copy reply with wr_count == 0 occurs when the client
sends a Copy request with ca_src_offset beyond EOF in the input file.
(It happened because I was testing an old/broken version of my client,
 but I can reproduce it, if you need a bugfix to be tested. I don't know if
 the case of ca_src_offset+ca_count beyond EOF behaves the same?)
--> The RFC seems to require a reply of NFS4ERR_INVAL for this case.

rick

________________________________________
From: linux-nfs-owner@vger.kernel.org <linux-nfs-owner@vger.kernel.org> on behalf of Rick Macklem <rmacklem@uoguelph.ca>
Sent: Tuesday, October 15, 2019 10:50 PM
To: linux-nfs@vger.kernel.org
Subject: NFSv4.2 server replies to Copy with length == 0

During interop testing (FreeBSD client->Linux server) of NFSv4.2, my
client got into a loop. It was because the reply to Copy was NFS_OK,
but the length in the reply is 0.
(I'll fix the client to fail for this case so it doesn't loop, but...)

The server is Fedora 30 (5.2.18-200 kernel version).
It you think this might be fixed in a newer kernel or you'd like me to do
something with it to get more info, just let me know.

I've attached a snippet of the packet trace. (If the list strips it off, just email
me and I'll send it to you.)

rick

Re: NFSv4.2 server replies to Copy with length == 0

[J. Bruce Fields]

On Wed, Oct 16, 2019 at 06:22:42AM +0000, Rick Macklem wrote:
> It seems that the Copy reply with wr_count == 0 occurs when the client
> sends a Copy request with ca_src_offset beyond EOF in the input file.
> (It happened because I was testing an old/broken version of my client,
>  but I can reproduce it, if you need a bugfix to be tested. I don't know if
>  the case of ca_src_offset+ca_count beyond EOF behaves the same?)
> --> The RFC seems to require a reply of NFS4ERR_INVAL for this case.

I've never understood that INVAL requirement.  But I know it's been
discussed before, maybe there was some justification for it that I've
forgotten.

--b.

> 
> rick
> 
> ________________________________________
> From: linux-nfs-owner@vger.kernel.org <linux-nfs-owner@vger.kernel.org> on behalf of Rick Macklem <rmacklem@uoguelph.ca>
> Sent: Tuesday, October 15, 2019 10:50 PM
> To: linux-nfs@vger.kernel.org
> Subject: NFSv4.2 server replies to Copy with length == 0
> 
> During interop testing (FreeBSD client->Linux server) of NFSv4.2, my
> client got into a loop. It was because the reply to Copy was NFS_OK,
> but the length in the reply is 0.
> (I'll fix the client to fail for this case so it doesn't loop, but...)
> 
> The server is Fedora 30 (5.2.18-200 kernel version).
> It you think this might be fixed in a newer kernel or you'd like me to do
> something with it to get more info, just let me know.
> 
> I've attached a snippet of the packet trace. (If the list strips it off, just email
> me and I'll send it to you.)
> 
> rick

Re: NFSv4.2 server replies to Copy with length == 0

[Kornievskaia, Olga]

On 10/16/19, 11:58 AM, "J. Bruce Fields" <bfields@fieldses.org> wrote:

    NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
    
    
    
    
    On Wed, Oct 16, 2019 at 06:22:42AM +0000, Rick Macklem wrote:
    > It seems that the Copy reply with wr_count == 0 occurs when the client
    > sends a Copy request with ca_src_offset beyond EOF in the input file.
    > (It happened because I was testing an old/broken version of my client,
    >  but I can reproduce it, if you need a bugfix to be tested. I don't know if
    >  the case of ca_src_offset+ca_count beyond EOF behaves the same?)
    > --> The RFC seems to require a reply of NFS4ERR_INVAL for this case.
    
    I've never understood that INVAL requirement.  But I know it's been
    discussed before, maybe there was some justification for it that I've
    forgotten.
    

Sigh, well, I don’t know if we should consider adding the check to the NFS server to be NFS spec compliant. VFS layer didn't want the check and instead the preference has been to keep read() semantics of returning a short read (when the len was beyond the end of the file or if the source) to something beyond the end of the file. On the client if VFS did read of len=0 then VFS itself we return 0, thus this doesn't protect against other clients sending an NFS copy with len=0. And in NFS, receiving copy with len=0 means copy to the end of the file. It's not implemented for any "intra" or "inter" code. 

    --b.
    
    >
    > rick
    >
    > ________________________________________
    > From: linux-nfs-owner@vger.kernel.org <linux-nfs-owner@vger.kernel.org> on behalf of Rick Macklem <rmacklem@uoguelph.ca>
    > Sent: Tuesday, October 15, 2019 10:50 PM
    > To: linux-nfs@vger.kernel.org
    > Subject: NFSv4.2 server replies to Copy with length == 0
    >
    > During interop testing (FreeBSD client->Linux server) of NFSv4.2, my
    > client got into a loop. It was because the reply to Copy was NFS_OK,
    > but the length in the reply is 0.
    > (I'll fix the client to fail for this case so it doesn't loop, but...)
    >
    > The server is Fedora 30 (5.2.18-200 kernel version).
    > It you think this might be fixed in a newer kernel or you'd like me to do
    > something with it to get more info, just let me know.
    >
    > I've attached a snippet of the packet trace. (If the list strips it off, just email
    > me and I'll send it to you.)
    >
    > rick
    

Re: NFSv4.2 server replies to Copy with length == 0

[J. Bruce Fields]

On Wed, Oct 16, 2019 at 07:53:45PM +0000, Kornievskaia, Olga wrote:
> On 10/16/19, 11:58 AM, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>     On Wed, Oct 16, 2019 at 06:22:42AM +0000, Rick Macklem wrote:
>     > It seems that the Copy reply with wr_count == 0 occurs when the
>     > client sends a Copy request with ca_src_offset beyond EOF in the
>     > input file.  (It happened because I was testing an old/broken
>     > version of my client, but I can reproduce it, if you need a
>     > bugfix to be tested. I don't know if the case of
>     > ca_src_offset+ca_count beyond EOF behaves the same?) --> The RFC
>     > seems to require a reply of NFS4ERR_INVAL for this case.
>     
>     I've never understood that INVAL requirement.  But I know it's
>     been discussed before, maybe there was some justification for it
>     that I've forgotten.
> 
> Sigh, well, I don’t know if we should consider adding the check to the
> NFS server to be NFS spec compliant. VFS layer didn't want the check
> and instead the preference has been to keep read() semantics of
> returning a short read (when the len was beyond the end of the file or
> if the source) to something beyond the end of the file.

I'm inclined to think the spec's just wrong.

And how else could a client possibly interpret a 0 return?

> On the client if VFS did read of len=0 then VFS itself we return 0,
> thus this doesn't protect against other clients sending an NFS copy
> with len=0.  And in NFS, receiving copy with len=0 means copy to the
> end of the file. It's not implemented for any "intra" or "inter" code. 

A call with len=0 sounds like a different case.

--b.

Re: NFSv4.2 server replies to Copy with length == 0

[Rick Macklem]

I have added nfsv4@ietf.org to the cc list, since I think it is important that
everyone involved in NFSv4.2 be aware of this.

>J. Bruce Fields wrote:
>On Wed, Oct 16, 2019 at 07:53:45PM +0000, Kornievskaia, Olga wrote:
>> On 10/16/19, 11:58 AM, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>     On Wed, Oct 16, 2019 at 06:22:42AM +0000, Rick Macklem wrote:
>>     > It seems that the Copy reply with wr_count == 0 occurs when the
>>     > client sends a Copy request with ca_src_offset beyond EOF in the
>>     > input file.  (It happened because I was testing an old/broken
>>     > version of my client, but I can reproduce it, if you need a
>>     > bugfix to be tested. I don't know if the case of
>>     > ca_src_offset+ca_count beyond EOF behaves the same?) --> The RFC
>>     > seems to require a reply of NFS4ERR_INVAL for this case.
>>
>>     I've never understood that INVAL requirement.  But I know it's
>>     been discussed before, maybe there was some justification for it
>>     that I've forgotten.
No longer overly relevant, since the RFC is now a published spec.
(It may have just been consistent with the old copy_file_range(2) semantics?)

>>
>> Sigh, well, I don’t know if we should consider adding the check to the
>> NFS server to be NFS spec compliant. VFS layer didn't want the check
>> and instead the preference has been to keep read() semantics of
>> returning a short read (when the len was beyond the end of the file or
>> if the source) to something beyond the end of the file.
>
>I'm inclined to think the spec's just wrong.
The RFC is not wrong, although it might not be your preferred semantic.
(It would only be wrong if it was unimplementable.)

As for patching the server, I am not sure if that is a good idea now or not?
- I would like to see the Linux NFSv4.2 server conform to the RFC, however
   it has already shipped in its current form.
--> I think that all extant clients need to handle both reply semantics and,
       once that is the case, patching the server to conform to the RFC would
       be nice.

I have now found two cases where the Linux NFSv4.2 server does not conform
to RFC-7862. One is as above and the other is a reply to Seek of NFS4ERR_NXIO
when the sa_offset argument == file_size (instead of replying NFS_OK along
with sr_eof == true).

Since I now know about these two cases, I can code the client to handle
both of them. Of course, I do worry about others that I have not found yet.

>And how else could a client possibly interpret a 0 return?
The FreeBSD client looped, because it does a Copy in a loop until all bytes
were copied and, with a reply length == 0, it didn't make progress or
recognize an error.

Fortunately, this client is still under development (not yet released), so I can
add code to handle wr_count == 0 in the same way as NFS4ERR_INVAL.

>> On the client if VFS did read of len=0 then VFS itself we return 0,
>> thus this doesn't protect against other clients sending an NFS copy
>> with len=0.  And in NFS, receiving copy with len=0 means copy to the
>> end of the file. It's not implemented for any "intra" or "inter" code.
Are you saying that an NFSv4.2 Copy request with a ca_count == 0
will not work for the Linux NFSv4.2 server?
(I guess I'd better test this one, too.)

>A call with len=0 sounds like a different case.
>
>--b.

rick

Re: NFSv4.2 server replies to Copy with length == 0

[Rick Macklem]

Rick wrote:
>I have added nfsv4@ietf.org to the cc list, since I think it is important that
>everyone involved in NFSv4.2 be aware of this.
>
>>J. Bruce Fields wrote:
>>On Wed, Oct 16, 2019 at 07:53:45PM +0000, Kornievskaia, Olga wrote:
>>> On 10/16/19, 11:58 AM, "J. Bruce Fields" <bfields@fieldses.org> wrote:
>>>     On Wed, Oct 16, 2019 at 06:22:42AM +0000, Rick Macklem wrote:
>>>     > It seems that the Copy reply with wr_count == 0 occurs when the
>>>     > client sends a Copy request with ca_src_offset beyond EOF in the
>>>     > input file.  (It happened because I was testing an old/broken
>>>     > version of my client, but I can reproduce it, if you need a
>>>     > bugfix to be tested. I don't know if the case of
>>>     > ca_src_offset+ca_count beyond EOF behaves the same?) --> The RFC
>>>     > seems to require a reply of NFS4ERR_INVAL for this case.
>>>
>>>     I've never understood that INVAL requirement.  But I know it's
>>>     been discussed before, maybe there was some justification for it
>>>     that I've forgotten.
>No longer overly relevant, since the RFC is now a published spec.
>(It may have just been consistent with the old copy_file_range(2) semantics?)
>
>>>
>>> Sigh, well, I don’t know if we should consider adding the check to the
>>> NFS server to be NFS spec compliant. VFS layer didn't want the check
>>> and instead the preference has been to keep read() semantics of
>>> returning a short read (when the len was beyond the end of the file or
>>> if the source) to something beyond the end of the file.
>>
>>I'm inclined to think the spec's just wrong.
>The RFC is not wrong, although it might not be your preferred semantic.
>(It would only be wrong if it was unimplementable.)
>
>As for patching the server, I am not sure if that is a good idea now or not?
>- I would like to see the Linux NFSv4.2 server conform to the RFC, however
>   it has already shipped in its current form.
>--> I think that all extant clients need to handle both reply semantics and,
>       once that is the case, patching the server to conform to the RFC would
>       be nice.
>
>I have now found two cases where the Linux NFSv4.2 server does not conform
>to RFC-7862. One is as above and the other is a reply to Seek of NFS4ERR_NXIO
>when the sa_offset argument == file_size (instead of replying NFS_OK along
>with sr_eof == true).
>
>Since I now know about these two cases, I can code the client to handle
>both of them. Of course, I do worry about others that I have not found yet.
>
>>And how else could a client possibly interpret a 0 return?
>The FreeBSD client looped, because it does a Copy in a loop until all bytes
>were copied and, with a reply length == 0, it didn't make progress or
>recognize an error.
>
>Fortunately, this client is still under development (not yet released), so I can
>add code to handle wr_count == 0 in the same way as NFS4ERR_INVAL.
>
>>> On the client if VFS did read of len=0 then VFS itself we return 0,
>>> thus this doesn't protect against other clients sending an NFS copy
>>> with len=0.  And in NFS, receiving copy with len=0 means copy to the
>>> end of the file. It's not implemented for any "intra" or "inter" code.
>Are you saying that an NFSv4.2 Copy request with a ca_count == 0
>will not work for the Linux NFSv4.2 server?
>(I guess I'd better test this one, too.)
Tested it and it does not work, at least for Fedora30.
The server just returns 0 instead of doing a copy to EOF on the input file.
I think you should implement this, although my client does not do this now.

>>A call with len=0 sounds like a different case.
Yes, as above.
>>
>>--b.

rick

Re: NFSv4.2 server replies to Copy with length == 0

[J. Bruce Fields]

On Thu, Oct 17, 2019 at 04:43:58AM +0000, Rick Macklem wrote:
> >>On Wed, Oct 16, 2019 at 07:53:45PM +0000, Kornievskaia, Olga wrote:
> >>> On the client if VFS did read of len=0 then VFS itself we return 0,
> >>> thus this doesn't protect against other clients sending an NFS copy
> >>> with len=0.  And in NFS, receiving copy with len=0 means copy to the
> >>> end of the file. It's not implemented for any "intra" or "inter" code.
> >Are you saying that an NFSv4.2 Copy request with a ca_count == 0
> >will not work for the Linux NFSv4.2 server?
> >(I guess I'd better test this one, too.)
> Tested it and it does not work, at least for Fedora30.
> The server just returns 0 instead of doing a copy to EOF on the input file.
> I think you should implement this, although my client does not do this now.

Agreed.

--b.

Re: NFSv4.2 server replies to Copy with length == 0

[J. Bruce Fields]

On Thu, Oct 17, 2019 at 02:16:36AM +0000, Rick Macklem wrote:
> I have now found two cases where the Linux NFSv4.2 server does not
> conform to RFC-7862. One is as above and the other is a reply to Seek
> of NFS4ERR_NXIO when the sa_offset argument == file_size (instead of
> replying NFS_OK along with sr_eof == true).

Huh.  Looks like that's documented behavior of Linux's seek.  (See the
ERRORS section of the lseek(2) man page.)  Looks like Solaris also
returns -ENXIO in this case:

	https://docs.oracle.com/cd/E26502_01/html/E29032/lseek-2.html

And freebsd too:

	https://www.freebsd.org/cgi/man.cgi?query=lseek&sektion=2

I wonder where that spec language came from?

Our NFS server could translate an -ENXIO return into 0 and sr_eof ==
true easily enough, assuming -ENXIO is really only ever returned in that
case.

I haven't tested, but from a quick check of the Linux client code I
think that would require a matching fix on the client side to translate
sr_eof == 0 *back* to ENXIO.

I don't know if it's worth it.

--b.

Re: NFSv4.2 server replies to Copy with length == 0

[Tom Talpey]

On 10/17/2019 11:22 AM, J. Bruce Fields wrote:
> On Thu, Oct 17, 2019 at 02:16:36AM +0000, Rick Macklem wrote:
>> I have now found two cases where the Linux NFSv4.2 server does not
>> conform to RFC-7862. One is as above and the other is a reply to Seek
>> of NFS4ERR_NXIO when the sa_offset argument == file_size (instead of
>> replying NFS_OK along with sr_eof == true).
> 
> Huh.  Looks like that's documented behavior of Linux's seek.  (See the
> ERRORS section of the lseek(2) man page.)  Looks like Solaris also
> returns -ENXIO in this case:
> 
> 	https://docs.oracle.com/cd/E26502_01/html/E29032/lseek-2.html
> 
> And freebsd too:
> 
> 	https://www.freebsd.org/cgi/man.cgi?query=lseek&sektion=2
> 
> I wonder where that spec language came from?

Those manpages look like ENXIO comes back only on sparse files. Perhaps
this is boilerplate from v4.0 before this kind of thing was common.

This should at least be discussed on nfsv4@ietf.org...

> Our NFS server could translate an -ENXIO return into 0 and sr_eof ==
> true easily enough, assuming -ENXIO is really only ever returned in that
> case.
> 
> I haven't tested, but from a quick check of the Linux client code I
> think that would require a matching fix on the client side to translate
> sr_eof == 0 *back* to ENXIO.
> 
> I don't know if it's worth it.

What Bad Thing would happen for the difference?

Re: NFSv4.2 server replies to Copy with length == 0

[Rick Macklem]

Tom Talpey wrote:
>On 10/17/2019 11:22 AM, J. Bruce Fields wrote:
>> On Thu, Oct 17, 2019 at 02:16:36AM +0000, Rick Macklem wrote:
>>> I have now found two cases where the Linux NFSv4.2 server does not
>>> conform to RFC-7862. One is as above and the other is a reply to Seek
>>> of NFS4ERR_NXIO when the sa_offset argument == file_size (instead of
>>> replying NFS_OK along with sr_eof == true).
>>
>> Huh.  Looks like that's documented behavior of Linux's seek.  (See the
>> ERRORS section of the lseek(2) man page.)  Looks like Solaris also
>> returns -ENXIO in this case:
>>
>>       https://docs.oracle.com/cd/E26502_01/html/E29032/lseek-2.html
>>
>> And freebsd too:
>>
>>       https://www.freebsd.org/cgi/man.cgi?query=lseek&sektion=2
>>
>> I wonder where that spec language came from?
>
>Those manpages look like ENXIO comes back only on sparse files. Perhaps
>this is boilerplate from v4.0 before this kind of thing was common.
As far as I know, a non-sparse file (no holes) behaves the same as a file with
holes in it. (I'm not sure if the POSIX draft is clear for the case where the
file system does not support holes, but I think most implementations handle
that the same way as a file with no holes on a file system that supports holes.)

>This should at least be discussed on nfsv4@ietf.org...
I think there needs to be a discussion on how to best deal with cases where
implementations have shipped to users with these glitches in them.

I think it might be better to document them, so that client coders can
implement work arounds (I am doing so for the three cases I've found)
instead of the server being patched to change its behaviour, causing
potentially more interoperability issues.
(That's why I re-posted this to nfsv4@ietf.org.)

>> Our NFS server could translate an -ENXIO return into 0 and sr_eof ==
>> true easily enough, assuming -ENXIO is really only ever returned in that
>> case.
>>
>> I haven't tested, but from a quick check of the Linux client code I
>> think that would require a matching fix on the client side to translate
>> sr_eof == 0 *back* to ENXIO.
>>
>> I don't know if it's worth it.
>
>What Bad Thing would happen for the difference?
Well, for a POSIX draft style client, the only effect is that a client may be
broken (not handle the sr_eof == true reply correctly) without the
implementors knowing that, since they tested against the Linux server.
(I believe this is the current status of the Linux client.)

For a server implementor (I get to wear both hats;-), the server can only
generate one reply or the other. I've implemented both with a tunable
that can be used to flip between them. I default to the NFS4ERR_NXIO
reply, since that is what the Linux client expects.

As for the Copy issues, the client can easily handle them, but the client
coder needs to know about them.
At this time the FreeBSD server code only implements what is in the RFC.
This seems sufficient for the testing I've done sofar.
The case that I think will break would be Linux code that does a
copy_file_range(infd, NULL, outfd, NULL, INT_MAX, 0) to copy an entire
file. I'll test this case to-day, but I'm think it will just get a NFS4ERR_INVAL
reply from the FreeBSD server.
(I can make this work for the Linux client, but it will take another "cheat"
 enabled via a tunable.)
--> Part of the confusion here comes from the fact that the Linux syscall
       semantics for copy_file_range() has changed. (I also where the
       copy_file_range(2) hat for FreeBSD, so I've got some work to do there, too.)

In summary, I think that, since Linux has been shipping this for some time,
documenting workarounds is a practical approach.

rick

Re: [nfsv4] NFSv4.2 server replies to Copy with length == 0

[Olga Kornievskaia]

On Thu, Oct 17, 2019 at 12:20 PM Rick Macklem <rmacklem@uoguelph.ca> wrote:
>
> Tom Talpey wrote:
> >On 10/17/2019 11:22 AM, J. Bruce Fields wrote:
> >> On Thu, Oct 17, 2019 at 02:16:36AM +0000, Rick Macklem wrote:
> >>> I have now found two cases where the Linux NFSv4.2 server does not
> >>> conform to RFC-7862. One is as above and the other is a reply to Seek
> >>> of NFS4ERR_NXIO when the sa_offset argument == file_size (instead of
> >>> replying NFS_OK along with sr_eof == true).
> >>
> >> Huh.  Looks like that's documented behavior of Linux's seek.  (See the
> >> ERRORS section of the lseek(2) man page.)  Looks like Solaris also
> >> returns -ENXIO in this case:
> >>
> >>       https://docs.oracle.com/cd/E26502_01/html/E29032/lseek-2.html
> >>
> >> And freebsd too:
> >>
> >>       https://www.freebsd.org/cgi/man.cgi?query=lseek&sektion=2
> >>
> >> I wonder where that spec language came from?
> >
> >Those manpages look like ENXIO comes back only on sparse files. Perhaps
> >this is boilerplate from v4.0 before this kind of thing was common.
> As far as I know, a non-sparse file (no holes) behaves the same as a file with
> holes in it. (I'm not sure if the POSIX draft is clear for the case where the
> file system does not support holes, but I think most implementations handle
> that the same way as a file with no holes on a file system that supports holes.)
>
> >This should at least be discussed on nfsv4@ietf.org...
> I think there needs to be a discussion on how to best deal with cases where
> implementations have shipped to users with these glitches in them.
>
> I think it might be better to document them, so that client coders can
> implement work arounds (I am doing so for the three cases I've found)
> instead of the server being patched to change its behaviour, causing
> potentially more interoperability issues.
> (That's why I re-posted this to nfsv4@ietf.org.)
>
> >> Our NFS server could translate an -ENXIO return into 0 and sr_eof ==
> >> true easily enough, assuming -ENXIO is really only ever returned in that
> >> case.
> >>
> >> I haven't tested, but from a quick check of the Linux client code I
> >> think that would require a matching fix on the client side to translate
> >> sr_eof == 0 *back* to ENXIO.
> >>
> >> I don't know if it's worth it.
> >
> >What Bad Thing would happen for the difference?
> Well, for a POSIX draft style client, the only effect is that a client may be
> broken (not handle the sr_eof == true reply correctly) without the
> implementors knowing that, since they tested against the Linux server.
> (I believe this is the current status of the Linux client.)
>
> For a server implementor (I get to wear both hats;-), the server can only
> generate one reply or the other. I've implemented both with a tunable
> that can be used to flip between them. I default to the NFS4ERR_NXIO
> reply, since that is what the Linux client expects.
>
> As for the Copy issues, the client can easily handle them, but the client
> coder needs to know about them.
> At this time the FreeBSD server code only implements what is in the RFC.
> This seems sufficient for the testing I've done sofar.
> The case that I think will break would be Linux code that does a
> copy_file_range(infd, NULL, outfd, NULL, INT_MAX, 0) to copy an entire
> file. I'll test this case to-day, but I'm think it will just get a NFS4ERR_INVAL
> reply from the FreeBSD server.

this might be problematic for the linux client because though there
isn't an official use of it but the libc cp might implement this doing
exactly what you say here copy_file_range(infd, NULL, outfd, NULL,
INT_MAX, 0). It doesn't try to determine the size of the file before
the copy and
just copy max bytes assuming that it'll get a short read at some point.

> (I can make this work for the Linux client, but it will take another "cheat"
>  enabled via a tunable.)
> --> Part of the confusion here comes from the fact that the Linux syscall
>        semantics for copy_file_range() has changed. (I also where the
>        copy_file_range(2) hat for FreeBSD, so I've got some work to do there, too.)
>
> In summary, I think that, since Linux has been shipping this for some time,
> documenting workarounds is a practical approach.
>
> rick
>
> _______________________________________________
> nfsv4 mailing list
> nfsv4@ietf.org
> https://www.ietf.org/mailman/listinfo/nfsv4

Re: [nfsv4] NFSv4.2 server replies to Copy with length == 0

[Rick Macklem]

Olga Kornievskaia wrote:
>On Thu, Oct 17, 2019 at 12:20 PM Rick Macklem <rmacklem@uoguelph.ca> >wrote:
>>
>> Tom Talpey wrote:
>> >On 10/17/2019 11:22 AM, J. Bruce Fields wrote:
>> >> On Thu, Oct 17, 2019 at 02:16:36AM +0000, Rick Macklem wrote:
>> >>> I have now found two cases where the Linux NFSv4.2 server does not
>> >>> conform to RFC-7862. One is as above and the other is a reply to Seek
>> >>> of NFS4ERR_NXIO when the sa_offset argument == file_size (instead of
>> >>> replying NFS_OK along with sr_eof == true).
>> >>
>> >> Huh.  Looks like that's documented behavior of Linux's seek.  (See the
>> >> ERRORS section of the lseek(2) man page.)  Looks like Solaris also
>> >> returns -ENXIO in this case:
>> >>
>> >>       https://docs.oracle.com/cd/E26502_01/html/E29032/lseek-2.html
>> >>
>> >> And freebsd too:
>> >>
>> >>       https://www.freebsd.org/cgi/man.cgi?query=lseek&sektion=2
>> >>
>> >> I wonder where that spec language came from?
>> >
>> >Those manpages look like ENXIO comes back only on sparse files. Perhaps
>> >this is boilerplate from v4.0 before this kind of thing was common.
>> As far as I know, a non-sparse file (no holes) behaves the same as a file with
>> holes in it. (I'm not sure if the POSIX draft is clear for the case where the
>> file system does not support holes, but I think most implementations handle
>> that the same way as a file with no holes on a file system that supports holes.)
>>
>> >This should at least be discussed on nfsv4@ietf.org...
>> I think there needs to be a discussion on how to best deal with cases where
>> implementations have shipped to users with these glitches in them.
>>
>> I think it might be better to document them, so that client coders can
>> implement work arounds (I am doing so for the three cases I've found)
>> instead of the server being patched to change its behaviour, causing
>> potentially more interoperability issues.
>> (That's why I re-posted this to nfsv4@ietf.org.)
>>
>> >> Our NFS server could translate an -ENXIO return into 0 and sr_eof ==
>> >> true easily enough, assuming -ENXIO is really only ever returned in that
>> >> case.
>> >>
>> >> I haven't tested, but from a quick check of the Linux client code I
>> >> think that would require a matching fix on the client side to translate
>> >> sr_eof == 0 *back* to ENXIO.
>> >>
>> >> I don't know if it's worth it.
>> >
>> >What Bad Thing would happen for the difference?
>> Well, for a POSIX draft style client, the only effect is that a client may be
>> broken (not handle the sr_eof == true reply correctly) without the
>> implementors knowing that, since they tested against the Linux server.
>> (I believe this is the current status of the Linux client.)
>>
>> For a server implementor (I get to wear both hats;-), the server can only
>> generate one reply or the other. I've implemented both with a tunable
>> that can be used to flip between them. I default to the NFS4ERR_NXIO
>> reply, since that is what the Linux client expects.
>>
>> As for the Copy issues, the client can easily handle them, but the client
>> coder needs to know about them.
>> At this time the FreeBSD server code only implements what is in the RFC.
>> This seems sufficient for the testing I've done sofar.
>> The case that I think will break would be Linux code that does a
>> copy_file_range(infd, NULL, outfd, NULL, INT_MAX, 0) to copy an entire
>> file. I'll test this case to-day, but I'm think it will just get a NFS4ERR_INVAL
>> reply from the FreeBSD server.
>
>this might be problematic for the linux client because though there
>isn't an official use of it but the libc cp might implement this doing
>exactly what you say here copy_file_range(infd, NULL, outfd, NULL,
>INT_MAX, 0). It doesn't try to determine the size of the file before
>the copy and
>just copy max bytes assuming that it'll get a short read at some point.
I just tested this and the Linux client does send a Copy with a
ca_count == INT_MAX.

Truth be told, it worked fine, because my server code neglected to do the
required ca_src_offset + ca_count > file_size --> NFS4ERR_INVAL.
I'm now adding the check, but it will only be enabled by a tunable not
set by default.

I am going to post on nfsv4@ietf.org, to see if we can just "clarify" the RFC
to do what Linux already does.

rick

> (I can make this work for the Linux client, but it will take another "cheat"
>  enabled via a tunable.)
> --> Part of the confusion here comes from the fact that the Linux syscall
>        semantics for copy_file_range() has changed. (I also where the
>        copy_file_range(2) hat for FreeBSD, so I've got some work to do there, too.)
>
> In summary, I think that, since Linux has been shipping this for some time,
> documenting workarounds is a practical approach.
>
> rick
>
> _______________________________________________
> nfsv4 mailing list
> nfsv4@ietf.org
> https://www.ietf.org/mailman/listinfo/nfsv4

Re: NFSv4.2 server replies to Copy with length == 0

[Rick Macklem]

>J. Bruce Fields wrote:
>On Thu, Oct 17, 2019 at 02:16:36AM +0000, Rick Macklem wrote:
>> I have now found two cases where the Linux NFSv4.2 server does not
>> conform to RFC-7862. One is as above and the other is a reply to Seek
>> of NFS4ERR_NXIO when the sa_offset argument == file_size (instead of
>> replying NFS_OK along with sr_eof == true).
>
>Huh.  Looks like that's documented behavior of Linux's seek.  (See the
>ERRORS section of the lseek(2) man page.)  Looks like Solaris also
>returns -ENXIO in this case:
>
>        https://docs.oracle.com/cd/E26502_01/html/E29032/lseek-2.html
>
>And freebsd too:
>
>       https://www.freebsd.org/cgi/man.cgi?query=lseek&sektion=2
>
>I wonder where that spec language came from?
Agreed that it is the POSIX draft behaviour.
And, yes, I don't know why the RFC specifies it differently, but when I discussed
it on nfsv4@ietf.org, the concensus was that the RFC indicated a reply
of:
  NFS_OK, sr_eof == true, sr_offset == file_size
for a request of:
  sa_offset == file_size, sa_what == NFS4_CONTENT_DATA
(In the nfsv4@ietf.org archive dated around 2019-08-12.)

>Our NFS server could translate an -ENXIO return into 0 and sr_eof ==
>true easily enough, assuming -ENXIO is really only ever returned in that
>case.
>
>I haven't tested, but from a quick check of the Linux client code I
>think that would require a matching fix on the client side to translate
>sr_eof == 0 *back* to ENXIO.
>
>I don't know if it's worth it.
Yep, agreed. All the FreeBSD client does is what you stated above, mapping
sr_eof == true -> ENXIO for the NFS4_CONTENT_DATA case.

rick
ps: When I re-read it, the comment I made related to Bruce's "wrong" was
      blunt (or maybe rude). I apologize for the tone. All I had intended to
      say was "although it might not be our preferred semantic, it appears
      to clear and implementable, so I do not think it can be "clarified" to
      be the way the Linux server does it". I actually prefer the way the Linux
      server does it, but it is too late now, imho.

--b.

Re: NFSv4.2 server replies to Copy with length == 0

[J. Bruce Fields]

On Thu, Oct 17, 2019 at 03:55:28PM +0000, Rick Macklem wrote:
> ps: When I re-read it, the comment I made related to Bruce's "wrong"
> was blunt (or maybe rude).

It's OK!  I'm possibly being annoying about this kinda trivial thing
anyway.

> I apologize for the tone. All I had intended to say was "although it
> might not be our preferred semantic, it appears to clear and
> implementable, so I do not think it can be "clarified" to be the way
> the Linux server does it". I actually prefer the way the Linux server
> does it, but it is too late now, imho.

For the COPY case at least, implementations seem to be rare and new.

Is it really that trivial to fix up the mismatch?

Actually, I guess so: if I'm implementing the copy using read and write,
then a 0-length read result from a nonzero read call can only mean end
of file, so there's no need to do a second check of the file length or
anything.

Hm.

--b.