RE nfs: opening a file with O_WRONLY|O_CREAT flags can result in permission denied error

[tilan@janestreet.com]

>>>> Hello,
>>>> 
>>>> We recently noticed there is a behavior change w.r.t opening a file 
>>>> with the O_WRONLY|O_CREAT flags over the NFSv3 protocol after
>>>> upgrading
>>>> the kernel from 6.1 LTS to 6.12 LTS. From the packets capturing, it
>>>> seems
>>>> like the kernel would now issue an additional CREATE rpc call to the
>>>> remote NFS server regardless if the target file pre-exists or not.
>>>> The CREATE rpc request could return an EACCES error if the client
>>>> only has 
>>>> the write permission to the pre-existing file but no write permission
>>>> on 
>>>> the directory containing the pre-existing file. This causes the
>>>> openat 
>>>> syscall to fail with permission denied error which is not expected.
>>>> 
>>>> After doing some code tracing, it seems like the new behavior was
>>>> introduced as part of 7c6c5249f061 ("NFS: add atomic_open for NFSv3
>>>> to
>>>> handle O_TRUNC correctly."). We would like to confirm if the current 
>>>> behavior that we are observing with the 6.12 kernel is expected given
>>>> that the new behavior breaks existing user's application code. We 
>>>> currently have a workaround by explicitly remove the O_CREAT flag
>>>> when 
>>>> opening a pre-existing file for write, but would still prefer not
>>>> have
>>>> to apply this workaround when upgrading to the newer kernel.
>>>
>>> What server are you using?
>>>
>> The permission error was reproduced against a vendor appliance, running
>> the same test against NFSD seems to yield a different result
>> (no permission errors) due to differences in behavior from the CREATE 
>> RPC implementation.
>>
>> NFSD behavior
>>
>>   - CREATE RPC (UNCHECKED mode) returns the filehandle of the 
>>     existing file along with the file's attributes
>>	  
>> Vendor appliance
>>
>>   - CREATE RPC (UNCHECKED mode) always attempts to create a new
>>     file and returns the new filehandle. This explains why we
>>     are seeing permission denied error from openat syscall.
>>	  
>> RFC 1813 states that 
>>
>>   "UNCHECKED means that the file should be created without checking
>>   for the existence of a duplicate file in the same directory. In this
>>   case, how.obj_attributes is a sattr3 describing the initial
>>   attributes for the file."
>>	 
>> It seems like the vendor's implementation matches more closely to what
>> the "standard" describes, but the behavior might not be what a normal
>> user would expect. I guess there is no win-win situation here.

> Compare RFC 7530's UNCHECKED4. It says explicitly that if the object
> already exists, a non-exclusive create does not recreate it. The
> existing object is used and the supplied attributes are applied. RFC
> 7530 and subsequent RFCs document what UNCHECKED was always meant to
> do, and NFSD's NFSv3 behavior is consistent with that.

> IMHO the correct interpretation, the one consistent with the rest of
> NFSv3, with NFSv4's clarifying text, and with POSIX, is NFSD's, not
> the vendor's. UNCHECKED governs error-vs-no-error on a duplicate,
> not recreate-vs-reuse.

> But to confirm this, look for an RFC 1813 erratum, or WG/implementor-list
> guidance, stating that UNCHECKED must replace or re-create a pre-existing
> file and return a fresh filehandle. It would also be sensible to check
> how Solaris NFSv3 behaves, as it is a reference implementation for
> RFC 1813.

I ran the same test against the Solaris 11.4, and the NFSv3's behavior
is consistent with the Linux nfsd. I guess this would be sufficient to
conclude that the vendor's implementation is deviated from the actual 
intent and the addition of NFSv3 atomic_open exposes this difference in
an very unfortunate way.


Thanks,
Tian