Re: EACCES race when opening just-created file

[Jeff Layton <jlayton@redhat.com>]

On Sat, 2016-07-23 at 16:12 -0400, Noah Misch wrote:
> With the Linux kernel as both NFS client and server, I'm seeing a race
> condition where open() can return EACCES for a just-created file.  Is this
> expected behavior?  POSIX implicitly allows it, but I failed to locate any
> discussion of it.  I have attached a test program; here, it witnesses the race
> for roughly a dozen of the 4096 files it creates (different files each time).
> One router separates client from server, with overall ping times as follows:
> 
> 100 packets transmitted, 100 received, 0% packet loss, time 99039ms
> rtt min/avg/max/mdev = 0.173/0.259/4.447/0.422 ms
> 
> This server has three similar clients, each of which witnesses the race.  I
> did not reproduce this in a different NFS setup where client and server were
> virtual machines on the same box.  Those are the only two environments I have
> tested so far.
> 
> -- Server Details
> > [nm@gcc45 0:0 00:36:42 ~]$ uname -a
> Linux gcc45 3.16.0-4-686-pae #1 SMP Debian 3.16.7-ckt11-1+deb8u3 (2015-08-04) i686 GNU/Linux
> > [nm@gcc45 0:0 00:36:37 ~]$ cat /proc/fs/nfs/exports
> # Version 1.1
> # Path Client(Flags) # IPs
> /home   gcc24.tetaneutral.net(rw,no_root_squash,async,wdelay,no_subtree_check,uuid=6797c43c:b0f64b64:bfca4840:f15789eb,sec=1)
> /home   gcc23.tetaneutral.net(rw,no_root_squash,async,wdelay,no_subtree_check,uuid=6797c43c:b0f64b64:bfca4840:f15789eb,sec=1)
> /home   gcc22.tetaneutral.net(rw,no_root_squash,async,wdelay,no_subtree_check,uuid=6797c43c:b0f64b64:bfca4840:f15789eb,sec=1)
> 
> -- Client Details
> > [nm@erpro8-fsf3 0:1 00:37:28 nfs]$ uname -a
> Linux erpro8-fsf3 4.1.4 #1 SMP PREEMPT Mon Aug 3 14:22:54 PDT 2015 mips64 GNU/Linux
> > [nm@erpro8-fsf3 0:1 00:37:49 nfs]$ mount|grep nfs
> gcc45.tetaneutral.net:/home on /home type nfs4 (rw,relatime,vers=4.0,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp6,port=0,timeo=600,retrans=2,sec=sys,clientaddr=2a03:7220:8083:9d00::1,local_lock=none,addr=2a03:7220:8081:8100::1)
> 
> Thanks,
> nm

This is due to a limitation of NFSv3 and NFSv4.0. When you do an
exclusive create, the atime and mtime get set to a particular set of
values (the verifier), and the client is expected to override those
values with a follow-on SETATTR call once it gets those values back.
This is to prevent problems during certain server reboot scenarios. 

The NFS server also sets the mode to 0000 during this time (since the
client can't even set the mode during the operation).

NFSv4.1 is less subject to this problem. You may want to try using that
and see if it's any better.
 
-- 
Jeff Layton <jlayton@redhat.com>