Re: nfsd thread limit and UDP ?

[bfields@fieldses.org (J. Bruce Fields)]

On Wed, Feb 20, 2019 at 11:28:53AM +0000, James Pearson wrote:
> On a very busy NFSv3 server (running CentOS 6), we recently upped the
> nfsd thread count to 1024 - but this caused client mount requests over
> UDP to fail.
> 
> We configure all our clients to use TCP for NFS mounts, but the
> automounter (automountd) on MacOS (up to version MacOS 10.12) seeds a
> 'null call' to the NFS server over UDP before attempting the mount -
> but the server appears to ignore any UDP requests - and the automount
> fails
> 
> I can also reproduce the issue on a Linux client via:
> 
>  mount -o udp,nfsvers=3 server:/export /mount/point
> 
> I've found, by trial and error, that the maximum number of nfsd
> threads that can be run on the server is 1017 before UDP mount
> requests fail

Thanks for investigating, that's very weird and interesting!

Just looking through the UDP code in net/sunrpc/svcsock.c....  I wonder
if it's this:

	svc_sock_setbufsize(svsk->sk_sock,
                                (serv->sv_nrthreads+3) * serv->sv_max_mesg,
                                (serv->sv_nrthreads+3) * serv->sv_max_mesg);

sv_max_mesg will be about 2^20, so the result will be about 2^30 in your case.
Then svc_sock_setbufsize throws in another multiple of 2:

	sock->sk->sk_sndbuf = snd * 2;
        sock->sk->sk_rcvbuf = rcv * 2;

so we've got to be very close to overflowing sk_sndbuf and sk_rcvbuf, which are
ints.

--b.