Re: [PATCH RFC 0/2] SUNRPC: dispatch ready transports round-robin across clients

[Benjamin Coddington <ben.coddington@hammerspace.com>]

On 5 Oct 2026, at 8:18, Daire Byrne wrote:
> We use nconnect, set tcp_slot_table_entries=256 and use
> svc_rpc_per_connection_limit=4 in an attempt to allow for more
> requests in flight (NFSv3) and limit greedy clients.

Hey Daire - that limit is per connection, so a client's nconnect multiplies
it.  I suspect what you really want is the limit per client.  That isn't in
these patches, but with the svc_client added here there's an object to hang
it on.

> Now I understand that these patches don't affect the re-export
> server's connection to the upstream servers, but I am interested to
> see what effect they have on the re-export server's clients. If we can
> give a more equal share of requests to the clients does that in turn
> reduce the re-export server bottleneck? Do those greedy readers reduce
> their rops/s such that the re-export server doesn't flood the
> connection to the backend server with bulky read requests?

I don't think it will, or not by much.  All this changes is which client
gets the next free nfsd thread.  It never leaves a thread idle to hold a
client back, so if the readers are the only ones with requests queued they
still get every thread.  And a READ that's waiting on the source server
keeps its thread the whole time.  The readers only give something up when
another client has a request waiting for a thread.

So it depends where your GETATTRs and LOOKUPs are stuck.  If they're
waiting for an nfsd thread on the re-export server, this should help.  If
they already have a thread and are waiting in the NFS client behind the
READs going to the source server, it won't, and that sounds more like what
you're describing.

You can tell which if you can catch it happening.  On the re-export server
the sunrpc:svc_xprt_dequeue tracepoint prints qtime-us, which is how long
the connection waited for a thread.  And mountstats on the mount of the
source server splits GETATTR and LOOKUP into backlog wait and RTT.  If
qtime is small while the clients are suffering then these patches aren't
going to help.

> I was going to test the patches but I got as far as they don't apply
> cleanly to v7.2 and then got caught up in other things.

Sorry, the posted ones are on Chuck's nfsd-testing.  Here they are on v7.2:

  https://github.com/bcodding/linux.git nfsd-clientq-7.2

It's three commits on top of v7.2: the svc_clean_up_xprts() wake fix and
the two from this RFC.  That's the tree the numbers in the cover letter
came from.

Ben