Re: parallel file create rates (+high latency)

["J. Bruce Fields" <bfields@fieldses.org>]

On Mon, Apr 25, 2022 at 02:00:32PM +0100, Daire Byrne wrote:
> On Mon, 21 Feb 2022 at 13:59, Daire Byrne <daire@dneg.com> wrote:
> >
> > On Fri, 18 Feb 2022 at 07:46, NeilBrown <neilb@suse.de> wrote:
> > > I've ported it to mainline without much trouble.  I started some simple
> > > testing (parallel create/delete of the same file) and hit a bug quite
> > > easily.  I fixed that (eventually) and then tried with more than 1 CPU,
> > > and hit another bug.  But then it was quitting time.  If I can get rid
> > > of all the easy to find bugs, I'll post it with a CC to you, and you can
> > > find some more for me!
> >
> > That would be awesome! I have a real world production case for this
> > and it's a pretty heavy workload. If that doesn't shake out any bugs,
> > nothing will.
> >
> > The only caveat being that it will likely be restricted to NFSv3
> > testing due to the concurrency limitations with NFSv4.1+ (from the
> > other thread).
> >
> > Daire
> 
> Just to follow up on this again - I have been using Neil's patch for
> parallel file creates (thanks!) but I'm a bit confused as to why it
> doesn't seem to help in my NFS re-export case.
> 
> With the patch, I can achieve much higher parallel (multi process)
> creates directly on my re-export server to a high latency remote
> server mount, but when I re-export that to multiple clients, the
> aggregate create rate again degrades to that which we might expect
> either without the patch or if there was only one process creating the
> files in sequence.
> 
> My assumption was that the nfsd threads of the re-export server would
> act as multiple independent processes and it's clients would be spread
> across them such that they would also benefit from the parallel
> creates patch on the re-export server. So I expected many clients
> creating files in the same directory would achieve much higher
> aggregate performance.

That's the idea.

I've lost track, where's the latest version of Neil's patch?

--b.

> 
> Am I missing some other interaction here that limits parallel
> performance in my unusual re-export case?
> 
> Cheers,
> 
> Daire