Re: [PATCH] improve the performance of large sequential write NFS workloads

[Steve Rago <sar@nec-labs.com>]

On Wed, 2009-12-23 at 22:49 +0100, Trond Myklebust wrote:

> > When to send the commit is a complex question to answer.  If you delay
> > it long enough, the server's flusher threads will have already done most
> > of the work for you, so commits can be cheap, but you don't have access
> > to the necessary information to figure this out.  You can't delay it too
> > long, though, because the unstable pages on the client will grow too
> > large, creating memory pressure.  I have a second patch, which I haven't
> > posted yet, that adds feedback piggy-backed on the NFS write response,
> > which allows the NFS client to free pages proactively.  This greatly
> > reduces the need to send commit messages, but it extends the protocol
> > (in a backward-compatible manner), so it could be hard to convince
> > people to accept.
> 
> There are only 2 cases when the client should send a COMMIT: 
>      1. When it hits a synchronisation point (i.e. when the user calls
>         f/sync(), or close(), or when the user sets/clears a file
>         lock). 
>      2. When memory pressure causes the VM to wants to free up those
>         pages that are marked as clean but unstable.
> 
> We should never be sending COMMIT in any other situation, since that
> would imply that the client somehow has better information on how to
> manage dirty pages on the server than the server's own VM.
> 
> Cheers
>   Trond

#2 is the difficult one.  If you wait for memory pressure, you could
have waited too long, because depending on the latency of the commit,
you could run into low-memory situations.  Then mayhem ensues, the
oom-killer gets cranky (if you haven't disabled it), and stuff starts
failing and/or hanging.  So you need to be careful about setting the
threshold for generating a commit so that the client doesn't run out of
memory before the server can respond.

Steve