Re: [PATCH] improve the performance of large sequential write NFS workloads

Steve Rago
2009-12-23T23:13:44Z

On Wed, 2009-12-23 at 22:49 +0100, Trond Myklebust wrote:

> > When to send the commit is a complex question to answer. If you delay
> > it long enough, the server's flusher threads will have already done most
> > of the work for you, so commits can be cheap, but you don't have access
> > to the necessary information to figure this out. You can't delay it too
> > long, though, because the unstable pages on the client will grow too
> > large, creating memory pressure. I have a second patch, which I haven't
> > posted yet, that adds feedback piggy-backed on the NFS write response,
> > which allows the NFS client to free pages proactively. This greatly
> > reduces the need to send commit messages, but it extends the protocol
> > (in a backward-compatible manner), so it could be hard to convince
> > people to accept.
>
> There are only 2 cases when the client should send a COMMIT:
> 1. When it hits a synchronisation point (i.e. when the user calls
> f/sync(), or close(), or when the user sets/clears a file
> lock).
> 2. When memory pressure causes the VM to wants to free up those
> pages that are marked as clean but unstable.
>
> We should never be sending COMMIT in any other situation, since that
> would imply that the client somehow has better information on how to
> manage dirty pages on the server than the server's own VM.
>
> Cheers
> Trond

#2 is the difficult one. If you wait for memory pressure, you could
have waited too long, because depending on the latency of the commit,
you could run into low-memory situations. Then mayhem ensues, the
oom-killer gets cranky (if you haven't disabled it), and stuff starts
failing and/or hanging. So you need to be careful about setting the
threshold for generating a commit so that the client doesn't run out of
memory before the server can respond.

Steve