Re: Why must NFS access metadata in synchronous mode?

["Xin Zhao" <uszhaoxin@gmail.com>]

Question 1: ...and how many NFS implementations have you seen based on
that paper?
I don't know. I only read the NFS implementations distributed with
Linux kernel. But some paper mentioned that the soft update mechanism
suggested in that paper has been adopted by FreeBSD.

Question 2: NFS permissions are checked by the _server_, not the client.
That's true. But I was not saying that all metadata access must be
asynchronous. Even for permission checking, speculative execution
mechanism proposed in Ed Nightingale's "speculative execution ...."
paper published in SOSP 2005 can be used to avoid waiting. The basic
idea is that a NFS client speculatively assume permission checking
returns "OK" and set a checkpoint, then the client can go ahead to
send further requests. If the actual result turns out to be "OK", the
client can discard the checkpoint, otherwise, it rolls back to the
checking point. This can make waiting time overlap with the sending
time of subsequent requests.

Question 3: Cache consistency requirements are _much_ more stringent
for asynchronous operation.
I agree. But I am not sure how local file system like Ext3 handle this
problem. I don't think Ext3 must synchronously write metadata (I will
double check the ext3 code). If I remember correctly, when change
metadata, Ext3 just change it in memory and mark this page to be
dirty. The page will be flushed to disk afterward. If the server
exports an Ext3 code, it should be able to do the same thing. When a
client requests to change metadata, server writes to the mmaped
metadata page and then return to client instead of having to sync the
change to disk. With this mechanism, at least the client does not have
to wait for the disk flush time. Does it make sense? To prevent
interleave change on metadata before it is flushed to disk, the server
can even mark the metadata page to be read-only before it is flushed
to disk.

Xin
On 6/1/06, Trond Myklebust <trond.myklebust@fys.uio.no> wrote:
> On Thu, 2006-06-01 at 00:04 -0400, Xin Zhao wrote:
> > Until kernel 2.6.16, I think NFS still access metadata synchronously,
> > which may impact performance significantly. Several years ago, paper
> > "metadata update performance in file systems" already suggested using
> > asynchronous mode in metadata access.
>
> ...and how many NFS implementations have you seen based on that paper?
>
> > I am curious why NFS does not adopt this suggestion? Can someone explain this?
>
> a) NFS permissions are checked by the _server_, not the client.
>
> b) Cache consistency requirements are _much_ more stringent for
> asynchronous operation. Think for instance about an asynchronous
> mkdir(): how should the client guarantee exclusive semantics (i.e. that
> mkdir either creates a new directory or returns an EEXIST error)? how
> should it guarantee that the server will have enough disk space to
> satisfy your request? how should it guarantee that nobody will change
> the permissions on the parent directory before the metadata was synced
> to disk?,...
>
> People are considering how to implement this sort of thing using the
> NFSv4 concept of delegations and applying them to directories. It is not
> yet obvious how all the details will be solved.
>
> Trond
>
>