From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sage Weil Subject: Re: NFSv4/pNFS possible POSIX I/O API standards Date: Fri, 1 Dec 2006 08:47:31 -0800 (PST) Message-ID: References: <6.2.3.4.2.20061127213243.04f786c0@cic-mail.lanl.gov> <20061128055428.GA29891@infradead.org> <20061129090450.GA16296@infradead.org> <20061129094815.GE6429@schatzie.adilger.int> <1164795522.7557.45.camel@imp.csi.cam.ac.uk> <20061129082622.GA20285@cynthia.pants.nu> <20061130092548.GA1534@infradead.org> <1164950795.5761.25.camel@lade.trondhjem.org> <1164984094.5761.86.camel@lade.trondhjem.org> Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Christoph Hellwig , Brad Boyer , Anton Altaparmakov , Andreas Dilger , Gary Grider , linux-fsdevel@vger.kernel.org Return-path: Received: from pochacco.sd.dreamhost.com ([66.33.201.150]:15589 "EHLO pochacco.sd.dreamhost.com") by vger.kernel.org with ESMTP id S1030608AbWLAQru (ORCPT ); Fri, 1 Dec 2006 11:47:50 -0500 To: Trond Myklebust In-Reply-To: <1164984094.5761.86.camel@lade.trondhjem.org> Sender: linux-fsdevel-owner@vger.kernel.org List-Id: linux-fsdevel.vger.kernel.org On Fri, 1 Dec 2006, Trond Myklebust wrote: > 'ls --color' and 'find' don't give a toss about most of the arguments > from 'stat()'. They just want to know what kind of filesystem object > they are dealing with. We already provide that information in the > readdir() syscall via the 'd_type' field. Adding all the other stat() > information is just going to add unnecessary synchronisation burdens. 'ls -al' cares about the stat() results, but does not care about the relative timing accuracy wrt the preceeding readdir(). I'm not sure why 'ls --color' still calls stat when it can get that from the readdir() results, but either way it's asking more from the kernel/filesystem than it needs. >> Something like 'ls' certainly doesn't care, but in general applications do >> care that stat() results aren't cached. They expect the stat results to >> reflect the file's state at a point in time _after_ they decide to call >> stat(). For example, for process A to see how much data a just-finished >> process B wrote to a file... > > AFAICS, it will not change any consistency semantics. The main > irritation it will introduce will be that the NFS client will suddenly > have to do things like synchronising readdirplus() and file write() in > order to provide the POSIX guarantees that you mentioned. > > i.e: if someone has written data to one of the files in the directory, > then an NFS client will now have to flush that data out before calling > readdir so that the server returns the correct m/ctime or file size. > Previously, it could delay that until the stat() call. It sounds like you're talking about a single (asynchronous) client in a directory. In that case, the client need only flush if someone calls readdirplus() instead of readdir(), and since readdirplus() is effectively also a stat(), the situation isn't actually any different. The more interesting case is multiple clients in the same directory. In order to provide strong consistency, both stat() and readdir() have to talk to the server (or more complicated leasing mechanisms are needed). In that scenario, readdirplus() is asking for _less_ synchronization/consistency of results than readdir()+stat(), not more. i.e. both the readdir() and stat() would require a server request in order to achieve the standard POSIX semantics, while a readdirplus() would allow a single request. The NFS client already provibes weak consistency of stat() results for clients. Extending the interface doesn't suddenly require the NFS client to provide strong consistency, it just makes life easier for the implementation if it (or some other filesystem) chooses to do so. Consider two use cases. Process A is 'ls -al', who doesn't really care about when the size/mtime are from (i.e. sometime after opendir()). Process B waits for a process on another host to write to a file, and then calls stat() locally to check the result. In order for B to get the correct result, stat() _must_ return a value for size/mtime from _after_ the stat() initiated. That makes 'ls -al' slow, because it probably has to talk to the server to make sure files haven't been modified between the readdir() and stat(). In reality, 'ls -al' doesn't care, but the filesystem has no way to know that without the presense of readdirplus(). Alternatively, an NFS (or other distributed filesystem) client can cache file attributes to make 'ls -al' fast, and simply break process B (as NFS currently does). readdirplus() makes it clear what 'ls -al' doesn't need, allowing the client (if it so chooses) to avoid breaking B in the general case. That simply isn't possible to explicitly communicate with the existing interface. How is that not a win? I imagine that most of the time readdirplus() will hit something in the VFS that simply calls readdir() and stat(). But a smart NFS (or other network filesytem) client can can opt to send a readdirplus over the wire for readdirplus() without sacrificing stat() consistency in the general case. sage