Re: NFSv4/pNFS possible POSIX I/O API standards

[Anton Altaparmakov <aia21@cam.ac.uk>]

On Wed, 2006-11-29 at 01:48 -0800, Andreas Dilger wrote:
> On Nov 29, 2006  09:04 +0000, Christoph Hellwig wrote:
> >  - readdirplus
> > 
> > 	This one is completely unneeded as a kernel API.  Doing readdir
> > 	plus calls on the wire makes a lot of sense and we already do
> > 	that for NFSv3+.  Doing this at the syscall layer just means
> > 	kernel bloat - syscalls are very cheap.
> 
> The question is how does the filesystem know that the application is
> going to do readdir + stat every file?  It has to do this as a heuristic
> implemented in the filesystem to determine if the ->getattr() calls match
> the ->readdir() order.  If the application knows that it is going to be
> doing this (e.g. ls, GNU rm, find, etc) then why not let the filesystem
> take advantage of this information?  If combined with the statlite
> interface, it can make a huge difference for clustered filesystems.

The other thing is that a readdirplus at least for some file systems can
be implemented much more efficiently than readdir + stat because the
directory entry itself contains a lot of extra information.

To take NTFS as an example I know something about, the directory entry
caches the a/c/m time as well as the data file size (needed for "ls")
and the allocated on disk file size (needed for "du") as well as the
inode number corresponding to the name, the flags of the inode
(read-only, hidden, system, whether it is a file or directory, etc) and
some other tidbits so readdirplus on NTFS can simply return wanted
information without ever having to do a lookup() on the file name to
obtain the inode to then use that in the stat() system call...  The
potential decrease in work needed is tremendous in this case...

Imagine "ls -li" running with a single readdirplus() syscall and that is
all that happens on the kernel side, too.  Not a single file name needs
to be looked up and not a single inode needs to be loaded.  I don't
think anyone can deny that that would be a massive speedup of "ls -li"
for file systems whose directory entries store extra information to
traditional unix file systems...

Best regards,

        Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer, http://www.linux-ntfs.org/