Re: stat benchmark

[Theodore Tso <tytso@mit.edu>]

On Sun, Apr 27, 2008 at 09:43:05PM -0700, Ulrich Drepper wrote:
> On Thu, Apr 24, 2008 at 1:59 PM, Soeren Sandmann <sandmann@daimi.au.dk> wrote:
> >  So I am looking for ways to improve this.
> 
> Aside from what has already been proposed there is also the
> readdirplus() route.  Unfortunately the people behind this and related
> proposals vanished after the last discussions.  I was  hoping they
> come back with a revised proposal but perhaps not.  Maybe it's time to
> pick up the ball myself.
> 
> As a reminder, readdirplus() is an extended readdir() which also
> returns (a subset of) the stat information for the file at the same
> time.  The subset part is needed to account for the different
> information contained in the inodes.  For most applications the subset
> should be sufficient and therefore all that's needed is a single
> iteration over the directory.

I'm not sure this would help in the cold cache case, which is what
Soeren originally complained about.[1] The problem is whaever
information the user might need won't be store in the directory, so
the filesystem would end having to stat the file anyway, incurring a
disk seek, which was what the user was complaining about.  A
readdirplus() would save a whole bunch of system calls if the inode
was already cached, yes, but I'm not sure that's it would be worth the
effort given how small Linux's system call overhead would be.  But in
the cold cache case, you end up seeking all over the disk, and the
only thing you can do is to try to keep the inodes close to each
other, and to have either readdir() or the caller of readdir() sort
all of the returned directory entries by inode number to avoid seeking
all over the disk.

							- Ted


[1] At least, not without making filesystem modifications; the problem
is that if you're going to store the inode information (or part of the
inode information) in the directory it has to get updated each time
the inode gets updated, and that gets painful with hardlinks.  There
was a Usenix paper which explored this idea over ten years ago, and
it's possible --- but then all directory entries need to have linked
list so that that you can either (a) find the "master" directory entry
which contains the inode information and be able to migrate inode
information to another directory entry when the originl "master"
dirent gets deleted, or (b) you keep the subset of the inode
information in all of the directory entries of the hardlink, and have
to update them all each time the inode information changes.