Re: ext2 directory size bug (?)

[kaih@khms.westfalen.de (Kai Henningsen)]

viro@math.psu.edu (Alexander Viro)  wrote on 02.12.00 in <Pine.GSO.4.21.0012020034220.27040-100000@weyl.math.psu.edu>:

> On Sat, 2 Dec 2000, Chris Wedgwood wrote:
>
> > On Sat, Dec 02, 2000 at 12:14:34AM -0500, Alexander Viro wrote:
> >
> >     Not really. Anything that modifies directories holds both ->i_sem and
> >     ->i_zombie, lookups hold ->i_sem and emptiness checks (i.e. victim in
> >     rmdir and overwriting rename) hold ->i_zombie, readdir holds both.
> >
> > what performance issues does this raise in the cast of a directory
> > with _many_ files in it -- when we are renaming often involving that
> > directory?
> >
> > I ask this because certain MTAs do just that; and when you have
> > 10,000 to 100,000 messages queued I immagine you might spend much of
> > your time waiting for ->i_sem locks?
>
> And where do you get contending processes? 'Cause it takes at least two
> to get that...

More than one queue worker running, for example. On systems with that much  
mail, that's just about essential to have.

But I suspect scanning the directory is much worse than renaming. Scanning  
long ext2 (or traditional Unix, for that matter) directories gets *really*  
ugly. That's why Exim, for example, has the "split spool directory" code  
(works very similar to the traditional terminfo split).

> When you have that size of message queues your best bet is to split them
> into many directories, though - all FFS derivatives do linear searches, so
> locking or not, you are going to lose.

Exactly.

> > ext2 directories seem somewhat susepctable to corruption on badly
> > timed shutdowns anyhow; and I don't think there is any way to do
> > atomic writes to them with most disk hardware is there?

I don't think I've seen that. Possibly if you're doing massive directory  
creation just at that moment (unpacking a kernel source tarball, say), but  
in that case I'd call it expected on a non-journalling fs.

If anything, I've seen chopped-up regular files (usually stuff like spool  
files the MTA was just messing around with).

MfG Kai
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/