Re: observed significant performance improvement using "delaylog" in a real-world application

Dave Chinner
2010-08-11T23:44:09Z

On Wed, Aug 11, 2010 at 07:01:31PM +0200, Peter Niemayer wrote:
> On 08/11/2010 02:28 PM, Michael Monnerie wrote:
> >Thank you. Are those files located within one dir or do you use a hash
> >structure like squid cache does?
>
> There's only a shallow hierarchy (for functional, not for distribution
> reasons), so the relevant directories have thousands of files in them.
>
> I think after the "ext2"-age no serious file system ever had
> a real problem dealing with lots of files in one directory -
> or do you have contradicting information?

Define "lots of files". :)

>From my numbers, ext3/4 still fall way behind XFS and btrfs when it
comes to handling directories with tens of thousands of entries or
larger. Especially on cold-cache random lookups.

XFS also has quite sophisticated internal directory readahead, so
under the cold cache directory performance of XFS is far better than
ext3/4 can acheive, even for relatively small directories. IIRC this
difference in directory lookup performance was one of the prime
reasons kernel.org switched from ext3 to XFS a couple of years
back...

Cheers,

Dave.
--
Dave Chinner
david@fromorbit.com