Re: ext4 unlink performance

[Theodore Tso <tytso@mit.edu>]

On Thu, Nov 13, 2008 at 10:11:21PM -0600, Bruce Guenter wrote:
> 
> I had been running these benchmarks over dm-crypt (which is the target
> environment for which I was testing).  So I re-ran the tests both bare
> and with dm-crypt to compare.  The stalls reported by vmstat did not
> show up when running the test bare.
> 
> I also sat and watched the drive LED while the unlink test was running.
> The drive LED showed the occasional 1/2 second stall, but at the same
> time vmstat was showing 5-20 second stalls, so there that would seem to
> point to some kind of reporting problem.

Yeah, I'm guessing that's a red herring caused by how dm-crypt works.
The blocks had already been posted to block device, but it was taking
a while for the blocks to be written out to disk.

This is beginning to perhaps sound like a layout problem of some kind.
How big is the filesystem that you are testing against?  I'm guessing
that if you have 725,000 small files, that it can't be much more than
a gig or two, right?

Can you send me the output of 

    e2image -r /dev/XXX - | bzip2 ext4.e2i.bz2

for both the ext4 and ext3 filesystems, after you have loaded them
with your small files, and before you delete them?  This will show me
the where all of the files are located, and in fact I'll be able to
replicate the delete workload on my end and see exactly what's going on.

I will see the directory and file names of your workload, but
hopefully that won't be an issue for you.  Thanks,

							- Ted