Re: 32TB ext4 fsck times

[Eric Sandeen <sandeen@redhat.com>]

Ric Wheeler wrote:
> Nick Dokos wrote:
>> Now that 64-bit e2fsck can run to completion on a (newly-minted, never
>> mounted) filesystem, here are some numbers. They must be taken with
>> a large grain of salt of course, given the unrealistict situation, but
>> they might be reasonable lower bounds of what one might expect.
>>
>> First, the disks are 300GB SCSI 15K rpm - there are 28 disks per RAID
>> controller and they are striped into 2TiB volumes (that's a limitation
>> of the hardware). 16 of these volumes are striped together using LVM, to
>> make a 32TiB volume.
>>
>> The machine is a four-slot quad core AMD box with 128GB of memory and
>> dual-port FC adapters.
>>   
> Certainly a great configuration for this test....
> 
>> The filesystem was created with default values for everything, except
>> that the resize_inode feature is turned off. I cleared caches before the
>> run.
>>
>> # time e2fsck -n -f /dev/mapper/bigvg-bigvol
>> e2fsck 1.41.4-64bit (17-Apr-2009)
>> Pass 1: Checking inodes, blocks, and sizes
>> Pass 2: Checking directory structure
>> Pass 3: Checking directory connectivity
>> Pass 4: Checking reference counts
>> Pass 5: Checking group summary information
>> /dev/mapper/bigvg-bigvol: 11/2050768896 files (0.0% non-contiguous), 128808243/8203075584 blocks
>>
>> real	23m13.725s
>> user	23m8.172s
>> sys	0m4.323s
>>   
> 
> I am a bit surprised to see it run so slowly on an empty file system. 
> Not an apples to apples comparison, but on my f10 desktop with the older 
> fsck, I can fsck an empty 1TB S-ATA drive in just 23 seconds. An array 
> should get much better streaming bandwidth but be relatively slower for 
> random reads. I wonder if we are much seekier than we should be? Not 
> prefetching as much?

Nick, running this under blktrace would be interesting.  Just tracking
completions is probably sufficient, use the "-a complete" option....

-Eric