Re: XFS for lots of small files

[Eric Sandeen <sandeen@sandeen.net>]

Martin Steigerwald wrote:
> Am Sonntag 25 Mai 2008 schrieb Eric Sandeen:
>> Martin Steigerwald wrote:
>>> And there is quite some fragmentation on it:
>>>
>>> xfs_db> frag
>>> actual 653519, ideal 587066, fragmentation factor 10.17%
>> No, there's not.
> 
> OK, so there is or better was (see below) *some* fragmentation.
> 
>> You have 653519 extents out of an "ideal" 587066.
>>
>> That is 653519/587066 = 1.113 extents per file.
>>
>> It is not "quite some" fragmentation, it is near perfect (although this
>> is subjective, and also depends on the size of your files... if they
>> are all 8k then 1.113 extents per file might be a bit high; if they
>> average 1G then 1.113 extents on average is pretty darned good.)
> 
> They vary a lot. From KMail ~/Mail directory with hundred of thousands of 
> mails in maildir format to a picture and movie collection from various 
> digicams with 150KB over 2-4 MB to 50-200 MB in size and a music 
> collection and kernel sources and and and... would need to run a tool on 
> them to gather some statistic.
> 
> Anyway, nothing that can't be optimized:

Sure... I'd just argue that it's diminishing returns :)

> shambala> xfs_db -r /dev/sda5
> xfs_db> frag
> actual 683648, ideal 617593, fragmentation factor 9.66%
> xfs_db> quit
> 
> shambala> xfs_fsr /dev/sda5
> /home start inode=0
> 
> shambala> xfs_db -r /dev/sda5
> xfs_db> frag
> actual 620316, ideal 617584, fragmentation factor 0.44%
> xfs_db> quit
> 
> xfs_fsr copied over several gigabytes and the free space of the partition 
> temporarily more than once was 4 GB less than the 20 GB of free space it 
> had before and after invoking xfs_fsr ;)

fsr needs to preallocate space to "defragment into" so this is expected,
temporarily.

> Not that I noticed a difference up to now however.

right, my original reply was meant to imply that fragmentation is not
really a problem for you.  :)

And in the larger picture, I just wanted to point out that the
"fragmentation factor" can be pretty misleading.  It reports as (actual
- ideal) / actual.

Imagine a filesystem full of 8GB dvd iso images, each with 4 2GB
extents.  The fragmentation factor would be reported as (4X - 1X) / 4X =
75%.  Which looks "bad," but really isn't.

-Eric