Re: extremely slow file creation/deletion after xfs ran full

[Carsten Aulbert <Carsten.Aulbert@aei.mpg.de>]

[-- Attachment #1: Type: text/plain, Size: 1205 bytes --]

Hi

On 01/12/2015 05:37 PM, Brian Foster wrote:
> No, but it does show that there are a bunch of free inodes scattered
> throughout the existing records in most of the AGs. The finobt should
> definitely help avoid the allocation latency when this occurs.
> 

That is good to know/hope :)

> It is interesting that you have so many more free inodes in ag 0 (~53m
> as opposed to several hundreds/thousands in others). What does 'p count'
> show for each ag? Was this fs grown to the current size over time?
> 
"p count" seems to "thin out" over ag:

count = 513057792
count = 16596224
count = 15387584
count = 14958528
count = 4096960
count = 4340416
count = 4987968
count = 3321792
count = 5041856
count = 5485376
count = 5233088
count = 5810432
count = 5271552
count = 5464000
count = 365440

(if the full print output is interesting, it's attached).

The FS was never grown, the machine was installed on November 5th, 2012
and was (ab)using the FS ever since. On average there have been about
1-2 million file creations per day (ranging from a few kByte files to a
few 100 kBytes) and also an equally large number of deletions (after
some time). Thus overall, a somewhat busy server.

Cheers

Carsten


[-- Attachment #2: ag-print.log --]
[-- Type: text/plain, Size: 4175 bytes --]


AG 0
magicnum = 0x58414749
versionnum = 1
seqno = 0
length = 268435455
count = 513057792
root = 1286
level = 3
freecount = 53795884
newino = 128
dirino = null
unlinked[0-63] = 
uuid = 00000000-0000-0000-0000-000000000000
lsn = 0
crc = 0 (unchecked)
free_root = 0
free_level = 0

AG 1
magicnum = 0x58414749
versionnum = 1
seqno = 1
length = 268435455
count = 16596224
root = 123653
level = 3
freecount = 251
newino = 17696
dirino = null
unlinked[0-63] = 
uuid = 00000000-0000-0000-0000-000000000000
lsn = 0
crc = 0 (unchecked)
free_root = 0
free_level = 0

AG 2
magicnum = 0x58414749
versionnum = 1
seqno = 2
length = 268435455
count = 15387584
root = 644
level = 3
freecount = 45
newino = 5081504
dirino = null
unlinked[0-63] = 
uuid = 00000000-0000-0000-0000-000000000000
lsn = 0
crc = 0 (unchecked)
free_root = 0
free_level = 0

AG 3
magicnum = 0x58414749
versionnum = 1
seqno = 3
length = 268435455
count = 14958528
root = 317207
level = 3
freecount = 381
newino = 5034112
dirino = null
unlinked[0-63] = 
uuid = 00000000-0000-0000-0000-000000000000
lsn = 0
crc = 0 (unchecked)
free_root = 0
free_level = 0

AG 4
magicnum = 0x58414749
versionnum = 1
seqno = 4
length = 268435455
count = 4096960
root = 320520
level = 2
freecount = 11009
newino = 5079872
dirino = null
unlinked[0-63] = 
uuid = 00000000-0000-0000-0000-000000000000
lsn = 0
crc = 0 (unchecked)
free_root = 0
free_level = 0

AG 5
magicnum = 0x58414749
versionnum = 1
seqno = 5
length = 268435455
count = 4340416
root = 743
level = 2
freecount = 6748
newino = 5150208
dirino = null
unlinked[0-63] = 
uuid = 00000000-0000-0000-0000-000000000000
lsn = 0
crc = 0 (unchecked)
free_root = 0
free_level = 0

AG 6
magicnum = 0x58414749
versionnum = 1
seqno = 6
length = 268435455
count = 4987968
root = 155879
level = 2
freecount = 663
newino = 4937472
dirino = null
unlinked[0-63] = 
uuid = 00000000-0000-0000-0000-000000000000
lsn = 0
crc = 0 (unchecked)
free_root = 0
free_level = 0

AG 7
magicnum = 0x58414749
versionnum = 1
seqno = 7
length = 268435455
count = 3321792
root = 543815
level = 2
freecount = 595
newino = 13695232
dirino = null
unlinked[0-63] = 
uuid = 00000000-0000-0000-0000-000000000000
lsn = 0
crc = 0 (unchecked)
free_root = 0
free_level = 0

AG 8
magicnum = 0x58414749
versionnum = 1
seqno = 8
length = 268435455
count = 5041856
root = 507410
level = 2
freecount = 693
newino = 5057856
dirino = null
unlinked[0-63] = 
uuid = 00000000-0000-0000-0000-000000000000
lsn = 0
crc = 0 (unchecked)
free_root = 0
free_level = 0

AG 9
magicnum = 0x58414749
versionnum = 1
seqno = 9
length = 268435455
count = 5485376
root = 208
level = 2
freecount = 9089
newino = 5134912
dirino = null
unlinked[0-63] = 
uuid = 00000000-0000-0000-0000-000000000000
lsn = 0
crc = 0 (unchecked)
free_root = 0
free_level = 0

AG 10
magicnum = 0x58414749
versionnum = 1
seqno = 10
length = 268435455
count = 5233088
root = 821
level = 2
freecount = 37122
newino = 5060416
dirino = null
unlinked[0-63] = 
uuid = 00000000-0000-0000-0000-000000000000
lsn = 0
crc = 0 (unchecked)
free_root = 0
free_level = 0

AG 11
magicnum = 0x58414749
versionnum = 1
seqno = 11
length = 268435455
count = 5810432
root = 349837
level = 2
freecount = 2657
newino = 5102880
dirino = null
unlinked[0-63] = 
uuid = 00000000-0000-0000-0000-000000000000
lsn = 0
crc = 0 (unchecked)
free_root = 0
free_level = 0

AG 12
magicnum = 0x58414749
versionnum = 1
seqno = 12
length = 268435455
count = 5271552
root = 314
level = 2
freecount = 60497
newino = 5050528
dirino = null
unlinked[0-63] = 
uuid = 00000000-0000-0000-0000-000000000000
lsn = 0
crc = 0 (unchecked)
free_root = 0
free_level = 0

AG 13
magicnum = 0x58414749
versionnum = 1
seqno = 13
length = 268435455
count = 5464000
root = 992
level = 2
freecount = 1790275
newino = 5067808
dirino = null
unlinked[0-63] = 
uuid = 00000000-0000-0000-0000-000000000000
lsn = 0
crc = 0 (unchecked)
free_root = 0
free_level = 0

AG 14
magicnum = 0x58414749
versionnum = 1
seqno = 14
length = 123739108
count = 365440
root = 945
level = 2
freecount = 54544
newino = 5084224
dirino = null
unlinked[0-63] = 
uuid = 00000000-0000-0000-0000-000000000000
lsn = 0
crc = 0 (unchecked)
free_root = 0
free_level = 0

[-- Attachment #3: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs