* Warning: AG size is a multiple of stripe width?
@ 2011-06-17 11:36 Gim Leong Chin
2011-06-17 15:07 ` Eric Sandeen
2011-06-20 1:21 ` Lachlan McIlroy
0 siblings, 2 replies; 3+ messages in thread
From: Gim Leong Chin @ 2011-06-17 11:36 UTC (permalink / raw)
To: xfs
Hi,
I have a Sun workstation with eight Cheetah 15K.5 SAS 300 GB on RAID 1E (RAID 10) on LSI SAS3081E-R.
I am installing SLED 11 SP1 on it and I thought I will do a thorough optimization right down to the partition boundaries.
Since the default for XFS is to create four aggregation groups, and with the reasoning that Cheetah can do double the seeks of normal 7200 RPM drives, I have four aggregation groups per drive for a total of 16 for 70 GB /dev/sda2 partition, and eight per drive for total of 32 for /dev/sda3 partition (1011 GB).
I have aligned the partition start and end with the stripe width boundaries.
The stripe size is 64 kB, stripe width is 4*64 kB = 256 kB, in terms of 512 byte sectors:
70 GB /
No Start End Number
1 512 67109375 32 GB = 67108864 sectors = 131072 stripe sets
2 67109376 213910015 70 GB = 146800640 sectors = 286720 stripe sets
3 213910016 2335932415 Left = 2335932416 - 213910016 = 2122022400 sectors = 4144575 stripe sets
When I do the following:
mkfs.xfs -f -b size=4k -d agcount=16,su=64k,sw=4 -i size=256,align=1,attr=2 -l version=2,su=64k,lazy-count=1 -n version=2 -s size=512 -L / /dev/sda2
Warning: AG size is a multiple of stripe width. This can cause performance problems by aligning all AGs on the same disk. To avoid this, run mkfs with an AG size that is one stripe unit smaller, for example 1146864
agcount=16 agsize=1146880 blks
bsize=4096
sunit=16 swidth=64 blks
mkfs.xfs -f -b size=4k -d agcount=32,su=64k,sw=4 -i size=256,align=1,attr=2 -l version=2,su=64k,lazy-count=1 -n version=2 -s size=512 -L /home /dev/sda3
Warning: AG size is a multiple of stripe width. This can cause performance problems by aligning all AGs on the same disk. To avoid this, run mkfs with an AG size that is one stripe unit smaller, for example 8289136
agcount=32 agsize=8289152 blks
bsize=4096
sunit=16 swidth=64 blks
I am really puzzled since I thought all I am doing is distributing 4 aggregation groups per drive for sda2 and 8 per drive for sda3.
What have I done wrong and what is the flaw with my understanding?
Thank you!
Chin Gim Leong
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Warning: AG size is a multiple of stripe width?
2011-06-17 11:36 Warning: AG size is a multiple of stripe width? Gim Leong Chin
@ 2011-06-17 15:07 ` Eric Sandeen
2011-06-20 1:21 ` Lachlan McIlroy
1 sibling, 0 replies; 3+ messages in thread
From: Eric Sandeen @ 2011-06-17 15:07 UTC (permalink / raw)
To: Gim Leong Chin; +Cc: xfs
On 6/17/11 6:36 AM, Gim Leong Chin wrote:
> Hi,
>
>
> I have a Sun workstation with eight Cheetah 15K.5 SAS 300 GB on RAID
> 1E (RAID 10) on LSI SAS3081E-R.
>
> I am installing SLED 11 SP1 on it and I thought I will do a thorough
> optimization right down to the partition boundaries.
>
> Since the default for XFS is to create four aggregation groups, and
> with the reasoning that Cheetah can do double the seeks of normal
> 7200 RPM drives, I have four aggregation groups per drive for a total
"Allocation groups" just FWIW :)
Probably no real reason to try to outfox the defaults by doubling
AGs though, at least at this point.
> of 16 for 70 GB /dev/sda2 partition, and eight per drive for total of
> 32 for /dev/sda3 partition (1011 GB).
>
> I have aligned the partition start and end with the stripe width
> boundaries. The stripe size is 64 kB, stripe width is 4*64 kB = 256
> kB, in terms of 512 byte sectors:
>
> 70 GB /
>
> No Start End Number
> 1 512 67109375 32 GB = 67108864 sectors = 131072 stripe sets
>
> 2 67109376 213910015 70 GB = 146800640 sectors = 286720 stripe sets
>
> 3 213910016 2335932415 Left = 2335932416 - 213910016 = 2122022400 sectors = 4144575 stripe sets
>
>
> When I do the following:
>
> mkfs.xfs -f -b size=4k -d agcount=16,su=64k,sw=4 -i size=256,align=1,attr=2 -l version=2,su=64k,lazy-count=1 -n version=2 -s size=512 -L / /dev/sda2
You are restating many defaults here, I'm not sure why...
I would probably just drop the agcount specification and let mkfs do its own thing here; left to its own devices it will choose 4 AGs.
> Warning: AG size is a multiple of stripe width. This can cause
> performance problems by aligning all AGs on the same disk. To avoid
> this, run mkfs with an AG size that is one stripe unit smaller, for
> example 1146864
mkfs.xfs -f -b size=4k -d agsize=1146864b,su=64k,sw=4 ...
does work too, if you really want that many AGs for some reason:
meta-data=testfile isize=256 agcount=16, agsize=1146864 blks
= sectsz=512 attr=2
data = bsize=4096 blocks=18349824, imaxpct=25
= sunit=16 swidth=64 blks
naming =version 2 bsize=4096 ascii-ci=0
log =internal log bsize=4096 blocks=8960, version=2
= sectsz=512 sunit=16 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
-Eric
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: Warning: AG size is a multiple of stripe width?
2011-06-17 11:36 Warning: AG size is a multiple of stripe width? Gim Leong Chin
2011-06-17 15:07 ` Eric Sandeen
@ 2011-06-20 1:21 ` Lachlan McIlroy
1 sibling, 0 replies; 3+ messages in thread
From: Lachlan McIlroy @ 2011-06-20 1:21 UTC (permalink / raw)
To: Gim Leong Chin; +Cc: xfs
----- Original Message -----
> Hi,
>
>
> I have a Sun workstation with eight Cheetah 15K.5 SAS 300 GB on RAID
> 1E (RAID 10) on LSI SAS3081E-R.
>
> I am installing SLED 11 SP1 on it and I thought I will do a thorough
> optimization right down to the partition boundaries.
>
> Since the default for XFS is to create four aggregation groups, and
> with the reasoning that Cheetah can do double the seeks of normal 7200
> RPM drives, I have four aggregation groups per drive for a total of 16
> for 70 GB /dev/sda2 partition, and eight per drive for total of 32 for
> /dev/sda3 partition (1011 GB).
>
> I have aligned the partition start and end with the stripe width
> boundaries.
> The stripe size is 64 kB, stripe width is 4*64 kB = 256 kB, in terms
> of 512 byte sectors:
>
> 70 GB /
>
> No Start End Number
> 1 512 67109375 32 GB = 67108864 sectors = 131072 stripe sets
>
> 2 67109376 213910015 70 GB = 146800640 sectors = 286720 stripe sets
>
> 3 213910016 2335932415 Left = 2335932416 - 213910016 = 2122022400
> sectors = 4144575 stripe sets
>
>
> When I do the following:
>
> mkfs.xfs -f -b size=4k -d agcount=16,su=64k,sw=4 -i
> size=256,align=1,attr=2 -l version=2,su=64k,lazy-count=1 -n version=2
> -s size=512 -L / /dev/sda2
>
> Warning: AG size is a multiple of stripe width. This can cause
> performance problems by aligning all AGs on the same disk. To avoid
> this, run mkfs with an AG size that is one stripe unit smaller, for
> example 1146864
>
> agcount=16 agsize=1146880 blks
> bsize=4096
> sunit=16 swidth=64 blks
>
> mkfs.xfs -f -b size=4k -d agcount=32,su=64k,sw=4 -i
> size=256,align=1,attr=2 -l version=2,su=64k,lazy-count=1 -n version=2
> -s size=512 -L /home /dev/sda3
>
> Warning: AG size is a multiple of stripe width. This can cause
> performance problems by aligning all AGs on the same disk. To avoid
> this, run mkfs with an AG size that is one stripe unit smaller, for
> example 8289136
Each allocation group has some headers that are updated frequently and if
the allocation group size is a multiple of the AG size then all of the AG
headers will land on the same disk. This may cause congestion on this
disk and reduce the performance of the rest of the stripe.
>
> agcount=32 agsize=8289152 blks
> bsize=4096
> sunit=16 swidth=64 blks
>
> I am really puzzled since I thought all I am doing is distributing 4
> aggregation groups per drive for sda2 and 8 per drive for sda3.
>
> What have I done wrong and what is the flaw with my understanding?
If you had concatenated the disks then your logic would make sense but
since you've striped the disks then all of the allocations groups will be
represented in part on each of the disks. So if you have 32 AGs then each
disk will need to accommodate (part of) each of the 32 AGs instead of
the recommended 4 AGs which will probably make the disks seek a lot more.
>
> Thank you!
>
>
> Chin Gim Leong
>
>
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2011-06-20 1:21 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-06-17 11:36 Warning: AG size is a multiple of stripe width? Gim Leong Chin
2011-06-17 15:07 ` Eric Sandeen
2011-06-20 1:21 ` Lachlan McIlroy
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox