mkfs.xfs questions

mkfs.xfs questions

[Jasmin Buchert]

Hi,

I'm planning to use XFS but have some questions..

Is there any real advantage of making the log size 32-64 MB and
what is the difference between log version 1 and 2 regarding to
efficency/performance?

Is it true that a small agcount is better for most systems
(Gentoo and some other sources recommend this)? It's a desktop machine.

Greetings,
Jasmin Buchert

Re: mkfs.xfs questions

[Christian Kujau]

On Wed, 29 Nov 2006, Jasmin Buchert wrote:
> Is there any real advantage of making the log size 32-64 MB and

>From 'man mkfs.xfs':

    If the  log  is  contained within the data section and size isn't
    specified, mkfs.xfs will try to select a suitable log
    size depending on the size of the filesystem.  The actual
    logsize depends on the filesystem block size and the directory
    block size.

    Otherwise, the size suboption is only needed if the log
    section of the filesystem should occupy less space than the size
    of the special file.

So, if you're not limited by very special space restrictions, you won't 
need the "size" option.

> what is the difference between log version 1 and 2 regarding to
> efficency/performance?

The "version" option should have no effect on performance, from 'man 
mkfs.xfs' again:

      Using the version suboption to specify a version 2 log enables the
      sunit  suboption,  and  allows  the  logbsize  to  be increased
      beyond  32K.

The "sunit" options can be tweaked to provide better performace in raid5 
environments, same for the "agcount" option: for special needs only but 
I'm not aware of any benchmarks for different sunit/agcount values.

> Is it true that a small agcount is better for most systems
> (Gentoo and some other sources recommend this)? It's a desktop machine.

Hm, some people are indeed suggesting this [0], you might ask the author 
of the doc why he's doing this or test this by yourself.

Christian.

[0] http://www.rootforum.de/wiki/howto/gentoo/basesystem#formatieren_der_partitionen
-- 
BOFH excuse #244:

Your cat tried to eat the mouse.

Re: mkfs.xfs questions

[Iustin Pop]

On Fri, Dec 01, 2006 at 04:23:41AM +0000, Christian Kujau wrote:
> On Wed, 29 Nov 2006, Jasmin Buchert wrote:
> >Is there any real advantage of making the log size 32-64 MB and
> 
> From 'man mkfs.xfs':
> 
>    If the  log  is  contained within the data section and size isn't
>    specified, mkfs.xfs will try to select a suitable log
>    size depending on the size of the filesystem.  The actual
>    logsize depends on the filesystem block size and the directory
>    block size.
> 
>    Otherwise, the size suboption is only needed if the log
>    section of the filesystem should occupy less space than the size
>    of the special file.
> 
> So, if you're not limited by very special space restrictions, you won't 
> need the "size" option.

I don't understand how you took that conclusion. The explanations refer
to the default log size. I believe the original poster asked about the
performance advantage of *raising* the log size above the default values
for internal logs, and my impression is that metadata-intensive
workloads benefit from increasing the log size (however no hard numbers
are available).

A while back when mkfs.xfs had more conservative default value, bigger log
sizes indeed helped for big filesystems.

Regards,
Iustin

Re: mkfs.xfs questions

[Christian Kujau]

On Fri, 1 Dec 2006, Iustin Pop wrote:
> I don't understand how you took that conclusion. The explanations refer
> to the default log size. I believe the original poster asked about the
> performance advantage of *raising* the log size above the default values
> for internal logs,

I was under the assumption that the OP asked about altering the size of 
the log at all and the manpage only states a reason for *decreasing* the 
logsize.

> and my impression is that metadata-intensive
> workloads benefit from increasing the log size (however no hard numbers
> are available).

As no numbers are known to me either, I did not see a point in 
increasing the log, hence my statement.

> A while back when mkfs.xfs had more conservative default value, bigger log
> sizes indeed helped for big filesystems.

As I've done a few benchmarks[0] for different filesystems lately I 
might find some time to play around with different fs tweaks...

Christian.

[0] http://nerdbynature.de/wp/?cat=4
-- 
BOFH excuse #353:

Second-system effect.

Re: mkfs.xfs questions

[Iustin Pop]

On Fri, Dec 01, 2006 at 11:59:08PM +0000, Christian Kujau wrote:
> On Fri, 1 Dec 2006, Iustin Pop wrote:
> >I don't understand how you took that conclusion. The explanations refer
> >to the default log size. I believe the original poster asked about the
> >performance advantage of *raising* the log size above the default values
> >for internal logs,
> 
> I was under the assumption that the OP asked about altering the size of 
> the log at all and the manpage only states a reason for *decreasing* the 
> logsize.
Ah, I see. Sorry for the confusion.

> >and my impression is that metadata-intensive
> >workloads benefit from increasing the log size (however no hard numbers
> >are available).
> 
> As no numbers are known to me either, I did not see a point in 
> increasing the log, hence my statement.
Hmm, I am pretty sure that it makes a difference, but only from personal
experience, not from benchmarks. A while ago, mkfs.xfs used to make <8M
logs even for big filesystems[0]. Nowadays it chooses a more sane value.

> >A while back when mkfs.xfs had more conservative default value, bigger log
> >sizes indeed helped for big filesystems.
> 
> As I've done a few benchmarks[0] for different filesystems lately I 
> might find some time to play around with different fs tweaks...

That would be interesting!

[0] http://oss.sgi.com/archives/xfs/2002-04/msg00443.html and the
    corresponding thread, especially
    http://oss.sgi.com/archives/xfs/2002-04/msg00441.html

Iustin

Re: mkfs.xfs questions

[Christian Kujau]

On Sat, 2 Dec 2006, Iustin Pop wrote:
> Hmm, I am pretty sure that it makes a difference, but only from personal
> experience, not from benchmarks. A while ago, mkfs.xfs used to make <8M
> logs even for big filesystems[0]. Nowadays it chooses a more sane value.

I could not stand my own curiosity, so here it is: 
http://nerdbynature.de/wp/?cat=4

I think I'll repeat the benchmarks with bigger test sizes. The 
testscript can easily be adjusted to test more options/values.

Christian.
-- 
BOFH excuse #248:

Too much radiation coming from the soil.

Re: mkfs.xfs questions

[Iustin Pop]

On Tue, Dec 05, 2006 at 05:46:15PM +0000, Christian Kujau wrote:
> On Sat, 2 Dec 2006, Iustin Pop wrote:
> >Hmm, I am pretty sure that it makes a difference, but only from personal
> >experience, not from benchmarks. A while ago, mkfs.xfs used to make <8M
> >logs even for big filesystems[0]. Nowadays it chooses a more sane value.
> 
> I could not stand my own curiosity, so here it is: 
> http://nerdbynature.de/wp/?cat=4
> 
> I think I'll repeat the benchmarks with bigger test sizes. The 
> testscript can easily be adjusted to test more options/values.

Hmm, I see tests with 32M logs and 64M logs. Try running the file
creation/deletion test also with 4M, 8M, 16M, and then I'll think there
will be a difference.

Iustin

Re: mkfs.xfs questions

[David Chinner]

On Tue, Dec 05, 2006 at 05:46:15PM +0000, Christian Kujau wrote:
> On Sat, 2 Dec 2006, Iustin Pop wrote:
> >Hmm, I am pretty sure that it makes a difference, but only from personal
> >experience, not from benchmarks. A while ago, mkfs.xfs used to make <8M
> >logs even for big filesystems[0]. Nowadays it chooses a more sane value.
> 
> I could not stand my own curiosity, so here it is: 
> http://nerdbynature.de/wp/?cat=4

One line summary:

"The results however are a bit boring and I for one have no reason to tweak
these options for a desktop machine."

For that data set size you tested. However you might find a
difference if your tests actually write the data back to disk
because a lot of the tests are running out of cache.

> I think I'll repeat the benchmarks with bigger test sizes. The 
> testscript can easily be adjusted to test more options/values.

I think you need to to have any hope of demonstrating a
difference in performance from the mkfs/mount options.

Typically, you need to be writing/reading files at least 2x the
size of memory and create/delete a fileset of at least 1,000,000
files to really determine differences in performance from
these parameters...

Cheers,

Dave.
-- 
Dave Chinner
Principal Engineer
SGI Australian Software Group

Re: mkfs.xfs questions

[Christian Kujau]

On Wed, 6 Dec 2006, David Chinner wrote:
> For that data set size you tested. However you might find a
> difference if your tests actually write the data back to disk
> because a lot of the tests are running out of cache.

Ha, thanks for reminding me!
Now I remember why all my other tests were done with 2GB of data, having 
1GB of RAM ;)

> Typically, you need to be writing/reading files at least 2x the
> size of memory

I started a test with 4GB of data (bonnie++ -s 4096m) an hour ago...

> and create/delete a fileset of at least 1,000,000
> files to really determine differences in performance from

hm, that would be 'bonnie++ -n 1000' instead of the "-n 100" I'm testing 
right now.

Thanks for the hints, David! If you/someone know any 
better/closer-to-real-usage benchmarks, please let me know. I'm testing 
with bonnie++ because it I don't know any better. is iozone any 
good? is tiobench still worth to try?

Christian.
-- 
BOFH excuse #208:

Your mail is being routed through Germany ... and they're censoring us.

Re: mkfs.xfs questions

[Christian Kujau]

On Tue, 5 Dec 2006, Christian Kujau wrote:
> I started a test with 4GB of data (bonnie++ -s 4096m) an hour ago...

took 16hrs to complete, wow:

http://nerdbynature.de/bench/amd64/2.6.19-git7_xfs.2/
http://nerdbynature.de/wp/?cat=4

C.
-- 
BOFH excuse #293:

You must've hit the wrong any key.

Re: mkfs.xfs questions

[David Chinner]

On Wed, Dec 06, 2006 at 03:57:55PM +0000, Christian Kujau wrote:
> On Tue, 5 Dec 2006, Christian Kujau wrote:
> >I started a test with 4GB of data (bonnie++ -s 4096m) an hour ago...
> 
> took 16hrs to complete, wow:
> 
> http://nerdbynature.de/bench/amd64/2.6.19-git7_xfs.2/
> http://nerdbynature.de/wp/?cat=4

You should try version 2 logs and larger log buffer sizes.

Also - external log I/O is not cached, so throughput to
the log is determined by the hardware. If the external
log is slow, it may be worse than an internal log in
terms of performance. Larger log buffers (v2 logs) should
improve performance for these tests on both internal
and external logs.

Cheers,

Dave.
-- 
Dave Chinner
Principal Engineer
SGI Australian Software Group