filesystem tuning hints?

filesystem tuning hints?

[joel]

Hello,

Please redirect me to an appropriate list if this is the wrong place -

This is perhaps a naive question, but please bear with me:

I recently had a chance to do some quick and dirty filesystem performance
comparisons on a server here before putting it into production. I tested
all the journaling filesystems available on stock suse linux enterprise
server v9, using bonnie, tiobench, iozone, and dbench, which all showed
similar trends - xfs tended to have steady performance and latency, jfs
had low performance but low cpu usage, reiserfs got the best numbers in
general, and ext3 results were all over the map. The dbench results are
fairly indicative of the results as a whole.

BTW - the mount options were basically "-noatime" on all filesystems.

I also tested ext2 just out of curiosity, and it thrashed all the others
by a large margin. Could I be doing something really really dumb here,
or is this just the cost of journalling?

Are there any dynamic kernel parameters which could bring any of the
journalled filesystems performance to a more respectable level?


Here are the dbench 3.04 results (MB/sec) plotted as nprocs vs fs type

n         ext2        ext3         jfs         reiser        xfs
-------------------------------------------------------------------
1        239.45      180.94       35.30       209.02       154.44
2        438.83      287.87       36.34       324.25       157.31
4        807.57      389.64       35.81       475.24       154.95
8       1018.24      398.31       30.66       396.14       146.62
16      1003.61      354.79       27.10       403.79       139.17
32      1006.60      180.83       25.40       330.46       120.81
64      1007.61      117.39       24.88       107.89        79.18
128     1010.10       67.70       18.60        43.62         6.41
256     1005.33       26.55        4.10        34.98         7.27
512      973.30       18.00        2.97        29.61         5.34
1024     613.40       17.64        4.36        27.16         4.79
2048      84.05       13.53       16.37        23.29         3.84


Thanks & Regards,

Joel

Re: filesystem tuning hints?

[Theodore Tso]

In order to answer your question about tuning a filesystem, it's a
requirement to know something about the workload, and you haven't
provided that.  In general if you haven't been using the latest
e2fsprogs creating ext3 with a larger journal size will help ---
unless you don't have the requisite memory to support having extra
pinned buffers, since your application may end up getting pushed out
to swap.  Many journalling filesystems can get much better performance
by putting the journal on a separate external device, especially if
that device is a battery-backed non-volitile ramdisk.  With ext3,
depending on your workload (if it is fsync-intensive, for example),
using data journalling with can help a lot, especially with an
external journal device.

Finally, be warned that many filesystem benchmarks may be quite
misleading, especially dbench, which doesn't match very many real-life
workloads at all.  It's designed to be half (the disk part) of an
artificial web-based benchmark that was in itself a pretty bad
benchmark that was easily subject to gaming and not very reflective of
real-world workloads.  Its main virtue is that it is easy to run and
can be useful to developers as a good stress/smoke test to detect that
new kernels haven't done something stupid, as long as you ignore its
numbers.  :-)

In many cases, using your actual workload will be the best benchmark.

Regards, 

						- Ted

Re: filesystem tuning hints?

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1017 bytes --]

On Tue, 18 Jul 2006 20:56:12 PDT, joel said:

> I also tested ext2 just out of curiosity, and it thrashed all the others
> by a large margin. Could I be doing something really really dumb here,
> or is this just the cost of journalling?

Journalling costs performance at the price of added I/O.

> Are there any dynamic kernel parameters which could bring any of the
> journalled filesystems performance to a more respectable level?

Dynamic parameters?  Probably not.  If you *really* care about performance,
you put the journal on a different physical drive (and maybe controller) than
the actual filesystem itself.  From 'man mkfs.ext3':

       -J journal-options
	...
	                   device=external-journal
                          Attach  the  filesystem  to the journal block device
                          located on external-journal.  The  external  journal
                          must already have been created using the command

                          mke2fs -O journal_dev external-journal


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]