filesystem benchmarking fun

[Chris Mason <chris.mason@oracle.com>]

Hello everyone,

I've been spending some time lately on filesystem benchmarking, in part
because my pet FS project is getting more stable and closer to release.
Now seems like a good time to step back and try to find out what
workloads we think are most important and see how well Linux is doing on
them.  So, I'll start with my favorite three benchmarks and why I think
they matter.  Over time I hope to collect a bunch of results for all of
us to argue about.

* fio: http://brick.kernel.dk/snaps/
Fio can abuse a file via just about every api in the kernel.  aio, dio,
syslets, splice etc.  It can thread, fork, record and playback traces
and provides good numbers for throughput and latencies on various
sequential and random io loads.

* fs_mark: http://developer.osdl.org/dev/doubt/fs_mark/index.html
This one covers most of the 'use the FS as a database' type workloads,
and can vary the number of files, directory depth etc.  It has detailed
timings for reads, writes, unlinks and fsyncs that make it good for
simulating mail servers and other setups.

* compilebench: http://oss.oracle.com/~mason/compilebench/
Tries to benchmark the filesystem allocator by aging the FS through
simulated kernel compiles, patch runs and other operations.

It's easy to get caught up in one benchmark or another and try to use
them for bragging rights.  But, what I want to do is talk about the
workloads we're trying to optimize for and our current methods for
measuring success.  If we don't have good benchmarks for a given
workload, I'd like to try and collect ideas on how to make one.

For example, I'll pick on xfs for a minute.  compilebench shows the
default FS you get from mkfs.xfs is pretty slow for untarring a bunch of
kernel trees.  Dave Chinner gave me some mount options that make it
dramatically better, but it still writes at 10MB/s on a sata drive that
can do 80MB/s.  Ext3 is better, but still only 20MB/s. 

Both are presumably picking a reasonable file and directory layout.
Still, our writeback algorithms are clearly not optimized for this kind
of workload.  Should we fix it?

-chris