Re: filesystem benchmarking fun

[Jens Axboe <jens.axboe@oracle.com>]

On Tue, May 22 2007, Matt Mackall wrote:
> On Tue, May 22, 2007 at 11:21:20AM -0700, Andrew Morton wrote:
> > On Tue, 22 May 2007 12:35:11 -0400
> > Chris Mason <chris.mason@oracle.com> wrote:
> > 
> > > On Wed, May 16, 2007 at 01:37:26PM -0700, Andrew Morton wrote:
> > > > On Wed, 16 May 2007 16:14:14 -0400
> > > > Chris Mason <chris.mason@oracle.com> wrote:
> > > > 
> > > > > On Wed, May 16, 2007 at 01:04:13PM -0700, Andrew Morton wrote:
> > > > > > > The good news is that if you let it run long enough, the times
> > > > > > > stabilize.  The bad news is:
> > > > > > > 
> > > > > > > create dir kernel-86 222MB in 15.85 seconds (14.03 MB/s)
> > > > > > > create dir kernel-87 222MB in 28.67 seconds (7.76 MB/s)
> > > > > > > create dir kernel-88 222MB in 18.12 seconds (12.27 MB/s)
> > > > > > > create dir kernel-89 222MB in 19.77 seconds (11.25 MB/s)
> > > > > > 
> > > > > > well hang on.  Doesn't this just mean that the first few runs were writing
> > > > > > into pagecache and the later ones were blocking due to dirty-memory limits?
> > > > > > 
> > > > > > Or do you have a sync in there?
> > > > > > 
> > > > > There's no sync,  but if you watch vmstat you can clearly see the log
> > > > > flushes, even when the overall create times are 11MB/s.  vmstat goes
> > > > > 30MB/s -> 4MB/s or less, then back up to 30MB/s.
> > > > 
> > > > How do you know that it is a log flush rather than, say, pdflush
> > > > hitting the blockdev inode and doing a big seeky write?
> > > 
> > > Ok, I did some more work to split out the two cases (block device inode
> > > writeback and log flushing).
> > > 
> > > I patched jbd's log_do_checkpoint to put all the blocks it wanted to
> > > write in a radix tree, then send them all down in order at the end.
> > 
> > Side note: we already have all of that capability in the kernel:
> > sync_inode(blockdev_inode, wbc) will do an ascending-LBA write of the whole
> > blockdev.
> 
> Why don't we simply plug the queue, do all the writes, and let the I/O
> scheduler sort it out instead?

The data set is too huge for that to work, even if you increased
nr_requests to some huuuge number.

-- 
Jens Axboe