Re: Advice sought on how to lock multiple pages in ->prepare_write and ->writepage

[Sonny Rao <sonny@burdell.org>]

On Mon, Jan 31, 2005 at 05:32:31PM -0800, Bryan Henderson wrote:
> Thanks for the numbers, though there are enough variables here that it's 
> hard to make any hard conclusions.

I agree, I just throught the data was interesting.

 
> When I've seen these comparisons in the past, it turned out to be one of 
> two things:
> 
> 1) The system with the smaller I/Os (I/O = unit seen by the device) had 
> more CPU time per megabyte in the code path to start I/O, so that it 
> started less I/O.  The small I/Os are a consequence of the lower 
> throughput, not a cause.  You can often rule this out just by looking at 
> CPU utilization.
 
Actually in this case, I was running a single threaded test on an
8-way machine.  So, the flushing should have been able to happen on a
different (mostly dedicated) CPU from the application that was writing.

In this case, the differences in filesystems was probably the cause.
What might be a better test is to take JFS, and "hobble" it so that it
won't submit multi-page BIOs (i.e. kill it's writepages aop)


> 2) The system with the smaller I/Os had a window tuning problem in which 
> it was waiting for previous I/O to complete before starting more, with 
> queues not full, and thus starting less I/O.  Some devices, with good 
> intentions, suck the Linux queue dry, one tiny I/O at a time, and then 
> perform miserably processing those tiny I/Os.  Properly tuned, the device 
> would buffer fewer I/Os and thus let the queues build inside Linux and 
> thus cause Linux to send larger I/Os.
> 
> People have done ugly queue plugging algorithms to try to defeat this 
> queue sucking by withholding I/O from a device willing to take it.  Others 
> defeat it by withholding I/O from a willing Linux block layer, instead 
> saving up I/O and submitting it in large bios.

Sure, it seems to work well in practice.  We've played a little bit
with adjusting the plugging/unplugging thresholds, and in general,
playing with them either made things worse or simply had no effect.  

Sonny
--
IBM LTC Kernel Perf.