From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sonny Rao Subject: Re: Advice sought on how to lock multiple pages in ->prepare_write and ->writepage Date: Tue, 1 Feb 2005 11:49:48 -0500 Message-ID: <20050201164948.GA3084@kevlar.burdell.org> References: <20050201001053.GB11044@kevlar.burdell.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Anton Altaparmakov , Andrew Morton , linux-fsdevel@vger.kernel.org, linux-fsdevel-owner@vger.kernel.org, linux-xfs@oss.sgi.com, viro@parcelfarce.linux.theplanet.co.uk To: Bryan Henderson Content-Disposition: inline In-Reply-To: Sender: linux-xfs-bounce@oss.sgi.com Errors-to: linux-xfs-bounce@oss.sgi.com List-Id: linux-fsdevel.vger.kernel.org On Mon, Jan 31, 2005 at 05:32:31PM -0800, Bryan Henderson wrote: > Thanks for the numbers, though there are enough variables here that it's > hard to make any hard conclusions. I agree, I just throught the data was interesting. > When I've seen these comparisons in the past, it turned out to be one of > two things: > > 1) The system with the smaller I/Os (I/O = unit seen by the device) had > more CPU time per megabyte in the code path to start I/O, so that it > started less I/O. The small I/Os are a consequence of the lower > throughput, not a cause. You can often rule this out just by looking at > CPU utilization. Actually in this case, I was running a single threaded test on an 8-way machine. So, the flushing should have been able to happen on a different (mostly dedicated) CPU from the application that was writing. In this case, the differences in filesystems was probably the cause. What might be a better test is to take JFS, and "hobble" it so that it won't submit multi-page BIOs (i.e. kill it's writepages aop) > 2) The system with the smaller I/Os had a window tuning problem in which > it was waiting for previous I/O to complete before starting more, with > queues not full, and thus starting less I/O. Some devices, with good > intentions, suck the Linux queue dry, one tiny I/O at a time, and then > perform miserably processing those tiny I/Os. Properly tuned, the device > would buffer fewer I/Os and thus let the queues build inside Linux and > thus cause Linux to send larger I/Os. > > People have done ugly queue plugging algorithms to try to defeat this > queue sucking by withholding I/O from a device willing to take it. Others > defeat it by withholding I/O from a willing Linux block layer, instead > saving up I/O and submitting it in large bios. Sure, it seems to work well in practice. We've played a little bit with adjusting the plugging/unplugging thresholds, and in general, playing with them either made things worse or simply had no effect. Sonny -- IBM LTC Kernel Perf.