Re: [PATCH RFC] mm: Implement balance_dirty_pages() through waiting for flusher thread

[Dave Chinner <david@fromorbit.com>]

On Tue, Jun 22, 2010 at 04:38:56PM +0200, Jan Kara wrote:
> On Tue 22-06-10 10:31:24, Christoph Hellwig wrote:
> > On Tue, Jun 22, 2010 at 09:52:34PM +0800, Wu Fengguang wrote:
> > > 2) most writeback will be submitted by one per-bdi-flusher, so no worry
> > >    of cache bouncing (this also means the per CPU counter error is
> > >    normally bounded by the batch size)
> > 
> > What counter are we talking about exactly?  Once balanance_dirty_pages
>   The new per-bdi counter I'd like to introduce.
> 
> > stops submitting I/O the per-bdi flusher thread will in fact be
> > the only thing submitting writeback, unless you count direct invocations
> > of writeback_single_inode.
>   Yes, I agree that the per-bdi flusher thread should be the only thread
> submitting lots of IO (there is direct reclaim or kswapd if we change
> direct reclaim but those should be negligible). So does this mean that
> also I/O completions will be local to the CPU running per-bdi flusher
> thread? Because the counter is incremented from the I/O completion
> callback.

By default we set QUEUE_FLAG_SAME_COMP, which means we hand
completions back to the submitter CPU during blk_complete_request().
Completion processing is then handled by a softirq on the CPU
selected for completion processing.

This was done, IIRC, because it provided some OLTP benchmark 1-2%
better results. It can, however, be turned off via
/sys/block/<foo>/queue/rq_affinity, and there's no guarantee that
the completion processing doesn't get handled off to some other CPU
(e.g. via a workqueue) so we cannot rely on this completion
behaviour to avoid cacheline bouncing.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>