Re: [PATCH 0/7] Per-bdi writeback flusher threads v20

[Wu Fengguang <fengguang.wu@intel.com>]

On Wed, Sep 23, 2009 at 10:08:40PM +0800, Chris Mason wrote:
> On Wed, Sep 23, 2009 at 09:05:41AM +0800, Wu Fengguang wrote:
> 
> [ timeslice based limits on number of pages sent by the bdi threads ]
> 
> > > 
> > > The reason I prefer the timeslice idea is that we don't need the
> > > hardware to tell us how fast it is.  We just write for a while and move
> > > on.
> > 
> > That makes sense.  Note that the triple (pages, page segments,
> > submission time) can somehow adapt to hardware capabilities
> > (and at least won't hurt fast arrays).
> > 
> > - max pages are set to large enough number for big arrays
> > - max page segments could be based on the existing blk_queue_nonrot()
> > - submission time = 1s, which is mainly a safeguard for slow devices
> >   (ie. usb stick), to prevent one single inode from taking too much
> >   time. This time limit has little performance impacts.
> > 
> > Possible merits are
> > - these parameters are concrete ones and easy to handle
> > - it's natural to implement related logics in the VFS level
> > - file systems can do nothing to get most benefits
> > 
> > Also the (now necessary) per-invocation limit could be somehow
> > eliminated when balance_dirty_pages() does not do IO itself.
> 
> I think there are probably a lot of good ways to improve on our single
> max number of pages metric from today

Yes, as always, it benefits to work out some prototype solutions for
evaluation and comparison.

> , but I'm worried about the
> calculation time finding page segments.  The radix tree
> isn't all that well suited to it.

I didn't mean to "calculate" the page segments. But rather to do this
in write_cache_pages:

        if (this page index is 1MB away from prev page index)
                wbc->page_segments--;

> But, if you've got a patch I'd be happy to run a comparison against it.
> Jens' box will be better at showing any CPU cost to the radix walking.

Thanks!

Regards,
Fengguang