Re: [PATCH 06/13] writeback: bdi write bandwidth estimation

[Wu Fengguang <fengguang.wu@intel.com>]

On Wed, Nov 24, 2010 at 08:50:47PM +0800, Peter Zijlstra wrote:
> On Wed, 2010-11-24 at 20:10 +0800, Wu Fengguang wrote:
> > > > +       /*
> > > > +        * When there lots of tasks throttled in balance_dirty_pages(), they
> > > > +        * will each try to update the bandwidth for the same period, making
> > > > +        * the bandwidth drift much faster than the desired rate (as in the
> > > > +        * single dirtier case). So do some rate limiting.
> > > > +        */
> > > > +       if (jiffies - bdi->write_bandwidth_update_time < elapsed)
> > > > +               goto snapshot;
> > >
> > > Why this goto snapshot and not simply return? This is the second call
> > > (bdi_update_bandwidth equivalent).
> > 
> > Good question. The loop inside balance_dirty_pages() normally run only
> > once, however wb_writeback() may loop over and over again. If we just
> > return here, the condition
> > 
> >         (jiffies - bdi->write_bandwidth_update_time < elapsed)
> > 
> > cannot be reset, then future bdi_update_bandwidth() calls in the same
> > wb_writeback() loop will never find it OK to update the bandwidth.
> 
> But the thing is, you don't want to reset that, it might loop so fast
> you'll throttle all of them, if you keep the pre-throttle value you'll
> eventually pass, no?

It (let's name it A) only resets the _local_ vars bw_* when it's sure
by the condition

        (jiffies - bdi->write_bandwidth_update_time < elapsed)

that someone else (name B) has updated the _global_ bandwidth in the
time range we planned. So there may be some time in A's range that is
not covered by B, but sure the range is not totally bypassed without
updating the bandwidth.

> > It does assume no races between CPUs.. We may need some per-cpu based
> > estimation. 
> 
> But that multi-writer race is valid even for the balance_dirty_pages()
> call, two or more could interleave on the bw_time and bw_written
> variables.

The race will only exist in each task's local vars (their bw_* will
overlap). But the update bdi->write_bandwidth* will be safeguarded
by the above check. When the task is scheduled back, it may find
updated write_bandwidth_update_time and hence give up his estimation.
This is rather tricky..

Thanks,
Fengguang