Re: [PATCH 3/6] cpuset write throttle

[Peter Zijlstra <a.p.zijlstra@chello.nl>]

[-- Attachment #1: Type: text/plain, Size: 2301 bytes --]

On Wed, 2007-10-03 at 20:56 -0700, Christoph Lameter wrote:
> On Wed, 3 Oct 2007, Ethan Solomita wrote:
> 
> > >> 	Unfortunately this eliminates one of the main reasons for the
> > >> per-cpuset throttling. If one cpuset is responsible for pushing one
> > >> disk/BDI to its dirty limit, someone in another cpuset can get throttled.
> > > 
> > > I think that is acceptable. All processes that write to one disk/BDI must 
> > > be affected by congestion on that device. We may have to deal with 
> > > fairness issues later if it indeed becomes a problem.
> > 
> > 	We do see a fairness issue. We've seen delays on the order of 100
> > seconds for just a few writes to disk, and latency is important to us.
> > Perhaps we can detect that the bdi already has a long queue of pending
> > writes and not force more writes at this time so long as the per-cpuset
> > dirty threshold is not too high.
> 
> Arghy.

clameter gone pirate. Its just that you're a few weeks late :-)

Perhaps you can keep a proportion in the cpu-set, and do a similar trick
that the process proportions do.

currently:

  limit = total_limit * p_bdi * (1 - p_task/8)

suggestion:

  limit = total_limit * p_bdi * (1 - p_task/8) * (1 - p_cpuset/4)

That would give a very busy cpuset a limit 1/4 lower than an idle
cpu-set, thereby the idle cpu-set can do light traffic before getting
throttled.

p_bdi is ratio of writeout completions
p_task is ratio of dirtiers
p_cpuset would also be a ratio of dirtiers

Another option would be:

  limit = cpuset_limit * p_bdi * (1 - p_task/8)

Each cpuset gets a pre-proportioned part of the total limit. Overlapping
cpusets would get into some arguments though.

Hmm, maybe combine the two:

  limit = cpuset_limit * p_bdi * (1 - p_task/8) * (1 - p_cpuset/4)

> > 	On a side note, get_dirty_limits() now returns two dirty counts, both
> > the dirty and bdi_dirty, yet its callers only ever want one of those
> > results. Could we change get_dirty_limits to only calculate one dirty
> > value based upon whether bdi is non-NULL? This would save calculation of
> > regular dirty when a bdi is passed.
> 
> Hmmmm.... I think Peter needs to consider this.

we need the total anyway, its where we start calculating the bdi thing
from.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]