Re: [PATCH 1/2] Customize sched domain via cpuset

[Peter Zijlstra <peterz@infradead.org>]

On Tue, 2008-04-01 at 15:29 +0200, Andi Kleen wrote:
> On Tue, Apr 01, 2008 at 01:56:08PM +0200, Peter Zijlstra wrote:
> > On Tue, 2008-04-01 at 13:40 +0200, Andi Kleen wrote:
> > > Hidetoshi Seto <seto.hidetoshi@jp.fujitsu.com> writes:
> > > 
> > > > Using cpuset, now we can partition the system into multiple sched domains.
> > > > Then, how about providing different characteristics for each domains?
> > > 
> > > Did you actually see much improvement in any relevant workload
> > > from tweaking these parameters?  If yes what did you change?
> > > And how much did it gain?
> > > 
> > > Ideally the kernel should perform well without much tweaking
> > > out of the box, simply because most users won't tweak. Adding a 
> > > lot of such parameters would imply giving up on good defaults which 
> > > is not a good thing.
> > 
> > >From what I understand they need very aggressive idle balancing; much
> > more so than what is normally healty.
> > 
> > I can see how something like that can be useful when you have a lot of
> > very short running tasks. These could pile up on a few cpus and leave
> > others idle.
> 
> Could the scheduler auto tune itself to this situation?
> 
> e.g. when it sees a row of very high run queue inbalances increase the
> frequency of the idle balancer?

Its not actually the idle balancer that's addressed here, but that runs
at 1/HZ, so no we can't do that faster unless you tie it to a hrtimer.

What it does do is more aggresively look for idle cpus on newidle and
fork. Normally we only consider the socket for these lookups, they want
a wider view.

Auto-tune, perhaps although I'm a bit skeptical of heuristics. We'd need
data on the avg 'atom' length of the tasks and idle-ness of remote cpus
and so on.

The thing is, even then it depends on the data footprint of these tasks
and the cost/benefit for your application.

By more aggresively migrating tasks you penalize through-put but get a
better worst case response time.

I'm just not sure we can make that decision for the user.