Re: [RFC PATCH v3 3/5] sched: nominate preferred wakeup cpu

[Vaidyanathan Srinivasan <svaidy@linux.vnet.ibm.com>]

* Peter Zijlstra <a.p.zijlstra@chello.nl> [2008-11-11 16:26:14]:

> On Tue, 2008-11-11 at 20:51 +0530, Srivatsa Vaddagiri wrote:
> > On Tue, Nov 11, 2008 at 09:07:58AM -0500, Gregory Haskins wrote:
> > > > Would it make sense to place the preferred_wakeup_cpu stuff in the
> > > > root_domain structure we already have?
> > > >   
> > > 
> > > From the description, this is exactly what the root-domains were created
> > > to solve.
> > > 
> > > Vaidyanathan,  just declare your object in "struct root_domain" and
> > > initialize it in init_rootdomain() in kernel/sched.c, and then access it
> > > via rq->rd to take advantage of this infrastructure.  It will
> > > automatically follow any partitioning that happens to be configured.
> > 
> > If I understand correctly, we may want to have more than one preferred
> > cpu in a given sched domain, taking into account node topology i.e if a
> > given sched domain encompasses two nodes, then we may like to designate
> > 2 preferred wakeup_cpu's, one per node. If that is the case, then
> > root_domain may not be of use here?
> 
> Agreed, in which case this sched_domain_attr stuff might work out better
> - but I'm not sure I fully get that.. will stare at that a bit more.

The current code that I posted assumes one preferred_wakeup_cpu per
partitioned domain.  Moving the variable to root_domain is a good idea
for this implementation.

In future when we need one preferred_wakeup_cpu per node per
partitioned domain, we will need a array for each partitioned domain.
Having the array in root_domain is better than having it in dattr.

Depending upon experimental results, we may choose to have only one
preferred_wakeup_cpu per partitioned domain.  When the system
utilisation is quite low, it is better to move all movable tasks from
each node to a selected node (0).  This will freeup all CPUs in other
nodes.  Just that we need to consider cache hotness and cross-node
memory access more carefully before crossing a node boundary for
consolidation.

--Vaidy