Re: [patch 1/4] x86, mtrr: lock stop machine during MTRR rendezvous sequence

[Peter Zijlstra <a.p.zijlstra@chello.nl>]

On Thu, 2011-06-23 at 11:33 +0200, Thomas Gleixner wrote:
> On Thu, 23 Jun 2011, Peter Zijlstra wrote:
> 
> > On Wed, 2011-06-22 at 15:20 -0700, Suresh Siddha wrote:
> > > +#ifdef CONFIG_SMP
> > > +       /*
> > > +        * If we are not yet online, then there can be no stop_machine() in
> > > +        * parallel. Stop machine ensures this by using get_online_cpus().
> > > +        *
> > > +        * If we are online, then we need to prevent a stop_machine() happening
> > > +        * in parallel by taking the stop cpus mutex.
> > > +        */
> > > +       if (cpu_online(raw_smp_processor_id()))
> > > +               mutex_lock(&stop_cpus_mutex);
> > > +#endif 
> > 
> > This reads like an optimization, is it really worth-while to not take
> > the mutex in the rare offline case?
>  
> You cannot block on a mutex when you are not online, in fact you
> cannot block on it when not active, so the check is wrong anyway.

Duh, yeah. Comment totally mislead me.

On that whole active thing, so cpu_active() is brought into life to sort
an cpu-down problem, where we want the lb to stop using a cpu before we
can re-build the sched_domains.

But now we're having trouble because of that on the cpu-up part, where
we update the sched_domains too late (CPU_ONLINE) and hence also set
cpu_active() too late (again CPU_ONLINE).

Couldn't we update the sched_domain tree on CPU_PREPARE_UP to include
the new cpu and then set cpu_active() right along with cpu_online()?

That would also sort your other wait for active while bringup issue..

Note, I'll now go and have my morning juice, so the above might be total
crap.