Re: [PATCH v2 tip/core/rcu 01/22] smpboot: Add common code for notification from dying CPU

["Paul E. McKenney" <paulmck-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>]

On Tue, Mar 17, 2015 at 05:56:21PM +0100, Peter Zijlstra wrote:
> On Tue, Mar 17, 2015 at 03:08:46PM +0100, Peter Zijlstra wrote:
> > On Tue, Mar 17, 2015 at 04:36:48AM -0700, Paul E. McKenney wrote:
> > > On Tue, Mar 17, 2015 at 09:18:07AM +0100, Peter Zijlstra wrote:
> > > > On Mon, Mar 16, 2015 at 11:37:45AM -0700, Paul E. McKenney wrote:
> > > > > From: "Paul E. McKenney" <paulmck-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
> > > > > 
> > > > > RCU ignores offlined CPUs, so they cannot safely run RCU read-side code.
> > > > > (They -can- use SRCU, but not RCU.)  This means that any use of RCU
> > > > > during or after the call to arch_cpu_idle_dead().  Unfortunately,
> > > > > commit 2ed53c0d6cc99 added a complete() call, which will contain RCU
> > > > > read-side critical sections if there is a task waiting to be awakened.
> > > > 
> > > > Got a little more detail there?
> > > 
> > > Quite possibly.  But exactly what sort of detail are you looking for?
> > 
> > What exact RCU usage you ran into that was problematic. It seems to
> > imply that calling complete() -- from a dead cpu -- which ends up in
> > try_to_wake_up() was the problem?
> 
> Hmm, I'm thinking its select_task_rq_*(). And yes, 'fixing' this in the
> wake-up path will penalize everybody for the benefit of the very rare
> case someone is doing a hotplug.
> 
> So yeah, maybe this is the best solution.. Ulgy though :/

Ugly indeed!  I end up doing a polling loop for the generic code.  For the
first round, I updated only architectures that were calling complete().
If that goes well, I will probably update some of the other architecture
as a code-consolidation measure.  Some architectures have special hardware
and firmware hooks, for example, s390 uses a special instruction to do
the wakeup directly.  Those will of course continue doing their own thing.

The ARM guys are trying to do something specific to their hardware, but
I have not heard from them lately.  I should ping them...

							Thanx, Paul