Re: rcu,sched: spinlock recursion on 3.5-rc2

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Tue, Jun 12, 2012 at 05:07:37PM +0200, Thomas Gleixner wrote:
> On Tue, 12 Jun 2012, Paul E. McKenney wrote:
> > On Tue, Jun 12, 2012 at 03:40:13PM +0200, Thomas Gleixner wrote:
> > > The torture thread got preempted. rcu_preempt_note_context_switch()
> > > tries to unlock the boosting rt mutex.
> > > 
> > > Though rcu_preempt_note_context_switch() is called with rq lock
> > > held. So it's not a surprise that the code will dead lock.
> > > 
> > > My brain hurts already from looking, so Paul to the rescue!
> > 
> > My brain hurts from beating my head on my desk.  It seems that attempts
> > to enhance PREEMPT_RCU's read-side performance require even more paranoia
> > than I normally bring to bear.  :-/
> > 
> > Please see below for what I expect is the relevant revert.
> > 
> > 							Thanx, Paul
> > 
> > ------------------------------------------------------------------------
> > 
> > Revert "rcu: Move PREEMPT_RCU preemption to switch_to() invocation"
> > 
> > This reverts commit 616c310e83b872024271c915c1b9ab505b9efad9
> > (Move PREEMPT_RCU preemption to switch_to() invocation) which can
> > result in runqueue deadlock.
> 
> Hmm, not sure. The deadlock was not triggered in switch_to. It was
> just at the beginning of __schedule()
> 
> need_resched:
> 	preempt_disable();
>         cpu = smp_processor_id();
> 	rq = cpu_rq(cpu);
>         rcu_note_context_switch(cpu);
> 	
> 	Which ends up in rcu_read_unlock_special() which tries to
> 	unlock the rtmutex.
> 
> So that code is still there ....

???

rcu_note_context_switch() does not call rcu_preempt_note_context_switch()
unless the patch is reverted.  No rcu_preempt_note_context_switch(),
no rcu_read_unlock_special().

Besides which, unless I am even more confused than normal, __schedule()
doesn't hold any runqueue locks at this point.  However, those locks
really are held across the call to switch_to() in context_switch().

I bet that gcc inlined context_switch().

							Thanx, Paul