Re: New RCU related warning due to rcu_preempt_depth() changes

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Wed, Apr 18, 2012 at 07:29:19AM +0200, Sasha Levin wrote:
> On Tue, Apr 17, 2012 at 6:45 PM, Paul E. McKenney
> <paulmck@linux.vnet.ibm.com> wrote:
> > On Tue, Apr 17, 2012 at 08:53:16AM -0700, Paul E. McKenney wrote:
> >> On Tue, Apr 17, 2012 at 05:36:59PM +0200, Sasha Levin wrote:
> >> > On Tue, Apr 17, 2012 at 5:05 PM, Paul E. McKenney
> >> > <paulmck@linux.vnet.ibm.com> wrote:
> >> > > On Tue, Apr 17, 2012 at 10:42:47AM +0200, Sasha Levin wrote:
> >> > >> Hi Paul,
> >> > >>
> >> > >> It looks like commit 7298b03 ("rcu: Move __rcu_read_lock() and
> >> > >> __rcu_read_unlock() to per-CPU variables") is causing the following
> >> > >> warning (I've added the extra fields on the second line):
> >> > >>
> >> > >> [   77.330920] BUG: sleeping function called from invalid context at
> >> > >> mm/memory.c:3933
> >> > >> [   77.336571] in_atomic(): 0, irqs_disabled(): 0, preempt count: 0,
> >> > >> preempt offset: 0, rcu depth: 1, pid: 5669, name: trinity
> >> > >> [   77.344135] no locks held by trinity/5669.
> >> > >> [   77.349644] Pid: 5669, comm: trinity Tainted: G        W
> >> > >> 3.4.0-rc3-next-20120417-sasha-dirty #83
> >> > >> [   77.354401] Call Trace:
> >> > >> [   77.355956]  [<ffffffff810e83f3>] __might_sleep+0x1f3/0x210
> >> > >> [   77.358811]  [<ffffffff81198eaf>] might_fault+0x2f/0xa0
> >> > >> [   77.361997]  [<ffffffff810e3228>] schedule_tail+0x88/0xb0
> >> > >> [   77.364671]  [<ffffffff826a01d3>] ret_from_fork+0x13/0x80
> >> > >>
> >> > >> As you can see, rcu_preempt_depth() returns 1 when running in that
> >> > >> context, which looks pretty odd.
> >> > >
> >> > > Ouch!!!
> >> > >
> >> > > So it looks like I missed a place where I need to save and restore
> >> > > the new per-CPU rcu_read_lock_nesting and rcu_read_unlock_special
> >> > > variables.  My (probably hopelessly naive) guess is that I need to add
> >> > > a rcu_switch_from() and rcu_switch_to() into schedule_tail(), but to
> >> > > make rcu_switch_from() take the task_struct pointer as an argument,
> >> > > passing in prev.
> >> > >
> >> > > Does this make sense, or am I still missing something here?
> >> >
> >> > I've let the test run for a bit more, and it appears that I'm getting
> >> > this warning from lots of different sources, would this
> >> > schedule_tail() fix all of them?
> >>
> >> If I understand the failure correctly, yes.  If the task switches without
> >> RCU paying attention, the nesting count for both the outgoing and the
> >> incoming tasks can get messed up.  The messed-up counts could easily
> >> cause problems downstream.
> >>
> >> Of course, there might well be additional bugs.
> >>
> >> I will put a speculative patch together and send it along.
> >
> > And here it is, testing just started.
> >
> >                                                        Thanx, Paul
> >
> > ------------------------------------------------------------------------
> >
> > rcu: Add RCU context switching to schedule_tail()
> >
> > The new rcu_read_lock_nesting and rcu_read_unlock_special per-CPU
> > variables must be saved and restored at every context switch, including
> > those involving schedule_tail().  This commit therefore adds the saving
> > and restoring to schedul_tail().
> >
> > Reported-by: Sasha Levin <levinsasha928@gmail.com>
> > Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
> > Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> 
> Looks good here.

Very good!  I have added your Tested-by.

							Thanx, Paul