Re: [PATCH 08/11 v2] nohz: Allow rcu extended quiescent state handling seperately from tick stop

[Frederic Weisbecker <fweisbec@gmail.com>]

On Thu, Oct 13, 2011 at 12:03:57AM -0700, Paul E. McKenney wrote:
> On Wed, Oct 12, 2011 at 11:57:52PM -0700, Paul E. McKenney wrote:
> > On Sat, Oct 08, 2011 at 04:01:00PM +0200, Frederic Weisbecker wrote:
> > > It is assumed that rcu won't be used once we switch to tickless
> > > mode and until we restart the tick. However this is not always
> > > true, as in x86-64 where we dereference the idle notifiers after
> > > the tick is stopped.
> > > 
> > > To prepare for fixing this, add two new APIs:
> > > tick_nohz_idle_enter_norcu() and tick_nohz_idle_exit_norcu().
> > > 
> > > If no use of RCU is made in the idle loop between
> > > tick_nohz_enter_idle() and tick_nohz_exit_idle() calls, the arch
> > > must instead call the new *_norcu() version such that the arch doesn't
> > > need to call rcu_idle_enter() and rcu_idle_exit().
> > > 
> > > Otherwise the arch must call tick_nohz_enter_idle() and
> > > tick_nohz_exit_idle() and also call explicitly:
> > > 
> > > - rcu_idle_enter() after its last use of RCU before the CPU is put
> > > to sleep.
> > > - rcu_idle_exit() before the first use of RCU after the CPU is woken
> > > up.
> > 
> > Thank you, Frederic!  I have queued this to replace the earlier
> > version.  The set is available on branch rcu/dyntick of
> > 
> > 	https://github.com/paulmckrcu/linux
> 
> Which reminds me...  About the ultimate objective, getting tick-free
> operation.  (Or, for the guys who want to eliminate the tick entirely,
> shutting up the hrtimer stuff that they want to replace it with.)
> 
> I believe that you will then need to have two levels of not-in-dynticks
> for processes, one for idle vs. not and another for when a process
> switches from user-space to kernel execution.  Correct, or am I
> confused?
> 
> The reason I ask is that commit e11f5981 currently only allows one
> level of not-in-dynticks for processes.  It is easy to add another
> level, but thought I should check beforehand.

Hmm, yeah looking at that patch, it's going to be hard to have a nesting
that looks like:

rcu_irq_enter();
rcu_user_enter();
rcu_irq_exit(); <-- with effective extended quiescent state starting there

I also need to be able to call rcu_user_enter() from non-irq path.

I don't truly understand the problem of the usermode helpers that
mess up the dynticks counts. May be we can somewhow fix it differently
from the offending callsite?