Re: [PATCH 05/32] nohz: Move rcu dynticks idle mode handling to idle enter/exit APIs

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Wed, Aug 31, 2011 at 04:41:00PM +0200, Peter Zijlstra wrote:
> On Wed, 2011-08-31 at 15:37 +0200, Frederic Weisbecker wrote:
> > > Why? rcu-sched can use a context-switch counter, rcu-preempt doesn't
> > > even need that. Remote cpus can notice those just fine.
> > 
> > If that's fine to only rely on context switches, which don't happen in
> > a bounded time in theory, then ok.
> 
> But (!PREEMPT) rcu already depends on that, and suffers this lack of
> time-bounds. What it does to expedite matters is force context switches,
> but nowhere is it written the GP is bounded by anything sane.

Ah, but it really is written, among other things, by the OOM killer.  ;-)

> > > But you then also start the tick again..
> > 
> > When we enter kernel? (minus interrupts)
> > No we only call rcu_exit_nohz(). 
> 
> So thinking more about all this:
> 
> rcu_exit_nohz() will make remote cpus wait for us, this is exactly what
> is needed because we might have looked at pointers. Lacking a tick we
> don't progress our own state but that is fine, !PREEMPT RCU wouldn't
> have been able to progress our state anyway since we haven't scheduled
> (there's nothing to schedule to except idle, see below).

Lacking a tick, the CPU also fails to respond to state updates from
other CPUs.

> Then when we leave the kernel (or go idle) we re-enter rcu_nohz state,
> and the other cpus will ignore our contribution (since we have entered a
> QS and can't be holding any pointers) the other CPUs can continue and
> complete the GP and run the callbacks.

This is true.

> I haven't fully considered PREEMPT RCU quite yet, but I'm thinking we
> can get away with something similar.

All the ways I know of to make PREEMPT_RCU live without a scheduling
clock tick while not in some form of dyntick-idle mode require either
IPIs or read-side memory barriers.  The special case where all CPUs
are in dyntick-idle mode and something needs to happen also needs to
be handled correctly.

Or are you saying that PREEMPT_RCU does not need a CPU to take
scheduling-clock interrupts while that CPU is in dyntick-idle mode?
That is true enough.

> So per the above we don't need the tick at all (for the case of
> nr_running=[0,1]), RCU will sort itself out.
> 
> Now I forgot where all you send IPIs from, and I'll go look at these
> patches once more.
> 
> As for call_rcu() for that we can indeed wake the tick (on leaving
> kernel space or entering idle, no need to IPI since we can't process
> anything before that anyway) or we could hand off our call list to a
> 'willing' victim.
> 
> But yeah, input from Paul would be nice...

In the call_rcu() case, I do have some code in preparation that allows
CPUs to have non-empty callback queues and still be tickless.  There
are some tricky corner cases, but it does look possible.  (Famous last
words...)

The reason for doing this is that people are enabling
CONFIG_RCU_FAST_NO_HZ on systems that have no business enabling it.
Bad choice of names on my part.

							Thanx, Paul