Re: [PATCH 05/32] nohz: Move rcu dynticks idle mode handling to idle enter/exit APIs

[Peter Zijlstra <peterz@infradead.org>]

On Wed, 2011-08-31 at 15:37 +0200, Frederic Weisbecker wrote:
> > Why? rcu-sched can use a context-switch counter, rcu-preempt doesn't
> > even need that. Remote cpus can notice those just fine.
> 
> If that's fine to only rely on context switches, which don't happen in
> a bounded time in theory, then ok.

But (!PREEMPT) rcu already depends on that, and suffers this lack of
time-bounds. What it does to expedite matters is force context switches,
but nowhere is it written the GP is bounded by anything sane.

> > But you then also start the tick again..
> 
> When we enter kernel? (minus interrupts)
> No we only call rcu_exit_nohz(). 

So thinking more about all this:

rcu_exit_nohz() will make remote cpus wait for us, this is exactly what
is needed because we might have looked at pointers. Lacking a tick we
don't progress our own state but that is fine, !PREEMPT RCU wouldn't
have been able to progress our state anyway since we haven't scheduled
(there's nothing to schedule to except idle, see below).

Then when we leave the kernel (or go idle) we re-enter rcu_nohz state,
and the other cpus will ignore our contribution (since we have entered a
QS and can't be holding any pointers) the other CPUs can continue and
complete the GP and run the callbacks.

I haven't fully considered PREEMPT RCU quite yet, but I'm thinking we
can get away with something similar.

So per the above we don't need the tick at all (for the case of
nr_running=[0,1]), RCU will sort itself out.

Now I forgot where all you send IPIs from, and I'll go look at these
patches once more.

As for call_rcu() for that we can indeed wake the tick (on leaving
kernel space or entering idle, no need to IPI since we can't process
anything before that anyway) or we could hand off our call list to a
'willing' victim.

But yeah, input from Paul would be nice...