Re: [PATCH v3 tip/core/rcu 3/9] rcu: Add synchronous grace-period waiting for RCU-tasks

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Wed, Aug 06, 2014 at 10:47:08AM +0200, Peter Zijlstra wrote:
> On Tue, Aug 05, 2014 at 06:21:39PM -0700, Paul E. McKenney wrote:
> > > Yeah, idle threads can be affected by the trampolines. That is, we can
> > > still hook a trampoline to some function in the idle loop.
> > > 
> > > But we should be able to make the hardware call that puts the CPU to
> > > sleep a quiescent state too. May need to be arch dependent. :-/
> > 
> > OK, my plan for this eventuality is to do the following:
> > 
> > 1.	Ignore the ->on_rq field, as idle tasks are always on a runqueue.
> > 
> > 2.	Watch the context-switch counter.
> > 
> > 3.	Ignore dyntick-idle state for idle tasks.
> > 
> > 4.	If there is no quiescent state from a given idle task after
> > 	a few seconds, schedule rcu_tasks_kthread() on top of the
> > 	offending CPU.
> > 
> > Your idea is an interesting one, but does require another set of
> > dyntick-idle-like functions and counters.  Or moving the current
> > rcu_idle_enter() and rcu_idle_exit() calls deeper into the idle loop.
> > 
> > Not sure which is a better approach.  Alternatively, we could just
> > rely on #4 above, on the grounds that battery life should not be
> > too badly degraded by the occasional RCU-tasks interference.
> > 
> > Note that this is a different situation than NO_HZ_FULL in realtime
> > environments, where the worst case causes trouble even if it happens
> > very infrequently.
> 
> Or you could shoot all CPUs with resched_cpu() which would have them
> cycle through schedule() even if there's nothing but the idle thread to
> run. That guarantees they'll go to sleep again in a !trampoline.

Good point, that would be an easier way to handle the idle threads than
messing with rcu_tasks_kthread()'s affinity.  Thank you!

> But I still very much hate the polling stuff...
> 
> Can't we abuse the preempt notifiers? Say we make it possible to install
> preemption notifiers cross-task, then the task-rcu can install a
> preempt-out notifier which completes the rcu-task wait.
> 
> After all, since we tagged it it was !running, and being scheduled out
> means it ran (once) and therefore isn't on a trampoline anymore.

Maybe I am being overly paranoid, but couldn't the task be preempted
in a trampoline, be resumed, execute one instruction (still in the
tramopoline) and be preempted again?

> And the tick, which checks to see if the task got to userspace can do
> the same, remove the notifier and then complete.

My main concern with this sort of approach is that I have to deal
with full-up concurrency (200 CPUs all complete tasks concurrently,
for example), which would make for a much larger and more complex patch.
Now, I do admit that it is quite possible that I will end up there anyway,
for example, if more people start using RCU-tasks, but I see no need to
hurry this process.  ;-)

							Thanx, Paul