Re: [PATCH, RFC] v7 scalable classic RCU implementation

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Mon, Oct 13, 2008 at 08:03:31PM +0200, Manfred Spraul wrote:
> Paul E. McKenney wrote:
>> Do you send a reschedule IPI to CPUs that are not in dyntick idle mode,
>> but who have failed to pass through a quiescent state?
>>   
> Actually, I never send reschedule IPIs.
>
> - For "usual" cpus, rcu_check_callbacks() checks for quiescent states.
> There should be a set_need_resched() if a cpu holds up a grace period for 
> too long. I just haven't implemented it yet.
> IMHO it doesn't make sense to perform a "for_each_cpu() 
> smd_send_reschedule()". rcu has a hook in each cpu, thus a 
> set_need_resched() by the per-cpu hook is faster/simpler.

One indeed only sends resched IPIs to CPUs that are not responding on
their own.  And in the case of rcutree.c, the loop is not over the CPUs
themselves, but rather over the leaf rcu_node structures (one per 64
CPUs) -- the per-CPU rcu_data structure is touched only for CPUs that
have not yet responded.

> - For nohz cpus, a poller function [schedule_work(), enabled interrupts] 
> peeks into the per-cpu data of the nohz cpu and checks if it is quiet or if 
> it passed through a quiescent state.
> If it didn't, then it sets a cpu_data->kick_poller flag and rcu_irq_exit() 
> reports the grace period.
> No need for an IPI either - rcu has a hook in the irq exit path.

I considered adding a cpu_quiet() on the irq exit path, but eventually
decided that I should instead place the added overhead in the infrequently
invoked force_quiescent_state() function.  Could be argued either way,
of course.

> Right now, I cheat if a nohz cpu is in a long-running nmi 
> [while(other_cpu_is_in_nmi()) cpu_relax()], but I think I can fix that with 
> an set_need_resched() in the rcu_nmi_exit().

Hmmm...  I don't see where the NMI exit path checks the TIF_NEED_RESCHED
flag, but I could easily be missing something.

I instead maintain separate counters for the NMI and irq entry/exit code,
which are checked in the force_quiescent_state() path.

							Thanx, Paul