Re: Will CPU 0 be forever prohibited from NO_HZ_FULL status?

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Thu, Dec 05, 2013 at 02:20:55AM +0100, Frederic Weisbecker wrote:
> On Wed, Dec 04, 2013 at 11:39:57AM -0800, Paul E. McKenney wrote:
> > Hello, Frederic,
> > 
> > Just realized that I could further decrease RT latency of one of my "shut
> > up RCU on NO_HZ_FULL CPUs" patches if I relied on CPU 0 always having
> > a scheduling-clock tick unless the entire system is idle.  The trick
> > is that I could then rely on CPU 0 to detect RCU CPU stall warnings,
> > and remove the checking from the other CPUs.
> > 
> > Thoughts?
> 
> You're right on time as I'm currently working on that :)
> So the plan is to allow timekeeping to be handled by a set of CPUs (cpu_housekeeping_mask
> which I guess should be ~nohz_full_mask & cpu_online_mask). I think it will be better
> for powersaving. I guess you could balance the RCU stall checks in this
> set of housekeeping CPUs?
> 
> It should be very easy to make the rcu sysidle stuff to support that housekeeping set,
> I just looked into it and all we need to do is to turn the several "cpu == tick_do_timer"
> checks into something like is_housekeeping_cpu(cpu). And may be a few easy details, like which
> CPU from the housekeeping set should get the kick IPI, well the first one available should be a good start,
> of course I expect some issues with cpu hotplug.
> But other than that, RCU sysidle detection is mostly ready to support tracking only a given subset
> of CPUs instead of all of them. That's in fact what it already does currently by excluding the
> fixed boot timekeeping CPU.
> 
> So I'm working on that and should have some patches ready soon.

Thank you for the info!  Nice to know that RCU will continue to be able
to rely on there being at least one housekeeping CPU.  ;-)

At that point, tick_nohz_full_cpu() would still be a good way for RCU
to distinguish housekeeping CPUs from working CPUs, correct?

> In fact I just realized that all the sysidle detection infrastructure is there and working
> but we forgot to plug it in the tick engine, and thus we are still running
> with periodic CPU 0 even with CONFIG_NO_HZ_FULL_SYSIDLE=y. Anyway I have a few changes
> ready to enable that, lets hope testing will be ok :)

Indeed!  ;-)

The CONFIG_NO_HZ_FULL_SYSIDLE=y might complicate things a bit.  But I
guess the problem would be a corner case -- the system entered sysidle
mode with a grace period pending, which should eventually wake up the
corresponding grace-period kthread, which might be prevented from ever
running due to high load or something.  If that problem arises, I will
fix it.

So there!  ;-)

							Thanx, Paul