[HACK] nohz: improve nohz idle hadling in nohz full cores (was Re: The tick when exit idle for nohz_full cpu)

[Luiz Capitulino <lcapitulino@redhat.com>]

On Thu, 3 Sep 2015 01:46:24 +0000
"Jiang, Yunhong" <yunhong.jiang@intel.com> wrote:

> Hi, Frederic
> 	I noticed currently the tick_nohz_idle_exit() will always restart the sched_tick, even when it's on the nohz_full cpu. I'm not sure if we can keep the sched_tick stopped if it's on a nohz_full_cpu. The sched tick will be enabled on tick_nohz_task_switch() if needed. Is it ok to keep the sched tick off in the scheduling process?

There's also the case where the tick is already de-activated when a nohz
full core enters idle. In this case, an RT app that's solely running in
the core will hit nohz idle handling overhead and will get the tick restarted
on wakeup... Just to be de-activated in the next interrupt.

I think what we want is:

 1. If the tick is already de-activated when entering idle, skip
    tick_nohz_idle_enter() and tick_nohz_idle_exit()

 2. Don't restart the tick when exiting idle for nohz full cores

I've tried to come up with something, I'm not entirely skipping
tick_nohz_idle_enter() and tick_nohz_idle_exit() because I'm not sure how
the bookkeeping there is used.

Frederic, does this look like something we could do?

diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 3319e16..ecee8ca 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -781,6 +781,8 @@ static void __tick_nohz_idle_enter(struct tick_sched *ts)
 	int cpu = smp_processor_id();
 
 	now = tick_nohz_start_idle(ts);
+	if (tick_nohz_full_cpu(cpu) && ts->tick_stopped)
+		return;
 
 	if (can_stop_idle_tick(cpu, ts)) {
 		int was_stopped = ts->tick_stopped;
@@ -909,7 +911,8 @@ void tick_nohz_idle_exit(void)
 		tick_nohz_stop_idle(ts, now);
 
 	if (ts->tick_stopped) {
-		tick_nohz_restart_sched_tick(ts, now);
+		if (!tick_nohz_full_cpu(smp_processor_id()))
+			tick_nohz_restart_sched_tick(ts, now);
 		tick_nohz_account_idle_ticks(ts);
 	}
 
-- 
2.1.0