[Query] Can we use normal timers (kernel/timer.c) while in NO_HZ_FULL mode?

[Query] Can we use normal timers (kernel/timer.c) while in NO_HZ_FULL mode?

[Viresh Kumar]

Hi,

While working on the ONESHOT_STOPPED mode I came across
another confusing scenario..

Normal timers (kernel/timer.c) don't configure clockevent devices
at all but they always rely on PERIODIC tick interrupts to get them
scheduled. i.e. normal timers would be only serviced at next tick
interrupt (in both LOW & HIGH resolution modes)..

Suppose we have entered into NO_HZ_FULL mode (we made
sure that there are no normal timers queued) and a normal
timer was added after that. We will add it to the timer list but
as there is no tick-sched timer, we wouldn't be able to service
the normal timer until next time tick fires again (MAX 1 second
currently)..

And once we remove this MAX 1 second limitation, we might
not service this normal timer for long..

Does this problem statement make sense? Or we don't have
any such problem?

--
viresh

Re: [Query] Can we use normal timers (kernel/timer.c) while in NO_HZ_FULL mode?

[Frederic Weisbecker]

On Wed, Jun 04, 2014 at 05:11:08PM +0530, Viresh Kumar wrote:
> Hi,
> 
> While working on the ONESHOT_STOPPED mode I came across
> another confusing scenario..
> 
> Normal timers (kernel/timer.c) don't configure clockevent devices
> at all but they always rely on PERIODIC tick interrupts to get them
> scheduled. i.e. normal timers would be only serviced at next tick
> interrupt (in both LOW & HIGH resolution modes)..
> 
> Suppose we have entered into NO_HZ_FULL mode (we made
> sure that there are no normal timers queued) and a normal
> timer was added after that. We will add it to the timer list but
> as there is no tick-sched timer, we wouldn't be able to service
> the normal timer until next time tick fires again (MAX 1 second
> currently)..
> 
> And once we remove this MAX 1 second limitation, we might
> not service this normal timer for long..
> 
> Does this problem statement make sense? Or we don't have
> any such problem?

Right, if we enqueue a timer when the tick is stopped, we call wake_up_nohz_cpu().
So here is two scenarios:

1) Target is remote. An IPI is sent if necessary and the next tick is reevaluated from the
target's irq exit

2) Target is local. We assume that nobody needs to enqueue a timer between
tick_nohz_idle_enter() and tick_nohz_idle_exit() calls. Because it's dead idle area.
But irqs can happen in nohz mode, and they can queue timers. Then the next tick
can be reevaluated on irq exit. But we should better check that no code like idle
drivers, governors and other idle stuff ever call a timer on the dead zone.

Now full nohz is actually special in that it can call the ipi locally if the target
is local. This is currently using the scheduler IPI but I may change that to use
irq work because I'm not sure that all archs support scheduler self-IPIs.

Re: [Query] Can we use normal timers (kernel/timer.c) while in NO_HZ_FULL mode?

[Viresh Kumar]

On 4 June 2014 18:53, Frederic Weisbecker <fweisbec@gmail.com> wrote:
> Right, if we enqueue a timer when the tick is stopped, we call wake_up_nohz_cpu().

Ahh, I looked at __mod_timer() :)

BTW, shouldn't we do something similar for mod_timer_pinned()
as well ?

Re: [Query] Can we use normal timers (kernel/timer.c) while in NO_HZ_FULL mode?

[Viresh Kumar]

On 4 June 2014 19:14, Viresh Kumar <viresh.kumar@linaro.org> wrote:
> BTW, shouldn't we do something similar for mod_timer_pinned()
> as well ?

Also, as we aren't forcing get_nohz_timer_target() to _not_ return
a NO_HZ_FULL, we might end up queuing a timer on NO_HZ_FULL
cpu with the unpinned timer APIs. And the same question might
stand true for this case as well?

Re: [Query] Can we use normal timers (kernel/timer.c) while in NO_HZ_FULL mode?

[Frederic Weisbecker]

On Wed, Jun 04, 2014 at 07:23:20PM +0530, Viresh Kumar wrote:
> On 4 June 2014 19:14, Viresh Kumar <viresh.kumar@linaro.org> wrote:
> > BTW, shouldn't we do something similar for mod_timer_pinned()
> > as well ?
> 
> Also, as we aren't forcing get_nohz_timer_target() to _not_ return
> a NO_HZ_FULL, we might end up queuing a timer on NO_HZ_FULL
> cpu with the unpinned timer APIs. And the same question might
> stand true for this case as well?

Right.

Re: [Query] Can we use normal timers (kernel/timer.c) while in NO_HZ_FULL mode?

[Frederic Weisbecker]

On Wed, Jun 04, 2014 at 07:14:43PM +0530, Viresh Kumar wrote:
> On 4 June 2014 18:53, Frederic Weisbecker <fweisbec@gmail.com> wrote:
> > Right, if we enqueue a timer when the tick is stopped, we call wake_up_nohz_cpu().
> 
> Ahh, I looked at __mod_timer() :)
> 
> BTW, shouldn't we do something similar for mod_timer_pinned()
> as well ?

Ah that looks right. In fact we should probably move the wake_up_nohz_cpu()
to internal_add_timer().

And that concerns nohz idle as well. If mod_timer() selects a CPU that doesn't
appear to be idle in get_nohz_timer_target() but then becomes idle afterward
before we lock its timer base, the CPU may well miss the timer.