Re: sched/isolation: tick_take_do_timer_from_boot() calls smp_call_function_single() with irqs disabled

[Oleg Nesterov <oleg@redhat.com>]

The more I grep the more I confused.

On 05/25, Thomas Gleixner wrote:
>
> Right. It does not happen because the kernel starts with jiffies as
> clocksource except on S390. The jiffies clocksource is not qualified to
> switch over to NOHZ mode for obvious reasons.

Not obvious for those who never looked at this code ;)

OK, clocksource_jiffies doesn't have CLOCK_SOURCE_VALID_FOR_HRES,

> fs_initcall(clocksource_done_booting)

So clocksource_done_booting() -> clocksource_select() should find another
CLOCK_SOURCE_VALID_FOR_HRES cs, then timekeeping_notify(best)

	- sets tk_core.timekeeper.tkr_mono.clock = best

	- calls tick_clock_notify() which sets .check_clocks on every cpu.

This makes tick_check_oneshot_change(false) return true. (I am ignoring the
highres=n case, tick_nohz_switch_to_nohz() doesn't really differ).

Now, on every CPU the next ->event_handler == tick_handle_periodic path
call hrtimer_switch_to_hres() which

	- sets ->event_handler == hrtimer_interrupt (tick_init_highres)
	  so tick_periodic/do_timer will be never called again

	- calls tick_setup_sched_timer() -> tick_nohz_activate() which
	  sets TS_FLAG_NOHZ

> [    1.918548] clocksource_done_booting: Switched to NOHZ // debug printk
>
> This is the point where tick_nohz_activate() is called first time and
> that does:
>
>   tick_sched_flag_set(ts, TS_FLAG_NOHZ);

See above, but I got lost, most probably I misunderstood these nontrivial
code paths.

> So up to this point the tick is never stopped neither on housekeeping
> nor on NOHZ FULL CPUs:
>
> tick_nohz_full_update_tick()
>   if (!tick_sched_flag_test(ts, TS_FLAG_NOHZ))
>     return;

OK... But tick_nohz_idle_update_tick() doesn't check TS_FLAG_NOHZ and
the tick_nohz_full_cpu() check can't help at boot time.

And I still don't understand why we can rely on can_stop_idle_tick() even
in tick_nohz_idle_stop_tick().

I'll try to read this code again tomorrow, but it will never fit my poor
little brain ;)

Oleg.