sched/isolation: tick_take_do_timer_from_boot() calls smp_call_function_single() with irqs disabled

[Oleg Nesterov <oleg@redhat.com>]

After the recent comment 5097cbcb38e6 ("sched/isolation: Prevent boot crash
when the boot CPU is nohz_full") the kernel no longer crashes, but there is
another problem.

In this case tick_setup_device() does tick_take_do_timer_from_boot() to
update tick_do_timer_cpu and this triggers WARN_ON_ONCE(irqs_disabled())
in smp_call_function_single().

I don't understand this code even remotely, I failed to find the fix.

Perhaps we can use smp_call_function_single_async() as a workaround ?

But I don't even understand why exactly we need smp_call_function()...
Lets suppose we change

	--- a/kernel/time/tick-common.c
	+++ b/kernel/time/tick-common.c
	@@ -231,9 +231,9 @@ static void tick_setup_device(struct tick_device *td,
	 
			} else if (tick_do_timer_boot_cpu != -1 &&
							!tick_nohz_full_cpu(cpu)) {
	-			tick_take_do_timer_from_boot();
	+//			tick_take_do_timer_from_boot();
				tick_do_timer_boot_cpu = -1;
	-			WARN_ON(READ_ONCE(tick_do_timer_cpu) != cpu);
	+			WRITE_ONCE(tick_do_timer_cpu), cpu);
	 #endif
			}

Can anyone explain what exactly can go wrong?

Race with tick_nohz_stop_tick() on boot CPU which can set
tick_do_timer_cpu = TICK_DO_TIMER_NONE? Is it really bad?

Something else?

Please help, I don't think I can find a sane solution :/

Oleg.