Re: [PATCH] rcu: Delay the RCU-selftests during boot.

[Sebastian Andrzej Siewior <bigeasy@linutronix.de>]

On 2022-03-01 21:42:17 [-0800], Paul E. McKenney wrote:
> There have been several different things requiring various pieces of RCU
> to be running quite early, and I need to do a bit of code archaeology to
> check this out.  One of these is what forced me to move to expedited grace
> periods once the scheduler started.  And the expedited grace periods do
> require the timers be working, at least in their current form.

expedited as in synchronize_rcu_expedited()? Because we don't do this
but then only after boot so it probably doesn't change a thing and makes
other complicated.

> I vaguely recall tracing requiring call_rcu() extremely early (as
> in before rcu_init() is called), but that was not a problem because
> there was no requirement that the callback be invoked before the first
> couple of sets of initcalls had completed.  The only challenge in this
> case was to avoid stomping on the callback lists at rcu_init() time,
> no timers required.

Okay.

> One straightforward approach would be to choose the call site of the
> rcu_tasks_initiate_self_tests() function based on whether or not the
> kernel was built with CONFIG_PREEMPT_RT.

But then we have this change between RT and !RT due to other
limitations. But my understanding is correct that RT does not have a
working synchronize_rcu() in an early_initcall() while !RT does, right?

> The other thought that comes to mind is that you may need the old-style
> softirq until ksoftirqd is spawned.  I doubt that real-time response is
> a big deal that early in boot.  ;-)

Puh. Given that the whole infrastructure is not prepared for that it is
probably more than a few lines plus a conditional jump in the IRQ
hot path. Also we do have scheduling at this point and IRQ-on/off works
which means possible dead locks because sleeping locks are acquired in
hard-IRQ context.

But let me summarize this: I did this:

|@@ -1600,6 +1600,7 @@ static noinline void __init kernel_init_
|
|        rcu_init_tasks_generic();
|        do_pre_smp_initcalls();
|+       rcu_tasks_initiate_self_tests();
|        lockup_detector_init();
|
|        smp_init();

To simply move the test from rcu_init_tasks_generic() to after
do_pre_smp_initcalls(). If we can't move rcu_init_tasks_generic() after
do_pre_smp_initcalls() or at least the test part because we need working
synchronize_rcu() in early_initcall() then I need to move the RT
requirements before. Simple ;)

The requirements are:

--- a/init/main.c
+++ b/init/main.c
@@ -1598,6 +1601,9 @@ static noinline void __init kernel_init_freeable(void)
 
        init_mm_internals();
 
+       spawn_ksoftirqd();
+       irq_work_init_threads();
+
        rcu_init_tasks_generic();
        do_pre_smp_initcalls();
        lockup_detector_init();

spawn_ksoftirqd() is what I mentioned. What I just figured out is
irq_work_init_threads() due to
	call_rcu_tasks_iw_wakeup()

I can't move this to hard-IRQ context because it invokes wake_up() which
acquires sleeping locks. If you say that rtp->cbs_wq has only one waiter
and something like rcuwait_wait_event() / rcuwait_wake_up() would work
as well then call_rcu_tasks_iw_wakeup() could be lifter to hard-IRQ
context and we need to worry only about spawn_ksoftirqd() :)

> 							Thanx, Paul
> 

Sebastian