Re: Unexpected rcuc osnoise on PREEMPT_RT with "rcu_nocbs=all rcu_nocb_poll"

[Frederic Weisbecker <frederic@kernel.org>]

Le Fri, Jan 19, 2024 at 04:38:23PM +0100, Leon Woestenberg a écrit :
> Hello all,
> 
> On 6.1.71-rt21 (PREEMPT_RT against vanilla) I am trying to find the
> root cause of an "rcuc" thread scheduled in on an isolated core (CPU
> #4).
> The isolated core should run a single non-kernel user-space-busy-loop
> thread (SCHED_OTHER, not FIFO or RR).
> 
> tsc=reliable isolcpus=domain,managed_irq,4-7 nohz_full=4-7
> rcu_nocbs=all rcu_nocb_poll
> /proc/irq/*/smp_affinity and cpumasks in
> /sys/devices/virtual/workqueue/ all show 000f or less (CPUs 0-3)
> 
> Using osnoise I see rcuc/4 popping up as a cause of noise, but my
> kernel command line should prevent that, right?
> 
> I see this in the kernel boot log, and I wonder if this is related to my issue:
> 
> [    0.063518] rcu:     RCU_SOFTIRQ processing moved to rcuc kthreads.
> 
> tree.c:102:static bool use_softirq = !IS_ENABLED(CONFIG_PREEMPT_RT);
> 
> Could this cause the rcuc threads to become active, even with the
> "rcu_nocbs=all rcu_nocb_poll" setting?

That's unexpected. The RCU load should be moved from rcuc to rcuo* kthreads.

Do you have CONFIG_RCU_STRICT_GRACE_PERIOD=y perhaps?

Also do you really need rcu_nocb_poll? I would expect this option to be useful
for debugging only, though I'm not aware of every usecases.

Thanks.

> 
> 
> Thanks, Leon.
> 
> 
> [    0.063516] rcu: Preemptible hierarchical RCU implementation.
> [    0.063517] rcu:     RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=16.
> [    0.063518] rcu:     RCU priority boosting: priority 1 delay 500 ms.
> [    0.063518] rcu:     RCU_SOFTIRQ processing moved to rcuc kthreads.
> [    0.063519]     No expedited grace period (rcu_normal_after_boot).
> [    0.063519]     Trampoline variant of Tasks RCU enabled.
> [    0.063519]     Rude variant of Tasks RCU enabled.
> [    0.063520]     Tracing variant of Tasks RCU enabled.
> [    0.063520] rcu: RCU calculated value of scheduler-enlistment delay
> is 25 jiffies.
> [    0.063521] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=16
> [    0.066896] NO_HZ: Full dynticks CPUs: 4-7.
> [    0.066897] rcu:     Offload RCU callbacks from CPUs: 0-15.
> [    0.066898] rcu:     Poll for callbacks from no-CBs CPUs.
> [    0.066901] rcu: srcu_init: Setting srcu_struct sizes based on contention.
> [    1.145094] rcu: Hierarchical SRCU implementation.
> [    1.145094] rcu:     Max phase no-delay instances is 1000.
> 
> 
> --
> Leon Woestenberg
> leon@sidebranch.com
> T: +31 40 711 42 76
> M: +31 6 472 30 372
> 
> Sidebranch Embedded Systems
> Eindhoven, The Netherlands
> http://www.sidebranch.com
>