Re: RCU stall warnings even with rcu_nocbs and rcu_nocb_poll

[Joel Fernandes <joel@joelfernandes.org>]

On Wed, Oct 05, 2022 at 06:03:39PM +0200, Daniel Bristot de Oliveira wrote:
> On 10/5/22 18:01, Florent Carli wrote:
> > Hello,
> > 
> > I'm trying to isolate some cores to run a CPU-bound real-time task and
> > even though I'm using rcu_nocbs and rcu_nocb_poll, I can see the rcuc
> > threads wake up, and I get RCU stall warnings on the isolated core.

Isolating a CPU does not mean that RCU activity is forbidden on it, even with
nohz_full, if the CPU enters the kernel (say syscall or interrupt), then it
may enter a read-side critical section so RCU will be watching it, and such
CPU will have to report quiescent state.

> > As per the documentation, if I use rcutree.kthread_prio with a
> > priority > my RT task, then the rcu stall does not happen.

Yes, it sounds like the main RCU GP thread (mostly called rcu_preempt thread)
in your system is competing with your RT task. Increase kthread_prio is
standard procedure to resolve this issue, as you did.

> > However I find it confusing: why are the rcuc threads woken up on the
> > isolated cores despite using rcu_nocbs and rcu_nocb_poll? In my (very
> > likely erroneous) understanding, I shouldn't have to fiddle with rcu
> > priority... In other words, how come I get rcu stalls on cores that
> > have no rcu callbacks?

It is not possible to provide further help without more info, in particular
are you using nohz_full and isolcpus options? Can you provide kernel
and #CPU configuration?

Happy to look further! Also adding some more folks who know a lot about this
stuff and +rcu list for archives.

thanks,

 - Joel

> 
> Adding Joel because we had a chat about it during lpc...
> 
> > Thanks.
> > 
> -- Daniel
>