Re: Next-level bug in SRCU implementation of RCU Tasks Trace + PREEMPT_RT

[Boqun Feng <boqun@kernel.org>]

On Fri, Mar 20, 2026 at 09:24:15AM -0700, Paul E. McKenney wrote:
[...]
> > > > In an alternative universe, BPF has a defer mechanism, and BPF core
> > > > would just call (for example):
> > > > 
> > > >     bpf_defer(call_srcu, ...); // <- a lockless defer
> > > > 
> > > > so the issue won't happen.
> > > 
> > > In theory, this is quite true.
> > > 
> > > In practice, unfortunately for keeping this part of RCU as simple as
> > > we might wish, when a BPF program gets attached to some function in
> > > the kernel, it does not know whether or not that function holds a given
> > > scheduler lock.  For example, there are any number of utility functions
> > > that can be (and are) called both with and without those scheduler
> > > locks held.  Worse yet, it might be attached to a function that is
> > > *never* invoked with a scheduler lock held -- until some out-of-tree
> > > module is loaded.  Which means that this module might well be loaded
> > > after BPF has JIT-ed the BPF program.
> > > 
> > 
> > Hmm.. maybe I failed to make myself more clear. I was suggesting we
> > treat BPF as a special context, and you cannot do everything, if there
> > is any call_srcu() needed, switch it to bpf_defer(). We should have the
> > same result as either 1) call_srcu() locklessly defer itself or 2) a
> > call_srcu_lockless().
> > 
> > Certainly we can call_srcu() do locklessly defer, but if it's only for
> > BPF, that looks like a whack-a-mole approach to me. Say later on we want
> > to use call_hazptr() in BPF for some reason (there is hoping!), then we
> > need to make it locklessly defer as well. Now we have two lockless logic
> > in both call_srcu() and call_hazptr(), if there is a third one, we need
> > to do that as well. So where's the end?
> 
> Except that by the same line of reasoning, how do the BPF guys figure out
> exactly which function calls they need to defer and under what conditions
> they need to defer them?  Keeping in mind that the list of functions and

Can't they just defer anything that is deferrable? If it's deferrable,
then what's the actual cost for BPF to defer it?

> corresponding conditions is subject to change as the kernel continues
> to change.
> 
> > The lockless defer request comes from BPF being special, a proper way to
> > deal with it IMO would be BPF has a general defer mechanism. Whether
> > call_srcu() or call_srcu_lockless() can do lockless defer is
> > orthogonal.
> 
> Fair point, and for the general defer mechanism, I hereby nominate
> the irq_work_queue() function.  We can use this both for RCU and
> for hazard pointers.  The code to make a call_srcu_lockless() and
> call_hazptr_lockless() that includes the relevant checks and that does
> the deferral will not be large, complex, or slow.  Especially assuming
> that we consolidate common checks.
> 
> > BTW, an example to my point, I think we have a deadlock even with the
> > old call_rcu_tasks_trace(), because at:
> > 
> > https://elixir.bootlin.com/linux/v6.19.8/source/kernel/rcu/tasks.h#L384
> > 
> > We do a:
> > 
> > 	mod_timer(&rtpcp->lazy_timer, rcu_tasks_lazy_time(rtp));
> > 
> > which means call_rcu_tasks_trace() may acquire timer base lock, and that
> > means if BPF was to trace a point where timer base lock is held, then we
> > may have a deadlock. So Now I wonder whether you had any magic to avoid
> > the deadlock pre-7.0 or we are just lucky ;-)
> 
> Test it and see!  ;-)
> 

"Program testing can be used to show the presence of bugs, but never to
show their absence!" ;-)

> > See, without a general defer mechanism, we will have a lot of fun
> > auditing all the primitives that BPF may use.
> 
> No, *we* only audit the primitives in our subsystem that BPF actually
> uses when BPF starts using them.  We let the *other* subsystems worry
> about *their* interactions with BPF.
> 

As an RCU mainatainer: fine

As a LOCKING maintainer: shake my head, because for every primitive that
BPF uses, now there could be a normal version and a _bpf/lockless()
version. That could create more maintenance issues, but only time can
tell.

> > > So we really do need to make some variant of call_srcu() that deals
> > > with this.
> > > 
> > > We do have some options.  First, we could make call_srcu() deal with it
> > > directly, or second, we could create something like call_srcu_lockless()
> > > or call_srcu_nolock() or whatever that can safely be invoked from any
> > > context, including NMI handlers, and that invokes call_srcu() directly
> > > when it determines that it is safe to do so.  The advantage of the second
> > > approach is that it avoids incurring the overhead of checking in the
> > > common case.
> > 
> > Within the RCU scope, I prefer the second option.
> 
> Works for me!
> 
> Would you guys like to implement this, or would you prefer that I do so?
> 

I feel I don't have cycles for it soon, I have a big backlog (including
making preempt_count 64bit on 64bit x86). But I will send the fix in the
current call_srcu() for v7.0 and work with Joel to get into Linus' tree.

I will definitely review it if you beat me to it ;-)

Regards,
Boqun

> 							Thanx, Paul
> 
> > Regards,
> > Boqun
> > 
[..]