Re: [RFC PATCH] rcu-tasks: Avoid using mod_timer() in call_rcu_tasks_generic()

[Boqun Feng <boqun@kernel.org>]

On Mon, Mar 23, 2026 at 10:50:18PM +0100, Kumar Kartikeya Dwivedi wrote:
> On Mon, 23 Mar 2026 at 16:17, Boqun Feng <boqun@kernel.org> wrote:
> >
> > On Sat, Mar 21, 2026 at 10:03:21AM -0700, Boqun Feng wrote:
> > > The following deadlock is possible:
> > >
> > > __mod_timer()
> > >   lock_timer_base()
> > >     raw_spin_lock_irqsave(&base->lock)   <- base->lock ACQUIRED
> > >   trace_timer_start()                    <- tp_btf/timer_start fires here
> > >     [probe_timer_start BPF program]
> > >       bpf_task_storage_delete()
> > >         bpf_selem_unlink(selem, false)   <- reuse_now=false
> > >           bpf_selem_free(false)
> > >             call_rcu_tasks_trace()
> > >               call_rcu_tasks_generic()
> > >                 raw_spin_trylock(rtpcp)  <- succeeds (different lock)
> > >                 mod_timer(lazy_timer)    <- lazy_timer is on this CPU's base
> > >                   lock_timer_base()
> > >                     raw_spin_lock_irqsave(&base->lock) <- SAME LOCK -> DEADLOCK
> > >
> > > because BPF can instrument a place while the timer base lock is held.
> > > Fix it by using an intermediate irq_work.
> > >
> > > Further, because a "timer base->lock" to a "rtpcp lock" lock dependency
> > > can be establish in this way, we cannot mod_timer() with a rtpcp lock
> > > held. Fix that as well.
> > >
> > > Fixes: d119357d0743 ("rcu-tasks: Treat only synchronous grace periods urgently")
> > > Cc: stable@kernel.org
> > > Signed-off-by: Boqun Feng <boqun@kernel.org>
> > > ---
> > > This is a follow-up of [1], and yes we can trigger a whole system
> > > deadlock freeze easily with (even non-recursively) tracing the
> > > timer_start tracepoint. I have a reproduce at:
> > >
> > >       https://github.com/fbq/rcu_tasks_deadlock
> > >
> > > Be very careful, since it'll freeze your system when run it.
> > >
> > > I've tested it on 6.17 and 6.19 and can confirm the deadlock could be
> > > triggered. So this is an old bug if it's a bug.
> > >
> > > It's up to BPF whether this is a bug or not, because it has existed for
> > > a while and nobody seems to get hurt(?).
> > >
> >
> > Ping BPF ;-) I know this was sent in a Saturday and only 2 days have
> > passed, but we are at the decision point about how hard/urgent we should
> > fix these "BPF deadlocks": As this patch shows, the deadlocks existed
> > before v7.0 (i.e. before SRCU switches). And yes, ideally we should fix
> > all of them, but given we are close to v7.0 release, I would like to
> > focus the new issue that SRCU introduces, because that one would likely
> > affect SCHED_EXT. Thoughts?
> 
> I tried both of your changes, thanks for working on these. I agree the

No problem. :)

> one reported by Andrea is more important, so that should be the first
> priority and should be sent as a fix for 7.0.
> 

Agreed. I will work with Joel to forward these fixes to Linus.

> It would be good to make the timer-related fix too, but it doesn't
> need to be rushed for 7.0. We're aware of the issue being fixed by
> this patch in timer tracepoints [0].
> It's a corner case which no one has hit thus far. We already made
> similar fixes where we could, e.g. [1], but it's difficult to make a
> similar change for local storage.
> 

Right, we need more time to cover all the cases. For example, there is 
tracepoint inside irq_work_queue() where we can instrument it within
BPF, that could open a lot of fun interactions. ;-)

> Given Paul said he plans into looking into call_{s,}rcu_nolock()
> anyway, the guard can be dropped once call_{s,}rcu_nolock()
> materializes.
> 
> Something like what you did in this patch would be prerequisite for
> the call_{s,}rcu_nolock() anyway, the assumption will be that it could
> be invoked from NMI, so reentrancy can happen anywhere, it doesn't
> matter much then whether it happens in the same context when the lock
> is held and we end up invoking the same path through call_srcu(), or
> when an NMI prog interrupts when the timer lock is held.
> 

Sounds good, I will work with Paul on these.

Regards,
Boqun

>   [0]: https://lore.kernel.org/bpf/CAP01T76xUCrDH4G2XikNvhPTn6ZbNTgQH59qt2Q_o0c9uudd8w@mail.gmail.com
>   [1]: https://lore.kernel.org/bpf/20260204055147.54960-2-alexei.starovoitov@gmail.com
> 
> >
> > [...]