[PATCH v3] mm/slab: don't use kfree_rcu sheaves on PREEMPT_RT in kvfree_call_rcu()

[PATCH v3] mm/slab: don't use kfree_rcu sheaves on PREEMPT_RT in kvfree_call_rcu()

[ThangNN99]

syzbot reports a possible circular locking dependency between
&p->pi_lock and the per-CPU kfree_rcu sheaf lock (_T->lock) on
PREEMPT_RT:

  __balance_push_cpu_stop()               [holds p->pi_lock]
    select_fallback_rq()
      cpuset_cpus_allowed_fallback()
        set_cpus_allowed_force()
          kfree_rcu(ac.user_mask)
            kvfree_call_rcu()
              kfree_rcu_sheaf()
                __kfree_rcu_sheaf()
                  local_trylock(&s->cpu_sheaves->lock)   <- _T->lock

set_cpus_allowed_force() uses kfree_rcu() instead of kfree() here
because all of its callers hold task_struct::pi_lock (a
raw_spinlock_t), and plain kfree() may sleep under PREEMPT_RT.

Commit 2a8bb29ec9b2 ("mm/slab: allow kfree_rcu_sheaf() on
PREEMPT_RT") made kvfree_call_rcu() try the sheaves fast path on
PREEMPT_RT too, since __kfree_rcu_sheaf() only trylocks there and
so cannot itself block. True, but the sheaf/barn locks it trylocks
are also taken as regular, blocking locks elsewhere, so lockdep
still records a lock-class ordering cycle against any raw_spinlock_t
already held by the caller, which is what syzbot caught.

The plain kfree_rcu()/kvfree_rcu() API gives kvfree_call_rcu() no
way to know the caller is in such a context, so keep it
conservative on PREEMPT_RT and skip the sheaves layer there,
falling back to the existing raw_spinlock_t-protected krcp list,
which is always safe to nest under another raw_spinlock_t. This
restores the pre-2a8bb29ec9b2 behavior of kvfree_call_rcu().

kfree_call_rcu_nolock(), added later in commit 3bc999d944b3
("mm/slab: introduce kfree_rcu_nolock()"), is untouched by this
patch. Note it would not be a safe substitute here either: it still
reaches __kfree_rcu_sheaf()'s local_trylock() on &s->cpu_sheaves->lock
unconditionally, so a caller already holding a raw_spinlock_t would
hit the same lockdep ordering cycle through that path too.

Reported-by: syzbot+acf142088e0182172e58@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=acf142088e0182172e58
Fixes: 2a8bb29ec9b2 ("mm/slab: allow kfree_rcu_sheaf() on PREEMPT_RT")
Signed-off-by: ThangNN99 <ngocthang2710.1999@gmail.com>
---
v3 (per Sebastian Andrzej Siewior's review on v2):
 - Say "raw_spinlock_t" instead of the vague "a raw spinlock"
   throughout the commit message and comment.
 - State plainly that *all* callers of set_cpus_allowed_force() hold
   task_struct::pi_lock, not just that it "can be called" with it
   held.

v2 (per automated review on v1):
 - Corrected commit message / comments: kfree_call_rcu_nolock() is not
   a safe alternative here either, since it still trylocks the same
   &s->cpu_sheaves->lock unconditionally.
 - Removed the now-unreachable CONFIG_PREEMPT_RT branch inside
   kfree_rcu_sheaf() left over by this fix (it can no longer run,
   since kvfree_call_rcu() already skips calling it on PREEMPT_RT).

 mm/slab_common.c | 20 ++++++++++----------
 mm/slub.c        |  6 +++---
 2 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/mm/slab_common.c b/mm/slab_common.c
index b19ba1b31484..015380ba8bcc 100644
--- a/mm/slab_common.c
+++ b/mm/slab_common.c
@@ -1667,15 +1667,8 @@ static bool kfree_rcu_sheaf(void *obj)
 {
 	struct kmem_cache *s;
 	struct slab *slab;
-	unsigned int free_flags = SLAB_FREE_DEFAULT;
-
-	/*
-	 * It is not safe to spin on PREEMPT_RT because the kernel might be
-	 * holding a raw spinlock and slab acquires sleeping locks.
-	 */
-	if (IS_ENABLED(CONFIG_PREEMPT_RT))
-		free_flags = SLAB_FREE_NOLOCK;
 
+	/* Callers on PREEMPT_RT never reach here, see kvfree_call_rcu(). */
 	if (is_vmalloc_addr(obj))
 		return false;
 
@@ -1685,7 +1678,7 @@ static bool kfree_rcu_sheaf(void *obj)
 
 	s = slab->slab_cache;
 	if (likely(!IS_ENABLED(CONFIG_NUMA) || slab_nid(slab) == numa_mem_id()))
-		return __kfree_rcu_sheaf(s, obj, free_flags);
+		return __kfree_rcu_sheaf(s, obj, SLAB_FREE_DEFAULT);
 
 	return false;
 }
@@ -2034,7 +2027,14 @@ void kvfree_call_rcu(struct kvfree_rcu_head *head, void *ptr)
 	if (!head)
 		might_sleep();
 
-	if (kfree_rcu_sheaf(ptr))
+	/*
+	 * Callers may hold a raw_spinlock_t here on PREEMPT_RT (e.g.
+	 * set_cpus_allowed_force(), whose callers all hold
+	 * task_struct::pi_lock), and the sheaf/barn locks are also taken
+	 * as blocking locks elsewhere, so trying them here creates a
+	 * lockdep-visible ordering conflict. Skip sheaves on PREEMPT_RT.
+	 */
+	if (!IS_ENABLED(CONFIG_PREEMPT_RT) && kfree_rcu_sheaf(ptr))
 		return;
 
 	// Queue the object but don't yet schedule the batch.
diff --git a/mm/slub.c b/mm/slub.c
index f9b56cb439e7..1e8bad7a018e 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -6088,10 +6088,10 @@ static void rcu_free_sheaf(struct rcu_head *head)
 /*
  * kvfree_call_rcu() can be called while holding a raw_spinlock_t. Since
  * __kfree_rcu_sheaf() may acquire a spinlock_t (sleeping lock on PREEMPT_RT),
- * this would violate lock nesting rules. Therefore, kvfree_call_rcu() avoids
- * this problem by passing SLAB_FREE_NOLOCK on PREEMPT_RT.
+ * this would violate lock nesting rules. kvfree_call_rcu() avoids this by
+ * bypassing the sheaves layer on PREEMPT_RT.
  *
- * However, lockdep still complains that it is invalid to acquire spinlock_t
+ * lockdep still complains that it is invalid to acquire spinlock_t
  * while holding raw_spinlock_t, even on !PREEMPT_RT where spinlock_t is a
  * spinning lock. Tell lockdep that acquiring spinlock_t is valid here
  * by temporarily raising the wait-type to LD_WAIT_CONFIG. Skip the lockdep map
-- 
2.43.0

Re: [PATCH v3] mm/slab: don't use kfree_rcu sheaves on PREEMPT_RT in kvfree_call_rcu()

[ThangNN99]

Thanks, both are pre-existing and out of scope for this patch:

- The GFP_NOWAIT/wakeup_kswapd() concern in __kfree_rcu_sheaf() applies
  identically before and after this patch on !PREEMPT_RT; this patch
  only changes the PREEMPT_RT branch. It also predates the commit this
  patch fixes: alloc_empty_sheaf() has used GFP_NOWAIT in this call
  chain since the sheaves feature was introduced in ec66e0d59952
  ("slab: add sheaf support for batching kfree_rcu() operations"), not
  since 2a8bb29ec9b2. It's a real question about the sheaf/barn
  allocation design itself, not something a small, targeted fix for
  the syzbot-reported splat should also try to redesign.

- Same as noted for the previous review: kfree_call_rcu_nolock() is
  intentionally left untouched by this patch. Agreed it doesn't fully
  deliver on its 'any context' contract on PREEMPT_RT either, for the
  reasons given (local_trylock ordering, and now also alloc_empty_sheaf()
  reaching zone->lock). That's a separate fix.

Both would be good candidates for their own patches from whoever
knows the sheaf/barn allocation path best; happy to help test if one
gets posted. This patch stays scoped to the specific circular-locking
splat in kvfree_call_rcu() reported by syzbot.

Thanks,
ThangNN99

Re: [PATCH v3] mm/slab: don't use kfree_rcu sheaves on PREEMPT_RT in kvfree_call_rcu()

[Sebastian Andrzej Siewior]

On 2026-08-31 20:32:22 [+0700], ThangNN99 wrote:
> --- a/mm/slab_common.c
> +++ b/mm/slab_common.c
> @@ -2034,7 +2027,14 @@ void kvfree_call_rcu(struct kvfree_rcu_head *head, void *ptr)
>  	if (!head)
>  		might_sleep();
>  
> -	if (kfree_rcu_sheaf(ptr))
> +	/*
> +	 * Callers may hold a raw_spinlock_t here on PREEMPT_RT (e.g.
> +	 * set_cpus_allowed_force(), whose callers all hold
> +	 * task_struct::pi_lock), and the sheaf/barn locks are also taken
> +	 * as blocking locks elsewhere, so trying them here creates a
> +	 * lockdep-visible ordering conflict. Skip sheaves on PREEMPT_RT.
> +	 */

You mix up things. A raw_spinlock_t should work in general and should
not cause a problem. The task_struct::pi_lock is special: It used during
wakes and the inner waitlock of the rtmutex acquires it even during a
trylock. For PREEMPT_RT we don't want to acquire any locks while the
pi_lock is held.

What about
  
    kvfree_rcu() is called by set_cpus_allowed_force() with
    task_struct::pi_lock acquired. On PREEMPT_RT the local_trylock()
    usage below will acquire the waitlock which must be avoided.
    Therefore avoid it on PREEMPT_RT.

Vlastimil?

> +	if (!IS_ENABLED(CONFIG_PREEMPT_RT) && kfree_rcu_sheaf(ptr))
>  		return;
>  
>  	// Queue the object but don't yet schedule the batch.

Sebastian

Re: [PATCH v3] mm/slab: don't use kfree_rcu sheaves on PREEMPT_RT in kvfree_call_rcu()

[Vlastimil Babka (SUSE)]

On 8/31/26 16:35, Sebastian Andrzej Siewior wrote:
> On 2026-08-31 20:32:22 [+0700], ThangNN99 wrote:
>> --- a/mm/slab_common.c
>> +++ b/mm/slab_common.c
>> @@ -2034,7 +2027,14 @@ void kvfree_call_rcu(struct kvfree_rcu_head *head, void *ptr)
>>  	if (!head)
>>  		might_sleep();
>>  
>> -	if (kfree_rcu_sheaf(ptr))
>> +	/*
>> +	 * Callers may hold a raw_spinlock_t here on PREEMPT_RT (e.g.
>> +	 * set_cpus_allowed_force(), whose callers all hold
>> +	 * task_struct::pi_lock), and the sheaf/barn locks are also taken
>> +	 * as blocking locks elsewhere, so trying them here creates a
>> +	 * lockdep-visible ordering conflict. Skip sheaves on PREEMPT_RT.
>> +	 */
> 
> You mix up things. A raw_spinlock_t should work in general and should
> not cause a problem. The task_struct::pi_lock is special: It used during
> wakes and the inner waitlock of the rtmutex acquires it even during a
> trylock. For PREEMPT_RT we don't want to acquire any locks while the
> pi_lock is held.
> 
> What about
>   
>     kvfree_rcu() is called by set_cpus_allowed_force() with
>     task_struct::pi_lock acquired. On PREEMPT_RT the local_trylock()
>     usage below will acquire the waitlock which must be avoided.
>     Therefore avoid it on PREEMPT_RT.

Thanks.

> Vlastimil?

IMHO it's not useful to iterate a fix through emails with apparently a LLM
agent, so let's continue here:

https://lore.kernel.org/all/20260831-b4-kfree_rcu_hotfix-v1-1-4f0fb882638b@kernel.org/

> 
>> +	if (!IS_ENABLED(CONFIG_PREEMPT_RT) && kfree_rcu_sheaf(ptr))
>>  		return;
>>  
>>  	// Queue the object but don't yet schedule the batch.
> 
> Sebastian

Re: [PATCH v3] mm/slab: don't use kfree_rcu sheaves on PREEMPT_RT in kvfree_call_rcu()

[Harry Yoo]

On Mon, Aug 31, 2026 at 04:35:00PM +0200, Sebastian Andrzej Siewior wrote:
> On 2026-08-31 20:32:22 [+0700], ThangNN99 wrote:
> > --- a/mm/slab_common.c
> > +++ b/mm/slab_common.c
> > @@ -2034,7 +2027,14 @@ void kvfree_call_rcu(struct kvfree_rcu_head *head, void *ptr)
> >  	if (!head)
> >  		might_sleep();
> >  
> > -	if (kfree_rcu_sheaf(ptr))
> > +	/*
> > +	 * Callers may hold a raw_spinlock_t here on PREEMPT_RT (e.g.
> > +	 * set_cpus_allowed_force(), whose callers all hold
> > +	 * task_struct::pi_lock), and the sheaf/barn locks are also taken
> > +	 * as blocking locks elsewhere, so trying them here creates a
> > +	 * lockdep-visible ordering conflict. Skip sheaves on PREEMPT_RT.
> > +	 */
> 
> You mix up things. A raw_spinlock_t should work in general and should
> not cause a problem. The task_struct::pi_lock is special: It used during
> wakes and the inner waitlock of the rtmutex acquires it even during a
> trylock. For PREEMPT_RT we don't want to acquire any locks while the
> pi_lock is held.

So... even can_spin_trylock() is not enough handle the pi_lock's
special case.

Probably we should make can_spin_trylock() check if current tasks's
pi_lock is acquired, to determine if it's safe to call _trylock()?

> What about
>   
>     kvfree_rcu() is called by set_cpus_allowed_force() with
>     task_struct::pi_lock acquired. On PREEMPT_RT the local_trylock()
>     usage below will acquire the waitlock which must be avoided.
>     Therefore avoid it on PREEMPT_RT.

-- 
Cheers,
Harry / Hyeonggon