[PATCH] locking/mutex: Restore RCU read-side protection for optimistic spinning

[PATCH] locking/mutex: Restore RCU read-side protection for optimistic spinning

[Yang Zi]

The optimistic spinning paths in kernel/locking/mutex.c dereference the
current lock owner's task_struct (via owner_on_cpu() -> owner->on_cpu and
task_cpu(owner)) without any lifetime protection on the owner pointer.

__mutex_owner() (kernel/locking/mutex.h) returns a bare task_struct
pointer that is read directly from lock->owner. mutex_spin_on_owner() and
mutex_can_spin_on_owner() then dereference it inside "while
(__mutex_owner(lock) == owner)" / immediately after the owner load, while
relying only on preempt_disable().

Since commit 6c2787f2a20c ("locking: Remove rcu_read_{,un}lock() for
preempt_{dis,en}able()"), the code has assumed that preempt_disable() is
equivalent to an RCU read-side critical section and therefore that the
owner's task_struct "won't go away during the spinning period". That
assumption does not hold on PREEMPT_LAZY kernels: preempt_disable() there
no longer actually disables preemption, so the RCU callback that frees a
just-exited owner's task_struct (put_task_struct_rcu_user() ->
call_rcu(), run in rcu_do_batch()) can fire and free the object while a
spinner still holds the stale pointer, leading to a slab use-after-free
read of owner->on_cpu.

This manifests in three independent syzkaller/KASAN reports with the same
root cause:

- BUG 123 (dw_edma_pcie, v7.1, PREEMPT(lazy)): finit_module ->
  driver_register -> bus_add_driver -> bus_for_each_dev -> __driver_attach
  -> device_lock -> __mutex_lock -> mutex_optimistic_spin ->
  mutex_can_spin_on_owner -> owner_on_cpu.

- BUG 132 (bna): read(2) -> uevent_show -> device_lock ->
  mutex_optimistic_spin -> mutex_can_spin_on_owner -> owner_on_cpu.

- BUG 208 (iavf, v7.1, PREEMPT(lazy)): delete_module ->
  pci_unregister_driver -> ... -> iavf_remove -> netdev_lock ->
  __mutex_lock -> mutex_optimistic_spin -> mutex_can_spin_on_owner ->
  owner_on_cpu.

Fix it by taking an explicit rcu_read_lock()/rcu_read_unlock() around the
owner dereference in both mutex_spin_on_owner() and mutex_can_spin_on_owner(),
restoring the pre-5.16 protection, and update the now-inaccurate comments.

Signed-off-by: Yang Zi <2959243019@qq.com>
---
diff --git a/kernel/locking/mutex.c b/kernel/locking/mutex.c
index 942a939cee95..cc2d7090bf36 100644
--- a/kernel/locking/mutex.c
+++ b/kernel/locking/mutex.c
@@ -389,14 +389,21 @@ bool mutex_spin_on_owner(struct mutex *lock, struct task_struct *owner,
 
     lockdep_assert_preemption_disabled();
 
+    /*
+     * Optimistic spinners run with preempt_disable() which, on classic
+     * PREEMPT_RCU, was enough to act as an RCU read-side critical
+     * section. On PREEMPT_LAZY kernels preempt_disable() no longer
+     * prevents RCU callbacks (e.g. the free of an exiting owner's
+     * task_struct via call_rcu() in rcu_do_batch()) from running, so take
+     * an explicit RCU read-side critical section to keep the owner alive.
+     */
+    rcu_read_lock();
     while (__mutex_owner(lock) == owner) {
         /*
-         * Ensure we emit the owner->on_cpu, dereference _after_
-         * checking lock->owner still matches owner. And we already
-         * disabled preemption which is equal to the RCU read-side
-         * crital section in optimistic spinning code. Thus the
-         * task_strcut structure won't go away during the spinning
-         * period
+         * Ensure we emit the owner->on_cpu dereference _after_
+         * checking lock->owner still matches owner. If that fails,
+         * owner might point to freed memory. If it still matches,
+         * the rcu_read_lock() ensures the memory stays valid.
          */
         barrier();
 
@@ -415,6 +422,7 @@ bool mutex_spin_on_owner(struct mutex *lock, struct task_struct *owner,
 
         cpu_relax();
     }
+    rcu_read_unlock();
 
     return ret;
 }
@@ -433,13 +441,16 @@ static inline int mutex_can_spin_on_owner(struct mutex *lock)
         return 0;
 
     /*
-     * We already disabled preemption which is equal to the RCU read-side
-     * crital section in optimistic spinning code. Thus the task_strcut
-     * structure won't go away during the spinning period.
+     * See the comment in mutex_spin_on_owner(): preempt_disable() is no
+     * longer an RCU read-side critical section on PREEMPT_LAZY kernels,
+     * so protect the owner dereference with an explicit RCU read-side
+     * critical section.
      */
+    rcu_read_lock();
     owner = __mutex_owner(lock);
     if (owner)
         retval = owner_on_cpu(owner);
+    rcu_read_unlock();
 
     /*
      * If lock->owner is not set, the mutex has been released. Return true

[PATCH] locking/mutex: Restore RCU read-side protection for optimistic spinning

[Yang Zi]

The optimistic spinning paths in kernel/locking/mutex.c dereference the
current lock owner's task_struct (via owner_on_cpu() -> owner->on_cpu and
task_cpu(owner)) without any lifetime protection on the owner pointer.

__mutex_owner() (kernel/locking/mutex.h) returns a bare task_struct
pointer that is read directly from lock->owner. mutex_spin_on_owner() and
mutex_can_spin_on_owner() then dereference it inside "while
(__mutex_owner(lock) == owner)" / immediately after the owner load, while
relying only on preempt_disable().

Since commit 6c2787f2a20c ("locking: Remove rcu_read_{,un}lock() for
preempt_{dis,en}able()"), the code has assumed that preempt_disable() is
equivalent to an RCU read-side critical section and therefore that the
owner's task_struct "won't go away during the spinning period". That
assumption does not hold on PREEMPT_LAZY kernels: preempt_disable() there
no longer actually disables preemption, so the RCU callback that frees a
just-exited owner's task_struct (put_task_struct_rcu_user() ->
call_rcu(), run in rcu_do_batch()) can fire and free the object while a
spinner still holds the stale pointer, leading to a slab use-after-free
read of owner->on_cpu.

This manifests in three independent syzkaller/KASAN reports with the same
root cause:

- BUG 123 (dw_edma_pcie, v7.1, PREEMPT(lazy)): finit_module ->
  driver_register -> bus_add_driver -> bus_for_each_dev -> __driver_attach
  -> device_lock -> __mutex_lock -> mutex_optimistic_spin ->
  mutex_can_spin_on_owner -> owner_on_cpu.

- BUG 132 (bna): read(2) -> uevent_show -> device_lock ->
  mutex_optimistic_spin -> mutex_can_spin_on_owner -> owner_on_cpu.

- BUG 208 (iavf, v7.1, PREEMPT(lazy)): delete_module ->
  pci_unregister_driver -> ... -> iavf_remove -> netdev_lock ->
  __mutex_lock -> mutex_optimistic_spin -> mutex_can_spin_on_owner ->
  owner_on_cpu.

Fix it by taking an explicit rcu_read_lock()/rcu_read_unlock() around the
owner dereference in both mutex_spin_on_owner() and mutex_can_spin_on_owner(),
restoring the pre-5.16 protection, and update the now-inaccurate comments.

Signed-off-by: Yang Zi <2959243019@qq.com>
---
diff --git a/kernel/locking/mutex.c b/kernel/locking/mutex.c
index 942a939cee95..cc2d7090bf36 100644
--- a/kernel/locking/mutex.c
+++ b/kernel/locking/mutex.c
@@ -389,14 +389,21 @@ bool mutex_spin_on_owner(struct mutex *lock, struct task_struct *owner,
 
     lockdep_assert_preemption_disabled();
 
+    /*
+     * Optimistic spinners run with preempt_disable() which, on classic
+     * PREEMPT_RCU, was enough to act as an RCU read-side critical
+     * section. On PREEMPT_LAZY kernels preempt_disable() no longer
+     * prevents RCU callbacks (e.g. the free of an exiting owner's
+     * task_struct via call_rcu() in rcu_do_batch()) from running, so take
+     * an explicit RCU read-side critical section to keep the owner alive.
+     */
+    rcu_read_lock();
     while (__mutex_owner(lock) == owner) {
         /*
-         * Ensure we emit the owner->on_cpu, dereference _after_
-         * checking lock->owner still matches owner. And we already
-         * disabled preemption which is equal to the RCU read-side
-         * crital section in optimistic spinning code. Thus the
-         * task_strcut structure won't go away during the spinning
-         * period
+         * Ensure we emit the owner->on_cpu dereference _after_
+         * checking lock->owner still matches owner. If that fails,
+         * owner might point to freed memory. If it still matches,
+         * the rcu_read_lock() ensures the memory stays valid.
          */
         barrier();
 
@@ -415,6 +422,7 @@ bool mutex_spin_on_owner(struct mutex *lock, struct task_struct *owner,
 
         cpu_relax();
     }
+    rcu_read_unlock();
 
     return ret;
 }
@@ -433,13 +441,16 @@ static inline int mutex_can_spin_on_owner(struct mutex *lock)
         return 0;
 
     /*
-     * We already disabled preemption which is equal to the RCU read-side
-     * crital section in optimistic spinning code. Thus the task_strcut
-     * structure won't go away during the spinning period.
+     * See the comment in mutex_spin_on_owner(): preempt_disable() is no
+     * longer an RCU read-side critical section on PREEMPT_LAZY kernels,
+     * so protect the owner dereference with an explicit RCU read-side
+     * critical section.
      */
+    rcu_read_lock();
     owner = __mutex_owner(lock);
     if (owner)
         retval = owner_on_cpu(owner);
+    rcu_read_unlock();
 
     /*
      * If lock->owner is not set, the mutex has been released. Return true

[PATCH] locking/mutex: Restore RCU read-side protection for optimistic spinning

[Yang Zi]

The optimistic spinning paths in kernel/locking/mutex.c dereference the
current lock owner's task_struct (via owner_on_cpu() -> owner->on_cpu and
task_cpu(owner)) without any lifetime protection on the owner pointer.

__mutex_owner() (kernel/locking/mutex.h) returns a bare task_struct
pointer that is read directly from lock->owner. mutex_spin_on_owner() and
mutex_can_spin_on_owner() then dereference it inside "while
(__mutex_owner(lock) == owner)" / immediately after the owner load, while
relying only on preempt_disable().

Fix it by taking an explicit rcu_read_lock()/rcu_read_unlock() around the
owner dereference in both mutex_spin_on_owner() and mutex_can_spin_on_owner(),
restoring the pre-5.16 protection, and update the now-inaccurate comments.

Signed-off-by: Yang Zi <2959243019@qq.com>
---
diff --git a/kernel/locking/mutex.c b/kernel/locking/mutex.c
index 942a939cee95..cc2d7090bf36 100644
--- a/kernel/locking/mutex.c
+++ b/kernel/locking/mutex.c
@@ -389,14 +389,21 @@ bool mutex_spin_on_owner(struct mutex *lock, struct task_struct *owner,
 
     lockdep_assert_preemption_disabled();
 
+    /*
+     * Optimistic spinners run with preempt_disable() which, on classic
+     * PREEMPT_RCU, was enough to act as an RCU read-side critical
+     * section. On PREEMPT_LAZY kernels preempt_disable() no longer
+     * prevents RCU callbacks (e.g. the free of an exiting owner's
+     * task_struct via call_rcu() in rcu_do_batch()) from running, so take
+     * an explicit RCU read-side critical section to keep the owner alive.
+     */
+    rcu_read_lock();
     while (__mutex_owner(lock) == owner) {
         /*
-         * Ensure we emit the owner->on_cpu, dereference _after_
-         * checking lock->owner still matches owner. And we already
-         * disabled preemption which is equal to the RCU read-side
-         * crital section in optimistic spinning code. Thus the
-         * task_strcut structure won't go away during the spinning
-         * period
+         * Ensure we emit the owner->on_cpu dereference _after_
+         * checking lock->owner still matches owner. If that fails,
+         * owner might point to freed memory. If it still matches,
+         * the rcu_read_lock() ensures the memory stays valid.
          */
         barrier();
 
@@ -415,6 +422,7 @@ bool mutex_spin_on_owner(struct mutex *lock, struct task_struct *owner,
 
         cpu_relax();
     }
+    rcu_read_unlock();
 
     return ret;
 }
@@ -433,13 +441,16 @@ static inline int mutex_can_spin_on_owner(struct mutex *lock)
         return 0;
 
     /*
-     * We already disabled preemption which is equal to the RCU read-side
-     * crital section in optimistic spinning code. Thus the task_strcut
-     * structure won't go away during the spinning period.
+     * See the comment in mutex_spin_on_owner(): preempt_disable() is no
+     * longer an RCU read-side critical section on PREEMPT_LAZY kernels,
+     * so protect the owner dereference with an explicit RCU read-side
+     * critical section.
      */
+    rcu_read_lock();
     owner = __mutex_owner(lock);
     if (owner)
         retval = owner_on_cpu(owner);
+    rcu_read_unlock();
 
     /*
      * If lock->owner is not set, the mutex has been released. Return true

Re: [PATCH] locking/mutex: Restore RCU read-side protection for optimistic spinning

[Peter Zijlstra]

On Tue, Aug 25, 2026 at 05:13:30PM +0800, Yang Zi wrote:
> The optimistic spinning paths in kernel/locking/mutex.c dereference the
> current lock owner's task_struct (via owner_on_cpu() -> owner->on_cpu and
> task_cpu(owner)) without any lifetime protection on the owner pointer.
> 
> __mutex_owner() (kernel/locking/mutex.h) returns a bare task_struct
> pointer that is read directly from lock->owner. mutex_spin_on_owner() and
> mutex_can_spin_on_owner() then dereference it inside "while
> (__mutex_owner(lock) == owner)" / immediately after the owner load, while
> relying only on preempt_disable().
> 
> Since commit 6c2787f2a20c ("locking: Remove rcu_read_{,un}lock() for
> preempt_{dis,en}able()"), the code has assumed that preempt_disable() is
> equivalent to an RCU read-side critical section and therefore that the
> owner's task_struct "won't go away during the spinning period". That
> assumption does not hold on PREEMPT_LAZY kernels: preempt_disable() there
> no longer actually disables preemption, so the RCU callback that frees a
> just-exited owner's task_struct (put_task_struct_rcu_user() ->
> call_rcu(), run in rcu_do_batch()) can fire and free the object while a
> spinner still holds the stale pointer, leading to a slab use-after-free
> read of owner->on_cpu.

This doesn't make sense, rcu_preempt was subsumed (again) into regular
rcu proper (along with rcu_bh). Any preempt disable region will prohibit
the GP from advancing.