[PATCH 0/2] futex/requeue: Fix requeue PI races

[PATCH 0/2] futex/requeue: Fix requeue PI races

[Yao Kai]

Hi,

This series fixes two independent issues in the requeue PI wait path.
They are grouped because both affect the handoff between the futex
requeue state machine and the target PI rtmutex.

Patch 1 restores rtmutex scheduling preparation when a waiter is moved to
the target PI futex before futex_do_wait() can schedule. It splits block
plug flushing from worker notification so that each operation runs at a
safe point.

Patch 2 prevents the requeue side from accessing the waiter's stack-based
futex_q after publishing Q_REQUEUE_PI_LOCKED. The waiter is instead woken
through the task pointer saved before publishing completion.

Both patches carry Fixes tags and are marked for stable backporting.

Build testing:

  - arm64 defconfig with W=1
  - arm64 PREEMPT_RT with KASAN_GENERIC, PROVE_LOCKING,
    DEBUG_RT_MUTEXES, and W=1
  - kernel/futex/requeue.o and kernel/sched/core.o built successfully
  - all patches pass checkpatch.pl --strict

Runtime testing:

  - Patch 1: the original rt_mutex_schedule() warning reproducer no longer
    triggers the warning
  - Patch 2: the original PREEMPT_RT KASAN reproducer no longer reports an
    invalid access
  - The futex requeue PI selftests and stress tests pass

Yao Kai (2):
  futex/requeue: Fix rtmutex schedule preparation for requeue PI
  futex/requeue: Prevent rcuwait use-after-free during requeue PI

 include/linux/sched/rt.h |  2 ++
 kernel/futex/requeue.c   | 24 ++++++++++++++++++---
 kernel/sched/core.c      | 45 +++++++++++++++++++++++++++++++++-------
 3 files changed, 60 insertions(+), 11 deletions(-)

-- 
2.43.0

[PATCH 1/2] futex/requeue: Fix rtmutex schedule preparation for requeue PI

[Yao Kai]

A waiter requeued onto a PI futex can reach
rt_mutex_wait_proxy_lock() without rtmutex schedule preparation:

  WARNING: CPU: 0 PID: 293 at kernel/sched/core.c:7606
  RIP: rt_mutex_schedule+0x43/0x50
  Call Trace:
   rt_mutex_slowlock_block.constprop.0+0x5b/0x320
   rt_mutex_wait_proxy_lock+0x3e/0x80
   futex_wait_requeue_pi+0x3ba/0x590
   do_futex+0x171/0x1f0

rt_mutex_schedule() requires current->sched_rt_mutex to be set. Normally,
rt_mutex_pre_schedule() sets it and submits pending work before the waiter
is enqueued. With requeue PI, another task can enqueue the waiter after its
futex_q becomes visible:

        waiter                          requeue task
        ------                          ------------
futex_wait_requeue_pi()
  futex_wait_setup()
    futex_queue(&q)
                                        futex_requeue()
                                          rt_mutex_start_proxy_lock()
                                            enqueue rt_waiter
                                            install pi_blocked_on
                                          requeue_futex()
                                            plist_del(&q->list)
  futex_do_wait()
    plist_node_empty(&q->list)
    skip schedule()
                                            plist_add(&q->list)
                                          futex_requeue_pi_complete()
                                            IN_PROGRESS -> DONE
  futex_requeue_pi_wakeup_sync() // DONE
  rt_mutex_wait_proxy_lock()
    rt_mutex_schedule()

futex_do_wait() mistakes the temporary removal for a wakeup and skips
schedule(). The waiter consequently enters rt_mutex_schedule() with
current->sched_rt_mutex clear. Its block plug remains unflushed, and
workqueue or io-wq users are not notified that the worker is going to
sleep.

rt_mutex_pre_schedule() cannot be used directly. Calling it before q is
published would set current->sched_rt_mutex while futex_do_wait() can still
call regular schedule(). Calling it after DONE would flush the block plug
after current->pi_blocked_on is installed, which can recurse into rtmutex
waiter setup.

Split the preparation instead. Flush the plug before futex_wait_setup()
publishes q. After futex_requeue_pi_wakeup_sync() returns DONE, enter the
rtmutex scheduling state and notify workqueue and io-wq users without
flushing the plug again. Split sched_submit_work() to support this ordering
and leave the scheduling state after the waiter has acquired the lock or
has been removed.

Fixes: d14f9e930b90 ("locking/rtmutex: Use rt_mutex specific scheduler helpers")
Cc: stable@vger.kernel.org
Signed-off-by: Yao Kai <yaokai34@huawei.com>
---
 include/linux/sched/rt.h |  2 ++
 kernel/futex/requeue.c   | 15 +++++++++++++-
 kernel/sched/core.c      | 45 +++++++++++++++++++++++++++++++++-------
 3 files changed, 53 insertions(+), 9 deletions(-)

diff --git a/include/linux/sched/rt.h b/include/linux/sched/rt.h
index 4e3338103654..3d8cbdbfe14b 100644
--- a/include/linux/sched/rt.h
+++ b/include/linux/sched/rt.h
@@ -51,7 +51,9 @@ static inline bool rt_or_dl_task_policy(struct task_struct *tsk)
 }
 
 #ifdef CONFIG_RT_MUTEXES
+void rt_mutex_pre_schedule_flush_plug(void);
 extern void rt_mutex_pre_schedule(void);
+void rt_mutex_pre_schedule_pi_blocked(void);
 extern void rt_mutex_schedule(void);
 extern void rt_mutex_post_schedule(void);
 
diff --git a/kernel/futex/requeue.c b/kernel/futex/requeue.c
index 79823ad13683..abc652b5b2dd 100644
--- a/kernel/futex/requeue.c
+++ b/kernel/futex/requeue.c
@@ -1,6 +1,7 @@
 // SPDX-License-Identifier: GPL-2.0-or-later
 
 #include <linux/plist.h>
+#include <linux/sched/rt.h>
 #include <linux/sched/signal.h>
 
 #include "futex.h"
@@ -820,6 +821,13 @@ int futex_wait_requeue_pi(u32 __user *uaddr, unsigned int flags,
 	q.rt_waiter = &rt_waiter;
 	q.requeue_pi_key = &key2;
 
+	/*
+	 * Once q is visible, requeue can enqueue rt_waiter and install
+	 * current->pi_blocked_on on our behalf. Flush plugged I/O before
+	 * that can happen.
+	 */
+	rt_mutex_pre_schedule_flush_plug();
+
 	/*
 	 * Prepare to wait on uaddr. On success, it holds hb->lock and q
 	 * is initialized.
@@ -865,6 +873,11 @@ int futex_wait_requeue_pi(u32 __user *uaddr, unsigned int flags,
 	case Q_REQUEUE_PI_DONE:
 		/* Requeue completed. Current is 'pi_blocked_on' the rtmutex */
 		pi_mutex = &q.pi_state->pi_mutex;
+		/*
+		 * The rt_waiter is already enqueued and the plug is flushed.
+		 * Enter schedule state and notify worker users.
+		 */
+		rt_mutex_pre_schedule_pi_blocked();
 		ret = rt_mutex_wait_proxy_lock(pi_mutex, to, &rt_waiter);
 
 		/*
@@ -875,6 +888,7 @@ int futex_wait_requeue_pi(u32 __user *uaddr, unsigned int flags,
 
 		futex_q_lockptr_lock(&q);
 		debug_rt_mutex_free_waiter(&rt_waiter);
+		rt_mutex_post_schedule();
 		/*
 		 * Fixup the pi_state owner and possibly acquire the lock if we
 		 * haven't already.
@@ -915,4 +929,3 @@ int futex_wait_requeue_pi(u32 __user *uaddr, unsigned int flags,
 	}
 	return ret;
 }
-
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 96226707c2f6..f481e49e16b8 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -7257,17 +7257,12 @@ void __noreturn do_task_dead(void)
 		cpu_relax();
 }
 
-static inline void sched_submit_work(struct task_struct *tsk)
+static DEFINE_WAIT_OVERRIDE_MAP(sched_map, LD_WAIT_CONFIG);
+
+static inline void sched_submit_work_notifiers(struct task_struct *tsk)
 {
-	static DEFINE_WAIT_OVERRIDE_MAP(sched_map, LD_WAIT_CONFIG);
 	unsigned int task_flags;
 
-	/*
-	 * Establish LD_WAIT_CONFIG context to ensure none of the code called
-	 * will use a blocking primitive -- which would lead to recursion.
-	 */
-	lock_map_acquire_try(&sched_map);
-
 	task_flags = tsk->flags;
 	/*
 	 * If a worker goes to sleep, notify and ask workqueue whether it
@@ -7277,7 +7272,10 @@ static inline void sched_submit_work(struct task_struct *tsk)
 		wq_worker_sleeping(tsk);
 	else if (task_flags & PF_IO_WORKER)
 		io_wq_worker_sleeping(tsk);
+}
 
+static inline void sched_submit_work_plug(struct task_struct *tsk)
+{
 	/*
 	 * spinlock and rwlock must not flush block requests.  This will
 	 * deadlock if the callback attempts to acquire a lock which is
@@ -7290,6 +7288,18 @@ static inline void sched_submit_work(struct task_struct *tsk)
 	 * make sure to submit it to avoid deadlocks.
 	 */
 	blk_flush_plug(tsk->plug, true);
+}
+
+static inline void sched_submit_work(struct task_struct *tsk)
+{
+	/*
+	 * Establish LD_WAIT_CONFIG context to ensure none of the code called
+	 * will use a blocking primitive -- which would lead to recursion.
+	 */
+	lock_map_acquire_try(&sched_map);
+
+	sched_submit_work_notifiers(tsk);
+	sched_submit_work_plug(tsk);
 
 	lock_map_release(&sched_map);
 }
@@ -7595,12 +7605,31 @@ const struct sched_class *__setscheduler_class(int policy, int prio)
  */
 #define fetch_and_set(x, v) ({ int _x = (x); (x) = (v); _x; })
 
+void rt_mutex_pre_schedule_flush_plug(void)
+{
+	lockdep_assert(!current->pi_blocked_on);
+	lock_map_acquire_try(&sched_map);
+	sched_submit_work_plug(current);
+	lock_map_release(&sched_map);
+}
+
 void rt_mutex_pre_schedule(void)
 {
 	lockdep_assert(!fetch_and_set(current->sched_rt_mutex, 1));
 	sched_submit_work(current);
 }
 
+void rt_mutex_pre_schedule_pi_blocked(void)
+{
+	lockdep_assert(current->pi_blocked_on);
+	lockdep_assert(!fetch_and_set(current->sched_rt_mutex, 1));
+
+	/* The plug was flushed before current->pi_blocked_on was installed. */
+	lock_map_acquire_try(&sched_map);
+	sched_submit_work_notifiers(current);
+	lock_map_release(&sched_map);
+}
+
 void rt_mutex_schedule(void)
 {
 	lockdep_assert(current->sched_rt_mutex);
-- 
2.43.0

[PATCH 2/2] futex/requeue: Prevent rcuwait use-after-free during requeue PI

[Yao Kai]

On PREEMPT_RT, FUTEX_CMP_REQUEUE_PI can trigger a KASAN report:

  BUG: KASAN: slab-out-of-bounds in _raw_spin_lock_irqsave+0x76/0xe0
  Call Trace:
   _raw_spin_lock_irqsave+0x76/0xe0
   try_to_wake_up+0xab/0x1540
   rcuwait_wake_up+0x39/0x60
   futex_requeue+0x18c3/0x1e10

The futex_q used by futex_wait_requeue_pi() is allocated on the waiter's
stack. An early wakeup can race with a PI requeue as follows:

        waiter                          requeue task
        ------                          ------------
futex_wait_requeue_pi()
  futex_do_wait()
    schedule()

        * timeout/signal wakes waiter *

  futex_requeue_pi_wakeup_sync()
    IN_PROGRESS -> WAIT
    rcuwait_wait_event()
                                        requeue_pi_wake_futex()
                                          task = READ_ONCE(q->task)
                                          futex_requeue_pi_complete()
                                            WAIT -> LOCKED
    return LOCKED
  return
// q lifetime ends
                                            rcuwait_wake_up()

futex_requeue_pi_complete() publishes LOCKED before calling
rcuwait_wake_up(). Once the waiter observes LOCKED, it can return from
futex_wait_requeue_pi() and let q go out of scope before rcuwait_wake_up()
reads q->requeue_wait.task and passes the stale pointer to
try_to_wake_up().

Skip rcuwait_wake_up() for Q_REQUEUE_PI_LOCKED. requeue_pi_wake_futex()
already saves q->task before publishing LOCKED and wakes the saved task
afterward.

Fixes: 07d91ef510fb1 ("futex: Prevent requeue_pi() lock nesting issue on RT")
Cc: stable@vger.kernel.org
Signed-off-by: Yao Kai <yaokai34@huawei.com>
---
 kernel/futex/requeue.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/kernel/futex/requeue.c b/kernel/futex/requeue.c
index abc652b5b2dd..59e587775d9b 100644
--- a/kernel/futex/requeue.c
+++ b/kernel/futex/requeue.c
@@ -155,8 +155,13 @@ static inline void futex_requeue_pi_complete(struct futex_q *q, int locked)
 	} while (!atomic_try_cmpxchg(&q->requeue_state, &old, new));
 
 #ifdef CONFIG_PREEMPT_RT
-	/* If the waiter interleaved with the requeue let it know */
-	if (unlikely(old == Q_REQUEUE_PI_WAIT))
+	/*
+	 * If the waiter interleaved with the requeue, let it know. For LOCKED,
+	 * q may already be invalid, so requeue_pi_wake_futex() wakes the saved
+	 * task instead.
+	 */
+	if (unlikely(old == Q_REQUEUE_PI_WAIT) &&
+	    new != Q_REQUEUE_PI_LOCKED)
 		rcuwait_wake_up(&q->requeue_wait);
 #endif
 }
-- 
2.43.0

Re: [PATCH 1/2] futex/requeue: Fix rtmutex schedule preparation for requeue PI

[Sebastian Andrzej Siewior]

On 2026-07-17 16:49:21 [+0800], Yao Kai wrote:
> A waiter requeued onto a PI futex can reach
> rt_mutex_wait_proxy_lock() without rtmutex schedule preparation:

>   WARNING: CPU: 0 PID: 293 at kernel/sched/core.c:7606
>   RIP: rt_mutex_schedule+0x43/0x50
>   Call Trace:
>    rt_mutex_slowlock_block.constprop.0+0x5b/0x320
>    rt_mutex_wait_proxy_lock+0x3e/0x80
>    futex_wait_requeue_pi+0x3ba/0x590
>    do_futex+0x171/0x1f0
> 
> rt_mutex_schedule() requires current->sched_rt_mutex to be set. Normally,
> rt_mutex_pre_schedule() sets it and submits pending work before the waiter
> is enqueued. With requeue PI, another task can enqueue the waiter after its
> futex_q becomes visible:
> 
>         waiter                          requeue task
>         ------                          ------------
> futex_wait_requeue_pi()
>   futex_wait_setup()
>     futex_queue(&q)
>                                         futex_requeue()
>                                           rt_mutex_start_proxy_lock()
>                                             enqueue rt_waiter
>                                             install pi_blocked_on
>                                           requeue_futex()
>                                             plist_del(&q->list)
>   futex_do_wait()
>     plist_node_empty(&q->list)
>     skip schedule()
>                                             plist_add(&q->list)
>                                           futex_requeue_pi_complete()
>                                             IN_PROGRESS -> DONE
>   futex_requeue_pi_wakeup_sync() // DONE
>   rt_mutex_wait_proxy_lock()
>     rt_mutex_schedule()
> 
> futex_do_wait() mistakes the temporary removal for a wakeup and skips
> schedule(). The waiter consequently enters rt_mutex_schedule() with
> current->sched_rt_mutex clear. Its block plug remains unflushed, and
> workqueue or io-wq users are not notified that the worker is going to
> sleep.

This has nothing to do with, does it? It is just usually the lock is
acquired and it is not blocked on. Do you have testcase for this or is
this just audit?

> rt_mutex_pre_schedule() cannot be used directly. Calling it before q is
> published would set current->sched_rt_mutex while futex_do_wait() can still
> call regular schedule(). Calling it after DONE would flush the block plug
> after current->pi_blocked_on is installed, which can recurse into rtmutex
> waiter setup.
> 
> Split the preparation instead. Flush the plug before futex_wait_setup()
> publishes q. After futex_requeue_pi_wakeup_sync() returns DONE, enter the
> rtmutex scheduling state and notify workqueue and io-wq users without
> flushing the plug again. Split sched_submit_work() to support this ordering
> and leave the scheduling state after the waiter has acquired the lock or
> has been removed.
> 
> Fixes: d14f9e930b90 ("locking/rtmutex: Use rt_mutex specific scheduler helpers")
> Cc: stable@vger.kernel.org
> Signed-off-by: Yao Kai <yaokai34@huawei.com>

what about the following? This might compile but lacks all kind of testing.

diff --git a/kernel/futex/requeue.c b/kernel/futex/requeue.c
index 79823ad136830..42a04e6e774c4 100644
--- a/kernel/futex/requeue.c
+++ b/kernel/futex/requeue.c
@@ -865,7 +865,9 @@ int futex_wait_requeue_pi(u32 __user *uaddr, unsigned int flags,
 	case Q_REQUEUE_PI_DONE:
 		/* Requeue completed. Current is 'pi_blocked_on' the rtmutex */
 		pi_mutex = &q.pi_state->pi_mutex;
+		rt_mutex_pre_schedule();
 		ret = rt_mutex_wait_proxy_lock(pi_mutex, to, &rt_waiter);
+		rt_mutex_post_schedule();
 
 		/*
 		 * See futex_unlock_pi()'s cleanup: comment.

Re: [PATCH 2/2] futex/requeue: Prevent rcuwait use-after-free during requeue PI

[Sebastian Andrzej Siewior]

On 2026-07-17 16:49:22 [+0800], Yao Kai wrote:
> On PREEMPT_RT, FUTEX_CMP_REQUEUE_PI can trigger a KASAN report:
> 
>   BUG: KASAN: slab-out-of-bounds in _raw_spin_lock_irqsave+0x76/0xe0
>   Call Trace:
>    _raw_spin_lock_irqsave+0x76/0xe0
>    try_to_wake_up+0xab/0x1540
>    rcuwait_wake_up+0x39/0x60
>    futex_requeue+0x18c3/0x1e10
> 
> The futex_q used by futex_wait_requeue_pi() is allocated on the waiter's
> stack. An early wakeup can race with a PI requeue as follows:
> 
>         waiter                          requeue task
>         ------                          ------------
> futex_wait_requeue_pi()
>   futex_do_wait()
>     schedule()
> 
>         * timeout/signal wakes waiter *
> 
>   futex_requeue_pi_wakeup_sync()
>     IN_PROGRESS -> WAIT
>     rcuwait_wait_event()
>                                         requeue_pi_wake_futex()
>                                           task = READ_ONCE(q->task)
>                                           futex_requeue_pi_complete()
>                                             WAIT -> LOCKED
>     return LOCKED
>   return
> // q lifetime ends
>                                             rcuwait_wake_up()
> 
> futex_requeue_pi_complete() publishes LOCKED before calling
> rcuwait_wake_up(). Once the waiter observes LOCKED, it can return from
> futex_wait_requeue_pi() and let q go out of scope before rcuwait_wake_up()
> reads q->requeue_wait.task and passes the stale pointer to
> try_to_wake_up().
> 
> Skip rcuwait_wake_up() for Q_REQUEUE_PI_LOCKED. requeue_pi_wake_futex()
> already saves q->task before publishing LOCKED and wakes the saved task
> afterward.
> 
> Fixes: 07d91ef510fb1 ("futex: Prevent requeue_pi() lock nesting issue on RT")
> Cc: stable@vger.kernel.org
> Signed-off-by: Yao Kai <yaokai34@huawei.com>
> ---
>  kernel/futex/requeue.c | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/futex/requeue.c b/kernel/futex/requeue.c
> index abc652b5b2dd..59e587775d9b 100644
> --- a/kernel/futex/requeue.c
> +++ b/kernel/futex/requeue.c
> @@ -155,8 +155,13 @@ static inline void futex_requeue_pi_complete(struct futex_q *q, int locked)
>  	} while (!atomic_try_cmpxchg(&q->requeue_state, &old, new));
>  
>  #ifdef CONFIG_PREEMPT_RT
> -	/* If the waiter interleaved with the requeue let it know */
> -	if (unlikely(old == Q_REQUEUE_PI_WAIT))
> +	/*
> +	 * If the waiter interleaved with the requeue, let it know. For LOCKED,
> +	 * q may already be invalid, so requeue_pi_wake_futex() wakes the saved
> +	 * task instead.
> +	 */
> +	if (unlikely(old == Q_REQUEUE_PI_WAIT) &&
> +	    new != Q_REQUEUE_PI_LOCKED)
>  		rcuwait_wake_up(&q->requeue_wait);

Your whole assumption is based on the requeue_state in
futex_requeue_pi_wakeup_sync() changes from Q_REQUEUE_PI_IN_PROGRESS to
Q_REQUEUE_PI_WAIT and the rcuwait_wait_event() does not wait because the
condition becomes true before that happens. So the rcuwait_wake_up()
could access q.requeue_wait which is allocated on behalf of the waiter
which is gone. Certainly possible. But if we skip the wait in thise case
we probably miss the 99% cases where the waiter did wait, no?

This looks similar to commit b549113738e8c ("futex: Prevent
use-after-free during requeue-PI").

>  #endif
>  }

Sebastian