* [PATCH v2 1/5] locking/rtmutex: Stop spinning on the owner when the waiter is woken [not found] <20261108141347.1525532-1-mathieu.desnoyers@efficios.com> @ 2026-11-08 14:13 ` Mathieu Desnoyers 2026-11-16 12:06 ` Sebastian Andrzej Siewior 2026-11-08 14:13 ` [PATCH v2 2/5] locking/rtmutex: Initialize the ww_mutex context of a waiter Mathieu Desnoyers 2026-11-08 14:13 ` [PATCH v2 3/5] locking/rtmutex: Clone the sort keys in rt_mutex_enqueue_pi() Mathieu Desnoyers 2 siblings, 1 reply; 15+ messages in thread From: Mathieu Desnoyers @ 2026-11-08 14:13 UTC (permalink / raw) To: peterz, tglx, mingo Cc: linux-kernel, Mathieu Desnoyers, will, boqun, longman, rostedt, bigeasy, clrkwllms, dvhart, dave, andrealmeid, shuah, juri.lelli, linux-rt-devel, linux-kselftest, stable The top waiter of an rt_mutex sets its sleep state. Then it spins in rtmutex_spin_on_owner() as long as the owner is running. A wakeup sets the waiter back to TASK_RUNNING. The spin loop does not check for this. So a timeout or a signal is only handled when the owner blocks, or when the waiter has to reschedule. The rtmutex_spin_on_owner() is just an optimization with the idea that if the owner is still running on a CPU, it will likely release it soon. If the owner gets preempted or schedules out on its own, the spinning will stop and the waiter will sleep. This came from PREEMPT_RT adaptive spinning for rt_spin_locks that allowed them to behave more like normal spin locks and speed up PREEMPT_RT performance. The problem is that when this was implemented for futex and normal mutexes it did not account for timeouts or other mechanisms where the owner releasing the lock wasn't the only way for the spinning task to exit the mutex acquire. The spinning currently does not account for anything else but the lock being released or the owner sleeping. The owner of a PI futex is a user task. It can run for a long time. With an owner spinning in userspace, a 1ms pthread_mutex_timedlock() returned up to 1.4s late in testing. The timeout signal or any signal to the waiter will not stop it from spinning, and the signal will be delayed until it exits the spin. With PREEMPT_RT, two contexts that take two ww_mutexes in opposite order spin on each other. Only something else that needs one of the two CPUs stops them. By default, the fair server does it after 950ms, to run SCHED_OTHER tasks. Luckily, when the spinning task needs to exit the mutex lock code, its state is set to TASK_RUNNING. By having the spin check the spinning task's state to see if it switches to running, it can exit the spin appropriately. Fixes: 992caf7f1724 ("locking/rtmutex: Add adaptive spinwait mechanism") Cc: stable@vger.kernel.org Reviewed-by: Steven Rostedt <rostedt@goodmis.org> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> --- kernel/locking/rtmutex.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/kernel/locking/rtmutex.c b/kernel/locking/rtmutex.c index 5a9534c715b8..323973be2074 100644 --- a/kernel/locking/rtmutex.c +++ b/kernel/locking/rtmutex.c @@ -1516,9 +1516,13 @@ static bool rtmutex_spin_on_owner(struct rt_mutex_base *lock, * - current is not longer the top waiter * - current is requested to reschedule (redundant * for CONFIG_PREEMPT_RCU=y) + * - current is woken, by its timeout, a signal or + * whatever else would end the sleep the spin + * stands in for * - the VCPU on which owner runs is preempted */ if (!owner_on_cpu(owner) || need_resched() || + task_is_running(current) || !data_race(rt_mutex_waiter_is_top_waiter(lock, waiter))) { res = false; break; -- 2.47.3 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v2 1/5] locking/rtmutex: Stop spinning on the owner when the waiter is woken 2026-11-08 14:13 ` [PATCH v2 1/5] locking/rtmutex: Stop spinning on the owner when the waiter is woken Mathieu Desnoyers @ 2026-11-16 12:06 ` Sebastian Andrzej Siewior 2026-11-16 18:36 ` Mathieu Desnoyers 0 siblings, 1 reply; 15+ messages in thread From: Sebastian Andrzej Siewior @ 2026-11-16 12:06 UTC (permalink / raw) To: Mathieu Desnoyers Cc: peterz, tglx, mingo, linux-kernel, will, boqun, longman, rostedt, clrkwllms, dvhart, dave, andrealmeid, shuah, juri.lelli, linux-rt-devel, linux-kselftest, stable On 2026-11-08 09:13:43 [-0500], Mathieu Desnoyers wrote: > Fixes: 992caf7f1724 ("locking/rtmutex: Add adaptive spinwait mechanism") > Cc: stable@vger.kernel.org > Reviewed-by: Steven Rostedt <rostedt@goodmis.org> > Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> > --- > kernel/locking/rtmutex.c | 4 ++++ > 1 file changed, 4 insertions(+) > > diff --git a/kernel/locking/rtmutex.c b/kernel/locking/rtmutex.c > index 5a9534c715b8..323973be2074 100644 > --- a/kernel/locking/rtmutex.c > +++ b/kernel/locking/rtmutex.c > @@ -1516,9 +1516,13 @@ static bool rtmutex_spin_on_owner(struct rt_mutex_base *lock, > * - current is not longer the top waiter > * - current is requested to reschedule (redundant > * for CONFIG_PREEMPT_RCU=y) > + * - current is woken, by its timeout, a signal or > + * whatever else would end the sleep the spin > + * stands in for > * - the VCPU on which owner runs is preempted > */ > if (!owner_on_cpu(owner) || need_resched() || > + task_is_running(current) || Wouldn't it make sense to return true in this case so it can re-evaluate the situation for timeout/ signal rather than visit the scheduler for no reason? Also, wouldn't it make sense to restrict this to rt_mutex_slowlock_block() (and maybe there to the !TASK_UNINTERRUPTIBLE case but this might result in too much code) rather than have it also available rtlock_slowlock_locked() where it makes no difference? That file is included so having rtlock_slowlock_locked() pass an argument would remove that branch from code generation. > !data_race(rt_mutex_waiter_is_top_waiter(lock, waiter))) { > res = false; > break; Sebastian ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2 1/5] locking/rtmutex: Stop spinning on the owner when the waiter is woken 2026-11-16 12:06 ` Sebastian Andrzej Siewior @ 2026-11-16 18:36 ` Mathieu Desnoyers 2026-11-18 14:19 ` Sebastian Andrzej Siewior 0 siblings, 1 reply; 15+ messages in thread From: Mathieu Desnoyers @ 2026-11-16 18:36 UTC (permalink / raw) To: Sebastian Andrzej Siewior Cc: peterz, tglx, mingo, linux-kernel, will, boqun, longman, rostedt, clrkwllms, dvhart, dave, andrealmeid, shuah, juri.lelli, linux-rt-devel, linux-kselftest, stable Hi Sebastian, > Wouldn't it make sense to return true in this case so it can re-evaluate > the situation for timeout/ signal rather than visit the scheduler for no > reason? Yes. With true, rt_mutex_slowlock_block() takes the wait lock again and sets the sleep state. Then it tries the lock, and checks the timeout, the signal and the ww_mutex wound. A wakeup that is none of these cannot make it spin forever: the sleep state is set again before the next spin. > Also, wouldn't it make sense to restrict this to > rt_mutex_slowlock_block() (and maybe there to the !TASK_UNINTERRUPTIBLE > case but this might result in too much code) rather than have it also > available rtlock_slowlock_locked() where it makes no difference? Agreed for the rtlock side. An rtlock waiter is in TASK_RTLOCK_WAIT. The lock handoff alone makes it running, and the handoff changes the owner, which already ends the spin. Any other wakeup goes to saved_state. I added a bool argument, as you suggest. I kept the check for TASK_UNINTERRUPTIBLE on the rt_mutex side. A ww_mutex wound also wakes an uninterruptible waiter, and the ABBA case of patch 5 needs the spin to stop there. With both changes, the futex and ww_mutex selftests pass, with and without PREEMPT_RT. That includes the PI cases of patch 4. Steven, the code changed after your Reviewed-by, so I dropped the tag in the version below. Is it still fine with you? The planned next version of the patch follows. Thanks, Mathieu -- >8 -- From: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Date: Fri, 6 Nov 2026 13:56:54 -0500 Subject: [PATCH] locking/rtmutex: Stop spinning on the owner when the waiter is woken The top waiter of an rt_mutex sets its sleep state. Then it spins in rtmutex_spin_on_owner() as long as the owner is running. A wakeup sets the waiter back to TASK_RUNNING. The spin loop does not check for this. The rtmutex_spin_on_owner() is just an optimization with the idea that if the owner is still running on a CPU, it will likely release it soon. If the owner gets preempted or schedules out on its own, the spinning will stop and the waiter will sleep. This came from PREEMPT_RT adaptive spinning for rt_spin_locks that allowed them to behave more like normal spin locks and speed up PREEMPT_RT performance. The problem is that when this was implemented for futex and normal mutexes it did not account for timeouts or other mechanisms where the owner releasing the lock wasn't the only way for the spinning task to exit the mutex acquire. The spinning currently does not account for anything else but the lock being released or the owner sleeping. The owner of a PI futex is a user task. It can run for a long time. With an owner spinning in userspace, a 1ms pthread_mutex_timedlock() returned up to 1.4s late in testing. The timeout signal or any signal to the waiter will not stop it from spinning, and the signal will be delayed until it exits the spin. With PREEMPT_RT, two contexts that take two ww_mutexes in opposite order spin on each other. The wound that resolves this wakes up the younger one, but it keeps spinning. Only something else that needs one of the two CPUs stops them. By default, the fair server does it after 950ms, to run SCHED_OTHER tasks. Luckily, when the spinning task needs to exit the mutex lock code, its state is set to TASK_RUNNING. By having the spin check the spinning task's state to see if it switches to running, it can exit the spin appropriately. On a wakeup, the spin returns true. The caller then takes the wait lock again and checks why it was woken, with no call to the scheduler. Only rt_mutex_slowlock_block() asks for the check. An rtlock waiter is made running by the lock handoff alone, and the handoff changes the owner, which already ends the spin. Sebastian Andrzej Siewior suggested both. Fixes: 992caf7f1724 ("locking/rtmutex: Add adaptive spinwait mechanism") Cc: stable@vger.kernel.org Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> --- kernel/locking/rtmutex.c | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) diff --git a/kernel/locking/rtmutex.c b/kernel/locking/rtmutex.c index 5a9534c715b8..93014df90eef 100644 --- a/kernel/locking/rtmutex.c +++ b/kernel/locking/rtmutex.c @@ -1494,7 +1494,8 @@ static __always_inline void __rt_mutex_unlock(struct rt_mutex_base *lock) #ifdef CONFIG_SMP static bool rtmutex_spin_on_owner(struct rt_mutex_base *lock, struct rt_mutex_waiter *waiter, - struct task_struct *owner) + struct task_struct *owner, + bool check_wakeup) { bool res = true; @@ -1510,6 +1511,13 @@ static bool rtmutex_spin_on_owner(struct rt_mutex_base *lock, * the rcu_read_lock() ensures the memory stays valid. */ barrier(); + /* + * Stop spinning when current is woken, for example by a + * timeout or a signal. Return true: the caller checks why, + * with no call to the scheduler. + */ + if (check_wakeup && task_is_running(current)) + break; /* * Stop spinning when: * - the lock owner has been scheduled out @@ -1531,7 +1539,8 @@ static bool rtmutex_spin_on_owner(struct rt_mutex_base *lock, #else static bool rtmutex_spin_on_owner(struct rt_mutex_base *lock, struct rt_mutex_waiter *waiter, - struct task_struct *owner) + struct task_struct *owner, + bool check_wakeup) { return false; } @@ -1665,7 +1674,7 @@ static int __sched rt_mutex_slowlock_block(struct rt_mutex_base *lock, owner = NULL; raw_spin_unlock_irq_wake(&lock->wait_lock, wake_q); - if (!owner || !rtmutex_spin_on_owner(lock, waiter, owner)) { + if (!owner || !rtmutex_spin_on_owner(lock, waiter, owner, true)) { lockevent_inc(rtmutex_slow_sleep); rt_mutex_schedule(); } @@ -1890,7 +1899,12 @@ static void __sched rtlock_slowlock_locked(struct rt_mutex_base *lock, owner = NULL; raw_spin_unlock_irq_wake(&lock->wait_lock, wake_q); - if (!owner || !rtmutex_spin_on_owner(lock, &waiter, owner)) { + /* + * Only the lock handoff makes an rtlock waiter running, + * and it changes the owner. Any other wakeup goes to + * saved_state. So the spin need not check for a wakeup. + */ + if (!owner || !rtmutex_spin_on_owner(lock, &waiter, owner, false)) { lockevent_inc(rtlock_slow_sleep); schedule_rtlock(); } -- 2.47.3 -- Mathieu Desnoyers EfficiOS Inc. https://www.efficios.com ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v2 1/5] locking/rtmutex: Stop spinning on the owner when the waiter is woken 2026-11-16 18:36 ` Mathieu Desnoyers @ 2026-11-18 14:19 ` Sebastian Andrzej Siewior 2026-11-18 17:27 ` Mathieu Desnoyers 0 siblings, 1 reply; 15+ messages in thread From: Sebastian Andrzej Siewior @ 2026-11-18 14:19 UTC (permalink / raw) To: Mathieu Desnoyers Cc: peterz, tglx, mingo, linux-kernel, will, boqun, longman, rostedt, clrkwllms, dvhart, dave, andrealmeid, shuah, juri.lelli, linux-rt-devel, linux-kselftest, stable On 2026-11-16 13:36:39 [-0500], Mathieu Desnoyers wrote: > Hi Sebastian, Hi, … > I kept the check for TASK_UNINTERRUPTIBLE on the rt_mutex side. A > ww_mutex wound also wakes an uninterruptible waiter, and the ABBA case > of patch 5 needs the spin to stop there. Not sure what is related to. I don't see this in this patch. TASK_UNINTERRUPTIBLE vs TASK_INTERRUPTIBLE would make a difference for signals here. … > From: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> > Date: Fri, 6 Nov 2026 13:56:54 -0500 > Subject: [PATCH] locking/rtmutex: Stop spinning on the owner when the waiter > is woken > > The top waiter of an rt_mutex sets its sleep state. Then it spins in > rtmutex_spin_on_owner() as long as the owner is running. A wakeup sets > the waiter back to TASK_RUNNING. The spin loop does not check for this. > > The rtmutex_spin_on_owner() is just an optimization with the idea that > if the owner is still running on a CPU, it will likely release it soon. > If the owner gets preempted or schedules out on its own, the spinning > will stop and the waiter will sleep. This came from PREEMPT_RT adaptive > spinning for rt_spin_locks that allowed them to behave more like normal > spin locks and speed up PREEMPT_RT performance. > > The problem is that when this was implemented for futex and normal > mutexes it did not account for timeouts or other mechanisms where > the owner releasing the lock wasn't the only way for the spinning > task to exit the mutex acquire. The spinning currently does not > account for anything else but the lock being released or the owner > sleeping. > > The owner of a PI futex is a user task. It can run for a long time. > With an owner spinning in userspace, a 1ms pthread_mutex_timedlock() > returned up to 1.4s late in testing. The timeout signal or any signal > to the waiter will not stop it from spinning, and the signal will be > delayed until it exits the spin. > > With PREEMPT_RT, two contexts that take two ww_mutexes in opposite > order spin on each other. The wound that resolves this wakes up the > younger one, but it keeps spinning. Only something else that needs one > of the two CPUs stops them. By default, the fair server does it after > 950ms, to run SCHED_OTHER tasks. > > Luckily, when the spinning task needs to exit the mutex lock code, its > state is set to TASK_RUNNING. By having the spin check the spinning > task's state to see if it switches to running, it can exit the spin > appropriately. > > On a wakeup, the spin returns true. The caller then takes the wait lock > again and checks why it was woken, with no call to the scheduler. Only > rt_mutex_slowlock_block() asks for the check. An rtlock waiter is made > running by the lock handoff alone, and the handoff changes the owner, > which already ends the spin. Sebastian Andrzej Siewior suggested both. For my taste this is overly verbose. In the spirit of "ain't nobody got time for that" what about rtmutex_spin_on_owner() does not abort spinning if the task state returns to TASK_RUNNING. This can happen if the task receives a signal or timeout. It will continue to spin on the lock as long as the owner is on the CPU and so delaying the processing of the signal or timeout condition. This affects PI futex users in general and ww_mutexes users on PREEMPT_RT. Stop spinning when the task state changes to TASK_RUNNING. Return true to let the caller check condition again instead going to the scheduler. Skip this in rtlock_slowlock_locked() which can not be interrupted and always needs to acquire the lock. > Fixes: 992caf7f1724 ("locking/rtmutex: Add adaptive spinwait mechanism") > Cc: stable@vger.kernel.org > Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> I agree with the change as-is so Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> but the text here is too verbose. > --- a/kernel/locking/rtmutex.c > +++ b/kernel/locking/rtmutex.c > @@ -1510,6 +1511,13 @@ static bool rtmutex_spin_on_owner(struct rt_mutex_base *lock, > * the rcu_read_lock() ensures the memory stays valid. > */ > barrier(); > + /* > + * Stop spinning when current is woken, for example by a > + * timeout or a signal. Return true: the caller checks why, > + * with no call to the scheduler. > + */ /* If became running, trylock again or abort (signal/ * timeout) */ > + if (check_wakeup && task_is_running(current)) > + break; > /* > * Stop spinning when: > * - the lock owner has been scheduled out > @@ -1890,7 +1899,12 @@ static void __sched rtlock_slowlock_locked(struct rt_mutex_base *lock, > owner = NULL; > raw_spin_unlock_irq_wake(&lock->wait_lock, wake_q); > - if (!owner || !rtmutex_spin_on_owner(lock, &waiter, owner)) { > + /* > + * Only the lock handoff makes an rtlock waiter running, > + * and it changes the owner. Any other wakeup goes to > + * saved_state. So the spin need not check for a wakeup. > + */ I would drop this entirely. It says below that the check is for signal/ timeout which is something we don't have here. So I don't think this is required. > + if (!owner || !rtmutex_spin_on_owner(lock, &waiter, owner, false)) { > lockevent_inc(rtlock_slow_sleep); > schedule_rtlock(); > } Sebastian ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2 1/5] locking/rtmutex: Stop spinning on the owner when the waiter is woken 2026-11-18 14:19 ` Sebastian Andrzej Siewior @ 2026-11-18 17:27 ` Mathieu Desnoyers 2026-11-18 18:51 ` Sebastian Andrzej Siewior 0 siblings, 1 reply; 15+ messages in thread From: Mathieu Desnoyers @ 2026-11-18 17:27 UTC (permalink / raw) To: Sebastian Andrzej Siewior Cc: peterz, tglx, mingo, linux-kernel, will, boqun, longman, rostedt, clrkwllms, dvhart, dave, andrealmeid, shuah, juri.lelli, linux-rt-devel, linux-kselftest, stable Hi Sebastian, > > I kept the check for TASK_UNINTERRUPTIBLE on the rt_mutex side. A > > ww_mutex wound also wakes an uninterruptible waiter, and the ABBA case > > of patch 5 needs the spin to stop there. > > Not sure what is related to. I don't see this in this patch. > TASK_UNINTERRUPTIBLE vs TASK_INTERRUPTIBLE would make a difference for > signals here. Sorry, that was unclear. The patch tests no state. You had suggested to limit the check to !TASK_UNINTERRUPTIBLE. I did not do that. A ww_mutex waiter sleeps in TASK_UNINTERRUPTIBLE, and the wound wakes it. That is the ABBA case of patch 5. So the spin must stop for an uninterruptible waiter too. > For my taste this is overly verbose. In the spirit of "ain't nobody got > time for that" what about [...] I will take your text in the next version. > I agree with the change as-is so > Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Thanks! > /* If became running, trylock again or abort (signal/ > * timeout) > */ I will take this comment. > I would drop this entirely. It says below that the check is for signal/ > timeout which is something we don't have here. So I don't think this is > required. I will drop it. Thanks, Mathieu -- Mathieu Desnoyers EfficiOS Inc. https://www.efficios.com ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2 1/5] locking/rtmutex: Stop spinning on the owner when the waiter is woken 2026-11-18 17:27 ` Mathieu Desnoyers @ 2026-11-18 18:51 ` Sebastian Andrzej Siewior 0 siblings, 0 replies; 15+ messages in thread From: Sebastian Andrzej Siewior @ 2026-11-18 18:51 UTC (permalink / raw) To: Mathieu Desnoyers Cc: peterz, tglx, mingo, linux-kernel, will, boqun, longman, rostedt, clrkwllms, dvhart, dave, andrealmeid, shuah, juri.lelli, linux-rt-devel, linux-kselftest, stable On 2026-11-18 12:27:45 [-0500], Mathieu Desnoyers wrote: > Hi Sebastian, Hi Mathieu, > > > I kept the check for TASK_UNINTERRUPTIBLE on the rt_mutex side. A > > > ww_mutex wound also wakes an uninterruptible waiter, and the ABBA case > > > of patch 5 needs the spin to stop there. > > > > Not sure what is related to. I don't see this in this patch. > > TASK_UNINTERRUPTIBLE vs TASK_INTERRUPTIBLE would make a difference for > > signals here. > > Sorry, that was unclear. The patch tests no state. You had suggested > to limit the check to !TASK_UNINTERRUPTIBLE. I did not do that. A > ww_mutex waiter sleeps in TASK_UNINTERRUPTIBLE, and the wound wakes it. > That is the ABBA case of patch 5. So the spin must stop for an > uninterruptible waiter too. Ah okay, that makes sense. Sorry for the confusion then :). Sebastian ^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v2 2/5] locking/rtmutex: Initialize the ww_mutex context of a waiter [not found] <20261108141347.1525532-1-mathieu.desnoyers@efficios.com> 2026-11-08 14:13 ` [PATCH v2 1/5] locking/rtmutex: Stop spinning on the owner when the waiter is woken Mathieu Desnoyers @ 2026-11-08 14:13 ` Mathieu Desnoyers 2026-11-16 13:58 ` Sebastian Andrzej Siewior 2026-11-08 14:13 ` [PATCH v2 3/5] locking/rtmutex: Clone the sort keys in rt_mutex_enqueue_pi() Mathieu Desnoyers 2 siblings, 1 reply; 15+ messages in thread From: Mathieu Desnoyers @ 2026-11-08 14:13 UTC (permalink / raw) To: peterz, tglx, mingo Cc: linux-kernel, Mathieu Desnoyers, will, boqun, longman, rostedt, bigeasy, clrkwllms, dvhart, dave, andrealmeid, shuah, juri.lelli, linux-rt-devel, linux-kselftest, stable With PREEMPT_RT and CONFIG_DEBUG_RT_MUTEXES, FUTEX_LOCK_PI reports no deadlock. Two threads that each block on a PI futex the other holds sleep forever. Without PREEMPT_RT, the second one gets EDEADLK. A chain deeper than max_lock_depth is also admitted. With PREEMPT_RT, ww_mutexes are built on rtmutexes. They can create a cycle in the lock graph legitimately, and the wound/die logic resolves it. So the PREEMPT_RT chain walk turns deadlock detection off when it finds a waiter with a ww_mutex context, waiter->ww_ctx. It also drops the -EDEADLK of a walk that started from such a waiter. But rt_mutex_init_waiter() does not set waiter->ww_ctx. Only __rt_mutex_slowlock_locked() sets it. So the waiters of PI futexes and rtlocks keep the 0x11 poison of CONFIG_DEBUG_RT_MUTEXES. Without that option, they keep what was on the stack, unless the compiler zeroes it. The chain walk then takes them for ww_mutex waiters. The ww_mutex build of the chain walk can also requeue such waiters. On two of them with equal priority, __waiter_less() would dereference the poison. Fix this by clearing waiter->ww_ctx in rt_mutex_init_waiter(). With this patch, the ABBA and the deep chain both get EDEADLK. The futex_lock_pi_inherit selftest added later in this series found this. Its deadlock and max_lock_depth cases cover it. Fixes: 6467822b8cc9 ("locking/rtmutex: Prevent spurious EDEADLK return caused by ww_mutexes") Cc: stable@vger.kernel.org Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> --- kernel/locking/rtmutex_common.h | 1 + 1 file changed, 1 insertion(+) diff --git a/kernel/locking/rtmutex_common.h b/kernel/locking/rtmutex_common.h index c38b7bdea7b3..1f56d40c3e66 100644 --- a/kernel/locking/rtmutex_common.h +++ b/kernel/locking/rtmutex_common.h @@ -218,6 +218,7 @@ static inline void rt_mutex_init_waiter(struct rt_mutex_waiter *waiter) RB_CLEAR_NODE(&waiter->tree.entry); waiter->wake_state = TASK_NORMAL; waiter->task = NULL; + waiter->ww_ctx = NULL; } static inline void rt_mutex_init_rtlock_waiter(struct rt_mutex_waiter *waiter) -- 2.47.3 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v2 2/5] locking/rtmutex: Initialize the ww_mutex context of a waiter 2026-11-08 14:13 ` [PATCH v2 2/5] locking/rtmutex: Initialize the ww_mutex context of a waiter Mathieu Desnoyers @ 2026-11-16 13:58 ` Sebastian Andrzej Siewior 2026-11-16 18:37 ` Mathieu Desnoyers 0 siblings, 1 reply; 15+ messages in thread From: Sebastian Andrzej Siewior @ 2026-11-16 13:58 UTC (permalink / raw) To: Mathieu Desnoyers Cc: peterz, tglx, mingo, linux-kernel, will, boqun, longman, rostedt, clrkwllms, dvhart, dave, andrealmeid, shuah, juri.lelli, linux-rt-devel, linux-kselftest, stable On 2026-11-08 09:13:44 [-0500], Mathieu Desnoyers wrote: > With PREEMPT_RT and CONFIG_DEBUG_RT_MUTEXES, FUTEX_LOCK_PI reports no > deadlock. Two threads that each block on a PI futex the other holds > sleep forever. Without PREEMPT_RT, the second one gets EDEADLK. A chain > deeper than max_lock_depth is also admitted. > > With PREEMPT_RT, ww_mutexes are built on rtmutexes. They can create a > cycle in the lock graph legitimately, and the wound/die logic resolves > it. So the PREEMPT_RT chain walk turns deadlock detection off when it > finds a waiter with a ww_mutex context, waiter->ww_ctx. It also drops > the -EDEADLK of a walk that started from such a waiter. I *think* this is unrelated because the futex caller shouldn't end up here. The __rt_mutex_slowlock() -> rt_mutex_handle_deadlock() should be different for the futex and ww_mutex case. > But rt_mutex_init_waiter() does not set waiter->ww_ctx. Only > __rt_mutex_slowlock_locked() sets it. So the waiters of PI futexes and > rtlocks keep the 0x11 poison of CONFIG_DEBUG_RT_MUTEXES. Without that > option, they keep what was on the stack, unless the compiler zeroes it. > The chain walk then takes them for ww_mutex waiters. Is this really the case? You have a lock chain where the ww_mutex and a regular rtmutex interact? This doesn't sound too crazy. __waiter_less() expects it to be NULL for those. > > The ww_mutex build of the chain walk can also requeue such waiters. On > two of them with equal priority, __waiter_less() would dereference the > poison. I *think* the enqueue part is only affected if a regular rtmutex already exists. Because the rtmutex enqueue would ignore ww_mutex/ treat it as equals. This will affect rt_mutex_adjust_prio_chain(), the futex interface which is the only user of the chainwalk. > Fix this by clearing waiter->ww_ctx in rt_mutex_init_waiter(). > > With this patch, the ABBA and the deep chain both get EDEADLK. The > futex_lock_pi_inherit selftest added later in this series found this. > Its deadlock and max_lock_depth cases cover it. > Fixes: 6467822b8cc9 ("locking/rtmutex: Prevent spurious EDEADLK return caused by ww_mutexes") > Cc: stable@vger.kernel.org > Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> This is the right thing to do. Either my analysis is wrong or the changelog confusing. I would argue that the full-chainwalk depends now on a random stack value _and_ __waiter_less() might access the random value if ww_mutex is enqueued and a regular mutex exists in the lock chain. Sebastian ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2 2/5] locking/rtmutex: Initialize the ww_mutex context of a waiter 2026-11-16 13:58 ` Sebastian Andrzej Siewior @ 2026-11-16 18:37 ` Mathieu Desnoyers 2026-11-18 14:40 ` Sebastian Andrzej Siewior 0 siblings, 1 reply; 15+ messages in thread From: Mathieu Desnoyers @ 2026-11-16 18:37 UTC (permalink / raw) To: Sebastian Andrzej Siewior Cc: peterz, tglx, mingo, linux-kernel, will, boqun, longman, rostedt, clrkwllms, dvhart, dave, andrealmeid, shuah, juri.lelli, linux-rt-devel, linux-kselftest, stable Hi Sebastian, > I *think* this is unrelated because the futex caller shouldn't end up > here. The __rt_mutex_slowlock() -> rt_mutex_handle_deadlock() should be > different for the futex and ww_mutex case. The futex caller does not go through __rt_mutex_slowlock(). It gets to the chain walk through the proxy lock: futex_lock_pi() rt_mutex_init_waiter(&rt_waiter) __rt_mutex_start_proxy_lock() task_blocks_on_rt_mutex(..., RT_MUTEX_FULL_CHAINWALK) rt_mutex_adjust_prio_chain() rt_mutex_adjust_prio_chain() reads waiter->ww_ctx in two places: if (IS_ENABLED(CONFIG_PREEMPT_RT) && waiter->ww_ctx && detect_deadlock) detect_deadlock = false; [...] if (IS_ENABLED(CONFIG_PREEMPT_RT) && orig_waiter && orig_waiter->ww_ctx) ret = 0; Both tests are under IS_ENABLED(CONFIG_PREEMPT_RT), not under build_ww_mutex(). So the futex build of the chain walk has them too. > Is this really the case? > You have a lock chain where the ww_mutex and a regular rtmutex interact? No ww_mutex is needed. Two PI futexes are enough. Both waiters come from rt_mutex_init_waiter() in futex_lock_pi(), so both hold the 0x11 poison in ww_ctx with CONFIG_DEBUG_RT_MUTEXES. The chain walk takes them for ww_mutex waiters. The deadlock case of futex_lock_pi_inherit (patch 4) is that ABBA. With PREEMPT_RT and CONFIG_DEBUG_RT_MUTEXES, it fails without this patch: the second FUTEX_LOCK_PI blocks. It passes with it. > I would argue that the full-chainwalk depends now > on a random stack value _and_ __waiter_less() might access the random > value if ww_mutex is enqueued and a regular mutex exists in the lock > chain. Yes, those are the two effects. I reproduced the first one only. The second one is from reading the code, which is why the changelog says "would dereference". You are right that the changelog is confusing. It starts with the futex symptom, and only then says where the value is read. In the planned next version, I would put the mechanism first, with the call path above. Does this read better? rt_mutex_init_waiter() does not set waiter->ww_ctx. Only __rt_mutex_slowlock_locked() sets it. So the waiters of PI futexes and rtlocks keep the 0x11 poison of CONFIG_DEBUG_RT_MUTEXES. Without that option, they keep what was on the stack, unless the compiler zeroes it. With PREEMPT_RT, ww_mutexes are built on rtmutexes. They can create a cycle in the lock graph legitimately, and the wound/die logic resolves it. So rt_mutex_adjust_prio_chain() turns deadlock detection off when it finds a waiter with a ww_ctx. It also drops the -EDEADLK of a walk that started from such a waiter. Both tests are under IS_ENABLED(CONFIG_PREEMPT_RT), so every build of the chain walk has them. FUTEX_LOCK_PI gets there with no ww_mutex in the chain: futex_lock_pi() rt_mutex_init_waiter() __rt_mutex_start_proxy_lock() task_blocks_on_rt_mutex() rt_mutex_adjust_prio_chain() The chain walk takes its waiters for ww_mutex waiters. With PREEMPT_RT and CONFIG_DEBUG_RT_MUTEXES, FUTEX_LOCK_PI then reports no deadlock. Two threads that each block on a PI futex the other holds sleep forever. Without PREEMPT_RT, the second one gets EDEADLK. FUTEX_LOCK_PI also blocks on a chain deeper than max_lock_depth, instead of returning EDEADLK. The ww_mutex build of the chain walk can also requeue such waiters. On two of them with equal priority, __waiter_less() would dereference the poison. Fix this by clearing waiter->ww_ctx in rt_mutex_init_waiter(). Thanks, Mathieu -- Mathieu Desnoyers EfficiOS Inc. https://www.efficios.com ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2 2/5] locking/rtmutex: Initialize the ww_mutex context of a waiter 2026-11-16 18:37 ` Mathieu Desnoyers @ 2026-11-18 14:40 ` Sebastian Andrzej Siewior 2026-11-18 17:28 ` Mathieu Desnoyers 0 siblings, 1 reply; 15+ messages in thread From: Sebastian Andrzej Siewior @ 2026-11-18 14:40 UTC (permalink / raw) To: Mathieu Desnoyers Cc: peterz, tglx, mingo, linux-kernel, will, boqun, longman, rostedt, clrkwllms, dvhart, dave, andrealmeid, shuah, juri.lelli, linux-rt-devel, linux-kselftest, stable On 2026-11-16 13:37:09 [-0500], Mathieu Desnoyers wrote: > Hi Sebastian, Hi Mathieu, > > I *think* this is unrelated because the futex caller shouldn't end up > > here. The __rt_mutex_slowlock() -> rt_mutex_handle_deadlock() should be > > different for the futex and ww_mutex case. > > The futex caller does not go through __rt_mutex_slowlock(). It gets to > the chain walk through the proxy lock: > > futex_lock_pi() > rt_mutex_init_waiter(&rt_waiter) > __rt_mutex_start_proxy_lock() > task_blocks_on_rt_mutex(..., RT_MUTEX_FULL_CHAINWALK) > rt_mutex_adjust_prio_chain() > > rt_mutex_adjust_prio_chain() reads waiter->ww_ctx in two places: > > if (IS_ENABLED(CONFIG_PREEMPT_RT) && waiter->ww_ctx && detect_deadlock) > detect_deadlock = false; > [...] > if (IS_ENABLED(CONFIG_PREEMPT_RT) && orig_waiter && orig_waiter->ww_ctx) > ret = 0; > > Both tests are under IS_ENABLED(CONFIG_PREEMPT_RT), not under > build_ww_mutex(). So the futex build of the chain walk has them too. Yes, this part. I realised this later… > > Is this really the case? > > You have a lock chain where the ww_mutex and a regular rtmutex interact? > > No ww_mutex is needed. Two PI futexes are enough. Both waiters come from > rt_mutex_init_waiter() in futex_lock_pi(), so both hold the 0x11 poison > in ww_ctx with CONFIG_DEBUG_RT_MUTEXES. The chain walk takes them for > ww_mutex waiters. > > The deadlock case of futex_lock_pi_inherit (patch 4) is that ABBA. With > PREEMPT_RT and CONFIG_DEBUG_RT_MUTEXES, it fails without this patch: the > second FUTEX_LOCK_PI blocks. It passes with it. right. That is an issue. > > I would argue that the full-chainwalk depends now > > on a random stack value _and_ __waiter_less() might access the random > > value if ww_mutex is enqueued and a regular mutex exists in the lock > > chain. > > Yes, those are the two effects. I reproduced the first one only. The > second one is from reading the code, which is why the changelog says > "would dereference". The deference _shouldn't_ happen because those two a different. However, we could have a mix of those two and which point it becomes relevant. > You are right that the changelog is confusing. It starts with the futex > symptom, and only then says where the value is read. In the planned next > version, I would put the mechanism first, with the call path above. Does > this read better? > > rt_mutex_init_waiter() does not set waiter->ww_ctx. Only rt_mutex_waiter::ww_ctx please > __rt_mutex_slowlock_locked() sets it. So the waiters of PI futexes > and rtlocks keep the 0x11 poison of CONFIG_DEBUG_RT_MUTEXES. Without or random garbage which. > that option, they keep what was on the stack, unless the compiler > zeroes it. > > With PREEMPT_RT, ww_mutexes are built on rtmutexes. They can create > a cycle in the lock graph legitimately, and the wound/die logic > resolves it. So rt_mutex_adjust_prio_chain() turns deadlock > detection off when it finds a waiter with a ww_ctx. It also drops > the -EDEADLK of a walk that started from such a waiter. Both tests > are under IS_ENABLED(CONFIG_PREEMPT_RT), so every build of the chain > walk has them. I would strip that paragraph because *I* think it is more important to mention the FUTEX case which no longer does proper chainwalk regardless of PREEMPT_RT. Yes, ww_mutex has > FUTEX_LOCK_PI gets there with no ww_mutex in the chain: > > futex_lock_pi() > rt_mutex_init_waiter() > __rt_mutex_start_proxy_lock() > task_blocks_on_rt_mutex() > rt_mutex_adjust_prio_chain() > > The chain walk takes its waiters for ww_mutex waiters. With > PREEMPT_RT and CONFIG_DEBUG_RT_MUTEXES, FUTEX_LOCK_PI then reports > no deadlock. Two threads that each block on a PI futex the other > holds sleep forever. Without PREEMPT_RT, the second one gets > EDEADLK. FUTEX_LOCK_PI also blocks on a chain deeper than > max_lock_depth, instead of returning EDEADLK. > > The ww_mutex build of the chain walk can also requeue such waiters. > On two of them with equal priority, __waiter_less() would > dereference the poison. The futex code for PI_LOCK performs a RT_MUTEX_FULL_CHAINWALK during locking. rt_mutex_adjust_prio_chain() then wrongly assumes that it is a ww_mutex (due to random garbage in rt_mutex_waiter::ww_ctx) and does not report a deadlock but expects the caller to handle it. > Fix this by clearing waiter->ww_ctx in rt_mutex_init_waiter(). rt_mutex_waiter::ww_ctx This, Fixes: and Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> > Thanks, > > Mathieu Sebastian ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2 2/5] locking/rtmutex: Initialize the ww_mutex context of a waiter 2026-11-18 14:40 ` Sebastian Andrzej Siewior @ 2026-11-18 17:28 ` Mathieu Desnoyers 2026-11-18 18:49 ` Sebastian Andrzej Siewior 0 siblings, 1 reply; 15+ messages in thread From: Mathieu Desnoyers @ 2026-11-18 17:28 UTC (permalink / raw) To: Sebastian Andrzej Siewior Cc: peterz, tglx, mingo, linux-kernel, will, boqun, longman, rostedt, clrkwllms, dvhart, dave, andrealmeid, shuah, juri.lelli, linux-rt-devel, linux-kselftest, stable Hi Sebastian, > The deference _shouldn't_ happen because those two a different. However, > we could have a mix of those two and which point it becomes relevant. Agreed. It needs a chain that mixes both. I dropped that paragraph. > I would strip that paragraph because *I* think it is more important to > mention the FUTEX case which no longer does proper chainwalk regardless > of PREEMPT_RT. One point. The two tests of ww_ctx in rt_mutex_adjust_prio_chain() are under IS_ENABLED(CONFIG_PREEMPT_RT). Without PREEMPT_RT, futex_lock_pi_inherit gets EDEADLK in both cases, with or without this patch. So I kept the PREEMPT_RT condition in the message. Did you have another path in mind? With your changes, the message reads: rt_mutex_init_waiter() does not set rt_mutex_waiter::ww_ctx. Only __rt_mutex_slowlock_locked() sets it. So the waiters of PI futexes and rtlocks keep the 0x11 poison of CONFIG_DEBUG_RT_MUTEXES, or random garbage without that option. The futex code for FUTEX_LOCK_PI performs a RT_MUTEX_FULL_CHAINWALK during locking: futex_lock_pi() rt_mutex_init_waiter() __rt_mutex_start_proxy_lock() task_blocks_on_rt_mutex() rt_mutex_adjust_prio_chain() With PREEMPT_RT, rt_mutex_adjust_prio_chain() then wrongly assumes that it is a ww_mutex, due to the garbage in rt_mutex_waiter::ww_ctx. It does not report a deadlock, but expects the caller to handle it. Two threads that each block on a PI futex the other holds then sleep forever. Without PREEMPT_RT, the second one gets EDEADLK. FUTEX_LOCK_PI also blocks on a chain deeper than max_lock_depth, instead of returning EDEADLK. Fix this by clearing rt_mutex_waiter::ww_ctx in rt_mutex_init_waiter(). With this patch, FUTEX_LOCK_PI returns EDEADLK in both cases. The futex_lock_pi_inherit selftest added later in this series found this, with PREEMPT_RT and CONFIG_DEBUG_RT_MUTEXES. Its deadlock and max_lock_depth cases cover it. > This, Fixes: and > > Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Thanks! Mathieu -- Mathieu Desnoyers EfficiOS Inc. https://www.efficios.com ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2 2/5] locking/rtmutex: Initialize the ww_mutex context of a waiter 2026-11-18 17:28 ` Mathieu Desnoyers @ 2026-11-18 18:49 ` Sebastian Andrzej Siewior 0 siblings, 0 replies; 15+ messages in thread From: Sebastian Andrzej Siewior @ 2026-11-18 18:49 UTC (permalink / raw) To: Mathieu Desnoyers Cc: peterz, tglx, mingo, linux-kernel, will, boqun, longman, rostedt, clrkwllms, dvhart, dave, andrealmeid, shuah, juri.lelli, linux-rt-devel, linux-kselftest, stable On 2026-11-18 12:28:54 [-0500], Mathieu Desnoyers wrote: > Hi Sebastian, Hi Mathieu, > > I would strip that paragraph because *I* think it is more important to > > mention the FUTEX case which no longer does proper chainwalk regardless > > of PREEMPT_RT. > > One point. The two tests of ww_ctx in rt_mutex_adjust_prio_chain() are > under IS_ENABLED(CONFIG_PREEMPT_RT). Without PREEMPT_RT, > futex_lock_pi_inherit gets EDEADLK in both cases, with or without this > patch. So I kept the PREEMPT_RT condition in the message. Did you have > another path in mind? Indeed. This supposed to be simple code… > With your changes, the message reads: > … Good, works for me. > Thanks! > > Mathieu Sebastian ^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v2 3/5] locking/rtmutex: Clone the sort keys in rt_mutex_enqueue_pi() [not found] <20261108141347.1525532-1-mathieu.desnoyers@efficios.com> 2026-11-08 14:13 ` [PATCH v2 1/5] locking/rtmutex: Stop spinning on the owner when the waiter is woken Mathieu Desnoyers 2026-11-08 14:13 ` [PATCH v2 2/5] locking/rtmutex: Initialize the ww_mutex context of a waiter Mathieu Desnoyers @ 2026-11-08 14:13 ` Mathieu Desnoyers 2026-11-16 15:11 ` Sebastian Andrzej Siewior 2 siblings, 1 reply; 15+ messages in thread From: Mathieu Desnoyers @ 2026-11-08 14:13 UTC (permalink / raw) To: peterz, tglx, mingo Cc: linux-kernel, Mathieu Desnoyers, will, boqun, longman, rostedt, bigeasy, clrkwllms, dvhart, dave, andrealmeid, shuah, juri.lelli, linux-rt-devel, linux-kselftest, stable An rt_mutex waiter has two copies of its sort keys. waiter->tree sorts the waiters of its lock. waiter->pi_tree sorts the pi_waiters of the lock owner. Only the top waiter of each lock is in pi_waiters. When the priority of a waiter changes, the chain walk updates waiter->tree. It copies waiter->tree to waiter->pi_tree for the top waiter only. A waiter that is not the top waiter keeps its old pi_tree keys. Such a waiter can become the top waiter later. remove_waiter() does this when the top waiter leaves. try_to_take_rt_mutex() does it when the lock changes owner. Both put the new top waiter in pi_waiters without a new copy. So pi_waiters sorts it by a priority it no longer has. The owner then inherits from the wrong waiter, at that waiter's current priority. In this example, O is the owner, a SCHED_OTHER task. L1 and L2 are two rt_mutexes. W1, W2 and W3 are waiters. Their SCHED_FIFO priority is under their name. T is any other task. It changes the priority of W2: T O W1 W2 W3 FIFO 80 FIFO 70 FIFO 50 locks L1, L2 blocks on L2 blocks on L1 (timed) blocks on L1 sched_setscheduler(W2) W2 now at FIFO 10 rt_mutex_adjust_pi(W2) W2->tree: FIFO 10 W2 not top: no copy W2->pi_tree: FIFO 70 <timeout> remove_waiter() rt_mutex_enqueue_pi(O, W2) W2 sorted as FIFO 70, ahead of W3 rt_mutex_adjust_prio(O) O inherits W2's FIFO 10 runs at FIFO 10 still waits The owner runs at SCHED_FIFO 10 while a SCHED_FIFO 50 task waits for it. This needs no race. Each step can come long after the one before. Fix this by copying the keys in rt_mutex_enqueue_pi(). All callers hold the wait_lock and the owner's pi_lock, as waiter_clone_prio() requires. Remove the three copies the callers made. With this patch, the owner runs at SCHED_FIFO 50. The futex_lock_pi_inherit selftest added later in this series tests this case (promoted_waiter). Fixes: f7853c342418 ("locking/rtmutex: Fix task->pi_waiters integrity") Cc: stable@vger.kernel.org Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> --- kernel/locking/rtmutex.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/kernel/locking/rtmutex.c b/kernel/locking/rtmutex.c index 323973be2074..975efa1d864c 100644 --- a/kernel/locking/rtmutex.c +++ b/kernel/locking/rtmutex.c @@ -517,6 +517,8 @@ rt_mutex_enqueue_pi(struct task_struct *task, struct rt_mutex_waiter *waiter) { lockdep_assert_held(&task->pi_lock); + /* The tree keys can change while @waiter is not in pi_waiters. */ + waiter_clone_prio(waiter, task); rb_add_cached(&waiter->pi_tree.entry, &task->pi_waiters, __pi_waiter_less); } @@ -1005,7 +1007,6 @@ static int __sched rt_mutex_adjust_prio_chain(struct task_struct *task, * and adjust the priority of the owner. */ rt_mutex_dequeue_pi(task, prerequeue_top_waiter); - waiter_clone_prio(waiter, task); rt_mutex_enqueue_pi(task, waiter); rt_mutex_adjust_prio(lock, task); @@ -1022,7 +1023,6 @@ static int __sched rt_mutex_adjust_prio_chain(struct task_struct *task, */ rt_mutex_dequeue_pi(task, waiter); waiter = rt_mutex_top_waiter(lock); - waiter_clone_prio(waiter, task); rt_mutex_enqueue_pi(task, waiter); rt_mutex_adjust_prio(lock, task); } else { @@ -1243,7 +1243,6 @@ static int __sched task_blocks_on_rt_mutex(struct rt_mutex_base *lock, waiter->task = task; waiter->lock = lock; waiter_update_prio(waiter, task); - waiter_clone_prio(waiter, task); /* Get the top priority waiter on the lock */ if (rt_mutex_has_waiters(lock)) -- 2.47.3 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v2 3/5] locking/rtmutex: Clone the sort keys in rt_mutex_enqueue_pi() 2026-11-08 14:13 ` [PATCH v2 3/5] locking/rtmutex: Clone the sort keys in rt_mutex_enqueue_pi() Mathieu Desnoyers @ 2026-11-16 15:11 ` Sebastian Andrzej Siewior 2026-11-16 18:35 ` Mathieu Desnoyers 0 siblings, 1 reply; 15+ messages in thread From: Sebastian Andrzej Siewior @ 2026-11-16 15:11 UTC (permalink / raw) To: Mathieu Desnoyers Cc: peterz, tglx, mingo, linux-kernel, will, boqun, longman, rostedt, clrkwllms, dvhart, dave, andrealmeid, shuah, juri.lelli, linux-rt-devel, linux-kselftest, stable On 2026-11-08 09:13:45 [-0500], Mathieu Desnoyers wrote: > An rt_mutex waiter has two copies of its sort keys. waiter->tree sorts > the waiters of its lock. waiter->pi_tree sorts the pi_waiters of the > lock owner. Only the top waiter of each lock is in pi_waiters. > > When the priority of a waiter changes, the chain walk updates > waiter->tree. It copies waiter->tree to waiter->pi_tree for the top > waiter only. A waiter that is not the top waiter keeps its old pi_tree > keys. > > Such a waiter can become the top waiter later. remove_waiter() does this > when the top waiter leaves. try_to_take_rt_mutex() does it when the lock > changes owner. Both put the new top waiter in pi_waiters without a new > copy. So pi_waiters sorts it by a priority it no longer has. The owner > then inherits from the wrong waiter, at that waiter's current priority. > > In this example, O is the owner, a SCHED_OTHER task. L1 and L2 are two > rt_mutexes. W1, W2 and W3 are waiters. Their SCHED_FIFO priority is > under their name. T is any other task. It changes the priority of W2: > > T O W1 W2 W3 > FIFO 80 FIFO 70 FIFO 50 > > locks L1, L2 > blocks on L2 > blocks on L1 (timed) > blocks on L1 > sched_setscheduler(W2) > W2 now at FIFO 10 > rt_mutex_adjust_pi(W2) > W2->tree: FIFO 10 > W2 not top: no copy > W2->pi_tree: FIFO 70 > <timeout> > remove_waiter() > rt_mutex_enqueue_pi(O, W2) > W2 sorted as FIFO 70, > ahead of W3 > rt_mutex_adjust_prio(O) > O inherits W2's FIFO 10 > runs at FIFO 10 still waits > > The owner runs at SCHED_FIFO 10 while a SCHED_FIFO 50 task waits for > it. This needs no race. Each step can come long after the one before. > > Fix this by copying the keys in rt_mutex_enqueue_pi(). All callers hold > the wait_lock and the owner's pi_lock, as waiter_clone_prio() requires. > Remove the three copies the callers made. > > With this patch, the owner runs at SCHED_FIFO 50. The > futex_lock_pi_inherit selftest added later in this series tests this > case (promoted_waiter). > > Fixes: f7853c342418 ("locking/rtmutex: Fix task->pi_waiters integrity") > Cc: stable@vger.kernel.org > Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> For this case, we miss one waiter_clone_prio() in remove_waiter(). Given how this is used, I don't see nothing wrong with moving before it is actually used. It might otherwise race in task_blocks_on_rt_mutex() since the pi-lock is dropped. I somehow assumed the adjust-prio will fix this. Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Sebastian ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2 3/5] locking/rtmutex: Clone the sort keys in rt_mutex_enqueue_pi() 2026-11-16 15:11 ` Sebastian Andrzej Siewior @ 2026-11-16 18:35 ` Mathieu Desnoyers 0 siblings, 0 replies; 15+ messages in thread From: Mathieu Desnoyers @ 2026-11-16 18:35 UTC (permalink / raw) To: Sebastian Andrzej Siewior Cc: peterz, tglx, mingo, linux-kernel, will, boqun, longman, rostedt, clrkwllms, dvhart, dave, andrealmeid, shuah, juri.lelli, linux-rt-devel, linux-kselftest, stable Hi Sebastian, > Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Thanks. I will add your tag in the next version. Mathieu -- Mathieu Desnoyers EfficiOS Inc. https://www.efficios.com ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2026-11-18 18:51 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <20261108141347.1525532-1-mathieu.desnoyers@efficios.com>
2026-11-08 14:13 ` [PATCH v2 1/5] locking/rtmutex: Stop spinning on the owner when the waiter is woken Mathieu Desnoyers
2026-11-16 12:06 ` Sebastian Andrzej Siewior
2026-11-16 18:36 ` Mathieu Desnoyers
2026-11-18 14:19 ` Sebastian Andrzej Siewior
2026-11-18 17:27 ` Mathieu Desnoyers
2026-11-18 18:51 ` Sebastian Andrzej Siewior
2026-11-08 14:13 ` [PATCH v2 2/5] locking/rtmutex: Initialize the ww_mutex context of a waiter Mathieu Desnoyers
2026-11-16 13:58 ` Sebastian Andrzej Siewior
2026-11-16 18:37 ` Mathieu Desnoyers
2026-11-18 14:40 ` Sebastian Andrzej Siewior
2026-11-18 17:28 ` Mathieu Desnoyers
2026-11-18 18:49 ` Sebastian Andrzej Siewior
2026-11-08 14:13 ` [PATCH v2 3/5] locking/rtmutex: Clone the sort keys in rt_mutex_enqueue_pi() Mathieu Desnoyers
2026-11-16 15:11 ` Sebastian Andrzej Siewior
2026-11-16 18:35 ` Mathieu Desnoyers
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox