[PATCH v2] rseq: fix hard lockup on granted time slice extension

[PATCH v2] rseq: fix hard lockup on granted time slice extension

[Niels Pressel]

In __exit_to_user_mode_loop(), TSE eligibility is checked while
IRQs are enabled. Granting a TSE might involve rearming the
hrtimers. However, hrtimer_rearm_deferred_tif() is expected to be
called with IRQs disabled (see include/linux/hrtimer_rearm.h:17).

Calling the function with IRQs enabled can lead to a hard lockup
because __hrtimer_rearm_deferred() acquires a raw spinlock (without
disabling IRQs) that is also acquired in hard IRQ context within
hrtimer_run_queues().

Lockdep flags the issue when running the rseq selftests on the
7.2-rc5 release:

    WARNING: ./include/linux/hrtimer_rearm.h:17 at irqentry_exit, CPU#1: slice_test

    ================================
    WARNING: inconsistent lock state
    7.2.0-rc5 #1 Tainted: G        W
    --------------------------------
    inconsistent {IN-HARDIRQ-W} -> {HARDIRQ-ON-W} usage.
    slice_test [HC0[0]:SC0[0]:HE1:SE1] takes:
    ffff95b82ec5c698 (hrtimer_bases.lock){?.-.}-{2:2}, at: __hrtimer_rearm_deferred
    {IN-HARDIRQ-W} state was registered at:
    lock_acquire
    _raw_spin_lock_irqsave
    hrtimer_run_queues
    update_process_times
    tick_periodic
    tick_handle_periodic
    timer_interrupt
    __handle_irq_event_percpu
    handle_irq_event_percpu
    handle_irq_event
    handle_level_irq
    __common_interrupt
    common_interrupt
    asm_common_interrupt
    _raw_spin_unlock_irqrestore
    __setup_irq
    request_threaded_irq
    hpet_time_init
    x86_late_time_init
    start_kernel
    x86_64_start_reservations
    x86_64_start_kernel
    common_startup_64

    Possible unsafe locking scenario:

        CPU0
        ----
    lock(hrtimer_bases.lock);
    <Interrupt>
        lock(hrtimer_bases.lock);

                    *** DEADLOCK ***

    Call Trace:
    <TASK>
    dump_stack_lvl
    print_usage_bug
    mark_lock.part.0
    __lock_acquire
    lock_acquire
    _raw_spin_lock
    __hrtimer_rearm_deferred
    irqentry_exit
    asm_sysvec_apic_timer_interrupt
    </TASK>

Originally, the issue was discovered because of intermittent lockups
when heavily using rseq TSEs.

Following the suggestion from Peter Zijlstra, fix this potential lockup
by reflowing __exit_to_user_mode_loop() to only enable IRQs after the
TSE check.

Fixes: 15dd3a948855 ("hrtimer: Push reprogramming timers into the interrupt return path")
Signed-off-by: Niels Pressel <npressel@ethz.ch>
---
Changes in v2:
- Move to enabling IRQS in __exit_to_user_mode_loop() after the rseq 
  TSE check
- Remove IRQ disable in __rseq_grant_slice_extension()
- Rebased on core/entry
- Link to v1: https://lore.kernel.org/all/20260802124423.51616-1-npressel@ethz.ch/

 include/linux/rseq_entry.h |  8 ++++----
 kernel/entry/common.c      | 10 +++++-----
 2 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/include/linux/rseq_entry.h b/include/linux/rseq_entry.h
index ed9da6e41a2a..cb41b30c9cb9 100644
--- a/include/linux/rseq_entry.h
+++ b/include/linux/rseq_entry.h
@@ -127,6 +127,8 @@ do {									\
 
 static __always_inline bool __rseq_grant_slice_extension(bool work_pending)
 {
+	lockdep_assert_irqs_disabled();
+
 	struct task_struct *curr = current;
 	struct rseq_slice_ctrl usr_ctrl;
 	union rseq_slice_state state;
@@ -219,10 +221,8 @@ static __always_inline bool __rseq_grant_slice_extension(bool work_pending)
 	 *
 	 * which would be inconsistent state.
 	 */
-	scoped_guard(irq) {
-		clear_tsk_need_resched(curr);
-		clear_preempt_need_resched();
-	}
+	clear_tsk_need_resched(curr);
+	clear_preempt_need_resched();
 	return true;
 
 efault:
diff --git a/kernel/entry/common.c b/kernel/entry/common.c
index e3d381fd3d25..9cefb37f4dd4 100644
--- a/kernel/entry/common.c
+++ b/kernel/entry/common.c
@@ -47,13 +47,13 @@ static __always_inline unsigned long __exit_to_user_mode_loop(struct pt_regs *re
 	 * items have been completed.
 	 */
 	while (ti_work & EXIT_TO_USER_MODE_WORK_LOOP) {
-
+		/* Check rseq slice extensions with IRQs disabled */
+		bool sched = (ti_work & (_TIF_NEED_RESCHED | _TIF_NEED_RESCHED_LAZY)) &&
+			      !rseq_grant_slice_extension(ti_work, TIF_SLICE_EXT_DENY);
 		local_irq_enable();
 
-		if (ti_work & (_TIF_NEED_RESCHED | _TIF_NEED_RESCHED_LAZY)) {
-			if (!rseq_grant_slice_extension(ti_work, TIF_SLICE_EXT_DENY))
-				schedule();
-		}
+		if (sched)
+			schedule();
 
 		if (ti_work & _TIF_UPROBE)
 			uprobe_notify_resume(regs);

base-commit: 05c033db7e9ad3c34f6968ec568cb6ee01051c57
-- 
2.50.1 (Apple Git-155)

[tip: core/urgent] entry/rseq: Fix hard lockup on granted time slice extension

[tip-bot2 for Niels Pressel]

The following commit has been merged into the core/urgent branch of tip:

Commit-ID:     f81845889e128d2e5f8f2f38eb7339cc74640f4f
Gitweb:        https://git.kernel.org/tip/f81845889e128d2e5f8f2f38eb7339cc74640f4f
Author:        Niels Pressel <npressel@ethz.ch>
AuthorDate:    Thu, 06 Aug 2026 13:34:29 +02:00
Committer:     Peter Zijlstra <peterz@infradead.org>
CommitterDate: Thu, 06 Aug 2026 14:42:22 +02:00

entry/rseq: Fix hard lockup on granted time slice extension

In __exit_to_user_mode_loop(), TSE eligibility is checked while
IRQs are enabled. Granting a TSE might involve rearming the
hrtimers. However, hrtimer_rearm_deferred_tif() is expected to be
called with IRQs disabled (see include/linux/hrtimer_rearm.h:17).

Calling the function with IRQs enabled can lead to a hard lockup
because __hrtimer_rearm_deferred() acquires a raw spinlock (without
disabling IRQs) that is also acquired in hard IRQ context within
hrtimer_run_queues().

Lockdep flags the issue when running the rseq selftests on the
7.2-rc5 release:

    WARNING: ./include/linux/hrtimer_rearm.h:17 at irqentry_exit, CPU#1: slice_test

Originally, the issue was discovered because of intermittent lockups
when heavily using rseq TSEs.

Following the suggestion from Peter Zijlstra, fix this potential lockup
by reflowing __exit_to_user_mode_loop() to only enable IRQs after the
TSE check.

Fixes: 15dd3a948855 ("hrtimer: Push reprogramming timers into the interrupt return path")
Signed-off-by: Niels Pressel <npressel@ethz.ch>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260806113429.38333-1-npressel@ethz.ch
---
 include/linux/rseq_entry.h |  8 ++++----
 kernel/entry/common.c      | 10 +++++-----
 2 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/include/linux/rseq_entry.h b/include/linux/rseq_entry.h
index ed9da6e..772e72d 100644
--- a/include/linux/rseq_entry.h
+++ b/include/linux/rseq_entry.h
@@ -132,6 +132,8 @@ static __always_inline bool __rseq_grant_slice_extension(bool work_pending)
 	union rseq_slice_state state;
 	struct rseq __user *rseq;
 
+	lockdep_assert_irqs_disabled();
+
 	if (!rseq_slice_extension_enabled())
 		return false;
 
@@ -219,10 +221,8 @@ static __always_inline bool __rseq_grant_slice_extension(bool work_pending)
 	 *
 	 * which would be inconsistent state.
 	 */
-	scoped_guard(irq) {
-		clear_tsk_need_resched(curr);
-		clear_preempt_need_resched();
-	}
+	clear_tsk_need_resched(curr);
+	clear_preempt_need_resched();
 	return true;
 
 efault:
diff --git a/kernel/entry/common.c b/kernel/entry/common.c
index e3d381f..e7dae46 100644
--- a/kernel/entry/common.c
+++ b/kernel/entry/common.c
@@ -47,13 +47,13 @@ static __always_inline unsigned long __exit_to_user_mode_loop(struct pt_regs *re
 	 * items have been completed.
 	 */
 	while (ti_work & EXIT_TO_USER_MODE_WORK_LOOP) {
-
+		/* Check rseq slice extensions with IRQs disabled */
+		bool sched = (ti_work & (_TIF_NEED_RESCHED | _TIF_NEED_RESCHED_LAZY)) &&
+			     !rseq_grant_slice_extension(ti_work, TIF_SLICE_EXT_DENY);
 		local_irq_enable();
 
-		if (ti_work & (_TIF_NEED_RESCHED | _TIF_NEED_RESCHED_LAZY)) {
-			if (!rseq_grant_slice_extension(ti_work, TIF_SLICE_EXT_DENY))
-				schedule();
-		}
+		if (sched)
+			schedule();
 
 		if (ti_work & _TIF_UPROBE)
 			uprobe_notify_resume(regs);

Re: [tip: core/urgent] entry/rseq: Fix hard lockup on granted time slice extension

[Thomas Gleixner]

On Thu, Aug 06 2026 at 12:48, tip-bot wrote:
> In __exit_to_user_mode_loop(), TSE eligibility is checked while
> IRQs are enabled. Granting a TSE might involve rearming the
> hrtimers. However, hrtimer_rearm_deferred_tif() is expected to be
> called with IRQs disabled (see include/linux/hrtimer_rearm.h:17).

Groan. Can people please use proper words and not random made up
acronyms. This is not a SMS service. And this file reference is
more than pointless.

> Following the suggestion from Peter Zijlstra, fix this potential lockup
> by reflowing __exit_to_user_mode_loop() to only enable IRQs after the
> TSE check.

Which is broken.

> @@ -132,6 +132,8 @@ static __always_inline bool __rseq_grant_slice_extension(bool work_pending)
>  	union rseq_slice_state state;
>  	struct rseq __user *rseq;
>  
> +	lockdep_assert_irqs_disabled();
> +
>  	if (!rseq_slice_extension_enabled())
>  		return false;

Care to look what the code there does?

	rseq = curr->rseq.usrptr;
	scoped_user_rw_access(rseq, efault) {

That's user access which requires interrupts to be enabled.
  
Niels' original patch was correct.

To answer Peter's question from the V1 submission:

>> Thomas, previously we would call hrtimer_rearm_deferred() before
>> re-enabling IRQs, but here it slipped past. And while disabling it will
>> cure the splat, I'm thinking it makes sense to reflow
>> __exit_to_user_mode_loop() to instead delay enabling IRQs.

Yes, but then we consolidated all the schedule() hrtimer interaction and
got a benefit when schedule() was invoked directly on the way
out. That's why we ended up moving into into the success path
of rseq_grant_slice_extension() because that obviously skips schedule().

That's why we have hrtimer_rearm_deferred_user_irq() in
__exit_to_user_mode_prepare() _before_ invoking exit_to_user_mode_loop()
to ensure that one of the TIF_NEED_RESCHED bits is set. If not and if
TIF_HRTIMER_REARM is set, then the rearming happens right there with
interrupts still disabled.

If one of the TIF_NEED_RESCHED bits is set, then enabling interrupts
right on top of the loop is safe because any interrupt/exception entry
needs to check the bit as well.

So the thing [wm]e f*cked up royally was to invoke
hrtimer_rearm_deferred_tif() with interrupts enabled and that needs to
be fixed.

Of course all of this can be figured out from the copious amount of
comments which got added to explain all of this magic.

Thanks,

        tglx

Re: [tip: core/urgent] entry/rseq: Fix hard lockup on granted time slice extension

[Peter Zijlstra]

On Fri, Aug 07, 2026 at 11:15:19PM +0200, Thomas Gleixner wrote:
> > @@ -132,6 +132,8 @@ static __always_inline bool __rseq_grant_slice_extension(bool work_pending)
> >  	union rseq_slice_state state;
> >  	struct rseq __user *rseq;
> >  
> > +	lockdep_assert_irqs_disabled();
> > +
> >  	if (!rseq_slice_extension_enabled())
> >  		return false;
> 
> Care to look what the code there does?
> 
> 	rseq = curr->rseq.usrptr;
> 	scoped_user_rw_access(rseq, efault) {
> 
> That's user access which requires interrupts to be enabled.

Well, you can do user access with IRQs disabled just fine, you just get
more efault. But yes, this one really wants the fault handler to page in
stuff if it is so needed.. Moo :-(

> Niels' original patch was correct.
> 
> To answer Peter's question from the V1 submission:
> 
> >> Thomas, previously we would call hrtimer_rearm_deferred() before
> >> re-enabling IRQs, but here it slipped past. And while disabling it will
> >> cure the splat, I'm thinking it makes sense to reflow
> >> __exit_to_user_mode_loop() to instead delay enabling IRQs.
> 
> Yes, but then we consolidated all the schedule() hrtimer interaction and
> got a benefit when schedule() was invoked directly on the way
> out. That's why we ended up moving into into the success path
> of rseq_grant_slice_extension() because that obviously skips schedule().
> 
> That's why we have hrtimer_rearm_deferred_user_irq() in
> __exit_to_user_mode_prepare() _before_ invoking exit_to_user_mode_loop()
> to ensure that one of the TIF_NEED_RESCHED bits is set. If not and if
> TIF_HRTIMER_REARM is set, then the rearming happens right there with
> interrupts still disabled.
> 
> If one of the TIF_NEED_RESCHED bits is set, then enabling interrupts
> right on top of the loop is safe because any interrupt/exception entry
> needs to check the bit as well.
> 
> So the thing [wm]e f*cked up royally was to invoke
> hrtimer_rearm_deferred_tif() with interrupts enabled and that needs to
> be fixed.
> 
> Of course all of this can be figured out from the copious amount of
> comments which got added to explain all of this magic.

Bah, its spread out over too damn many functions is what :/ Anyway, I
see you've not yet pulled the patch, let me go do that now.

Re: [tip: core/urgent] entry/rseq: Fix hard lockup on granted time slice extension

[Thomas Gleixner]

On Sat, Aug 08 2026 at 10:36, Peter Zijlstra wrote:
> On Fri, Aug 07, 2026 at 11:15:19PM +0200, Thomas Gleixner wrote:
>> 	rseq = curr->rseq.usrptr;
>> 	scoped_user_rw_access(rseq, efault) {
>> 
>> That's user access which requires interrupts to be enabled.
>
> Well, you can do user access with IRQs disabled just fine, you just get
> more efault.

Sure, but then you need a manual fault handler, which requires that
interrupts are enabled :)

>> So the thing [wm]e f*cked up royally was to invoke
>> hrtimer_rearm_deferred_tif() with interrupts enabled and that needs to
>> be fixed.
>> 
>> Of course all of this can be figured out from the copious amount of
>> comments which got added to explain all of this magic.
>
> Bah, its spread out over too damn many functions is what :/

That and comments are missing at crucial places :(