From: Thomas Gleixner <tglx@linutronix.de>
To: Peter Zijlstra <peterz@infradead.org>
Cc: David Woodhouse <dwmw2@infradead.org>,
Stefan Hajnoczi <stefanha@redhat.com>,
Jason Wang <jasowang@redhat.com>,
"x86@kernel.org" <x86@kernel.org>, hpa <hpa@zytor.com>,
dyoung <dyoung@redhat.com>, kexec <kexec@lists.infradead.org>,
linux-ext4 <linux-ext4@vger.kernel.org>,
"Michael S. Tsirkin" <mst@redhat.com>,
Stefano Garzarella <sgarzare@redhat.com>,
eperezma <eperezma@redhat.com>,
Paolo Bonzini <bonzini@redhat.com>,
ming.lei@redhat.com, Petr Mladek <pmladek@suse.com>,
John Ogness <jogness@linutronix.de>
Subject: [PATCH] sched: Prevent rescheduling when interrupts are disabled
Date: Mon, 16 Dec 2024 14:20:56 +0100 [thread overview]
Message-ID: <87a5cv932f.ffs@tglx> (raw)
In-Reply-To: <87seqr914v.ffs@tglx>
David reported a warning observed while loop testing kexec jump:
Interrupts enabled after irqrouter_resume+0x0/0x50
WARNING: CPU: 0 PID: 560 at drivers/base/syscore.c:103 syscore_resume+0x18a/0x220
kernel_kexec+0xf6/0x180
__do_sys_reboot+0x206/0x250
do_syscall_64+0x95/0x180
The corresponding interrupt flag trace:
hardirqs last enabled at (15573): [<ffffffffa8281b8e>] __up_console_sem+0x7e/0x90
hardirqs last disabled at (15580): [<ffffffffa8281b73>] __up_console_sem+0x63/0x90
That means __up_console_sem() was invoked with interrupts enabled. Further
instrumentation revealed that in the interrupt disabled section of kexec
jump one of the syscore_suspend() callbacks woke up a task, which set the
NEED_RESCHED flag. A later callback in the resume path invoked
cond_resched() which in turn led to the invocation of the scheduler:
__cond_resched+0x21/0x60
down_timeout+0x18/0x60
acpi_os_wait_semaphore+0x4c/0x80
acpi_ut_acquire_mutex+0x3d/0x100
acpi_ns_get_node+0x27/0x60
acpi_ns_evaluate+0x1cb/0x2d0
acpi_rs_set_srs_method_data+0x156/0x190
acpi_pci_link_set+0x11c/0x290
irqrouter_resume+0x54/0x60
syscore_resume+0x6a/0x200
kernel_kexec+0x145/0x1c0
__do_sys_reboot+0xeb/0x240
do_syscall_64+0x95/0x180
This is a long standing problem, which probably got more visible with
the recent printk changes. Something does a task wakeup and the
scheduler sets the NEED_RESCHED flag. cond_resched() sees it set and
invokes schedule() from a completely bogus context. The scheduler
enables interrupts after context switching, which causes the above
warning at the end.
Quite some of the code paths in syscore_suspend()/resume() can result in
triggering a wakeup with the exactly same consequences. They might not
have done so yet, but as they share a lot of code with normal operations
it's just a question of time.
The problem only affects the PREEMPT_NONE and PREEMPT_VOLUNTARY scheduling
models. Full preemption is not affected as cond_resched() is disabled and
the preemption check preemptible() takes the interrupt disabled flag into
account.
Cure the problem by adding a corresponding check into cond_resched().
Reported-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: David Woodhouse <dwmw2@infradead.org>
Cc: stable@vger.kernel.org
Closes: https://lore.kernel.org/all/7717fe2ac0ce5f0a2c43fdab8b11f4483d54a2a4.camel@infradead.org
---
kernel/sched/core.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -7276,7 +7276,7 @@ void rt_mutex_setprio(struct task_struct
#if !defined(CONFIG_PREEMPTION) || defined(CONFIG_PREEMPT_DYNAMIC)
int __sched __cond_resched(void)
{
- if (should_resched(0)) {
+ if (should_resched(0) && !irqs_disabled()) {
preempt_schedule_common();
return 1;
}
next prev parent reply other threads:[~2024-12-16 13:21 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-12-09 14:28 Lockdep warnings on kexec (virtio_blk, hrtimers) David Woodhouse
2024-12-10 1:56 ` Jason Wang
2024-12-11 12:42 ` Stefan Hajnoczi
2024-12-12 11:07 ` David Woodhouse
2024-12-12 13:34 ` Thomas Gleixner
2024-12-12 13:46 ` David Woodhouse
2024-12-12 18:04 ` Thomas Gleixner
2024-12-12 19:19 ` David Woodhouse
2024-12-13 0:14 ` Thomas Gleixner
2024-12-13 9:31 ` David Woodhouse
2024-12-13 9:43 ` David Woodhouse
2024-12-13 10:42 ` Thomas Gleixner
2024-12-13 11:09 ` Ming Lei
2024-12-13 11:31 ` Thomas Gleixner
2024-12-13 11:48 ` Ming Lei
2024-12-13 13:23 ` Thomas Gleixner
2024-12-13 14:07 ` David Woodhouse
2024-12-13 17:05 ` Thomas Gleixner
2024-12-13 17:17 ` David Woodhouse
2024-12-13 17:48 ` Rafael J. Wysocki
2024-12-13 17:32 ` Rafael J. Wysocki
2024-12-13 19:06 ` Rafael J. Wysocki
2024-12-13 20:16 ` David Woodhouse
2024-12-14 9:57 ` David Woodhouse
2024-12-16 12:14 ` Rafael J. Wysocki
2024-12-13 17:59 ` Rafael J. Wysocki
2024-12-13 13:17 ` David Woodhouse
2024-12-13 11:12 ` David Woodhouse
2024-12-13 11:33 ` Ming Lei
2024-12-13 11:20 ` Peter Zijlstra
2024-12-13 13:13 ` Thomas Gleixner
2024-12-16 13:20 ` Thomas Gleixner [this message]
2024-12-16 17:41 ` [PATCH] sched: Prevent rescheduling when interrupts are disabled David Woodhouse
2024-12-12 11:12 ` Lockdep warnings on kexec (virtio_blk, hrtimers) Ming Lei
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87a5cv932f.ffs@tglx \
--to=tglx@linutronix.de \
--cc=bonzini@redhat.com \
--cc=dwmw2@infradead.org \
--cc=dyoung@redhat.com \
--cc=eperezma@redhat.com \
--cc=hpa@zytor.com \
--cc=jasowang@redhat.com \
--cc=jogness@linutronix.de \
--cc=kexec@lists.infradead.org \
--cc=linux-ext4@vger.kernel.org \
--cc=ming.lei@redhat.com \
--cc=mst@redhat.com \
--cc=peterz@infradead.org \
--cc=pmladek@suse.com \
--cc=sgarzare@redhat.com \
--cc=stefanha@redhat.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.