From: Thomas Gleixner <tglx@linutronix.de>
To: Peter Zijlstra <peterz@infradead.org>
Cc: David Woodhouse <dwmw2@infradead.org>,
Stefan Hajnoczi <stefanha@redhat.com>,
Jason Wang <jasowang@redhat.com>,
"x86@kernel.org" <x86@kernel.org>, hpa <hpa@zytor.com>,
dyoung <dyoung@redhat.com>, kexec <kexec@lists.infradead.org>,
linux-ext4 <linux-ext4@vger.kernel.org>,
"Michael S. Tsirkin" <mst@redhat.com>,
Stefano Garzarella <sgarzare@redhat.com>,
eperezma <eperezma@redhat.com>,
Paolo Bonzini <bonzini@redhat.com>,
ming.lei@redhat.com, Petr Mladek <pmladek@suse.com>,
John Ogness <jogness@linutronix.de>
Subject: [PATCH] sched: Prevent rescheduling when interrupts are disabled
Date: Mon, 16 Dec 2024 14:20:56 +0100 [thread overview]
Message-ID: <87a5cv932f.ffs@tglx> (raw)
In-Reply-To: <87seqr914v.ffs@tglx>
David reported a warning observed while loop testing kexec jump:
Interrupts enabled after irqrouter_resume+0x0/0x50
WARNING: CPU: 0 PID: 560 at drivers/base/syscore.c:103 syscore_resume+0x18a/0x220
kernel_kexec+0xf6/0x180
__do_sys_reboot+0x206/0x250
do_syscall_64+0x95/0x180
The corresponding interrupt flag trace:
hardirqs last enabled at (15573): [<ffffffffa8281b8e>] __up_console_sem+0x7e/0x90
hardirqs last disabled at (15580): [<ffffffffa8281b73>] __up_console_sem+0x63/0x90
That means __up_console_sem() was invoked with interrupts enabled. Further
instrumentation revealed that in the interrupt disabled section of kexec
jump one of the syscore_suspend() callbacks woke up a task, which set the
NEED_RESCHED flag. A later callback in the resume path invoked
cond_resched() which in turn led to the invocation of the scheduler:
__cond_resched+0x21/0x60
down_timeout+0x18/0x60
acpi_os_wait_semaphore+0x4c/0x80
acpi_ut_acquire_mutex+0x3d/0x100
acpi_ns_get_node+0x27/0x60
acpi_ns_evaluate+0x1cb/0x2d0
acpi_rs_set_srs_method_data+0x156/0x190
acpi_pci_link_set+0x11c/0x290
irqrouter_resume+0x54/0x60
syscore_resume+0x6a/0x200
kernel_kexec+0x145/0x1c0
__do_sys_reboot+0xeb/0x240
do_syscall_64+0x95/0x180
This is a long standing problem, which probably got more visible with
the recent printk changes. Something does a task wakeup and the
scheduler sets the NEED_RESCHED flag. cond_resched() sees it set and
invokes schedule() from a completely bogus context. The scheduler
enables interrupts after context switching, which causes the above
warning at the end.
Quite some of the code paths in syscore_suspend()/resume() can result in
triggering a wakeup with the exactly same consequences. They might not
have done so yet, but as they share a lot of code with normal operations
it's just a question of time.
The problem only affects the PREEMPT_NONE and PREEMPT_VOLUNTARY scheduling
models. Full preemption is not affected as cond_resched() is disabled and
the preemption check preemptible() takes the interrupt disabled flag into
account.
Cure the problem by adding a corresponding check into cond_resched().
Reported-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: David Woodhouse <dwmw2@infradead.org>
Cc: stable@vger.kernel.org
Closes: https://lore.kernel.org/all/7717fe2ac0ce5f0a2c43fdab8b11f4483d54a2a4.camel@infradead.org
---
kernel/sched/core.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -7276,7 +7276,7 @@ void rt_mutex_setprio(struct task_struct
#if !defined(CONFIG_PREEMPTION) || defined(CONFIG_PREEMPT_DYNAMIC)
int __sched __cond_resched(void)
{
- if (should_resched(0)) {
+ if (should_resched(0) && !irqs_disabled()) {
preempt_schedule_common();
return 1;
}
next prev parent reply other threads:[~2024-12-16 13:20 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-12-09 14:28 Lockdep warnings on kexec (virtio_blk, hrtimers) David Woodhouse
2024-12-10 1:56 ` Jason Wang
2024-12-11 12:42 ` Stefan Hajnoczi
2024-12-12 11:07 ` David Woodhouse
2024-12-12 13:34 ` Thomas Gleixner
2024-12-12 13:46 ` David Woodhouse
2024-12-12 18:04 ` Thomas Gleixner
2024-12-12 19:19 ` David Woodhouse
2024-12-13 0:14 ` Thomas Gleixner
2024-12-13 9:31 ` David Woodhouse
2024-12-13 9:43 ` David Woodhouse
2024-12-13 10:42 ` Thomas Gleixner
2024-12-13 11:09 ` Ming Lei
2024-12-13 11:31 ` Thomas Gleixner
2024-12-13 11:48 ` Ming Lei
2024-12-13 13:23 ` Thomas Gleixner
2024-12-13 14:07 ` David Woodhouse
2024-12-13 17:05 ` Thomas Gleixner
2024-12-13 17:17 ` David Woodhouse
2024-12-13 17:48 ` Rafael J. Wysocki
2024-12-13 17:32 ` Rafael J. Wysocki
2024-12-13 19:06 ` Rafael J. Wysocki
2024-12-13 20:16 ` David Woodhouse
2024-12-14 9:57 ` David Woodhouse
2024-12-16 12:14 ` Rafael J. Wysocki
2024-12-13 17:59 ` Rafael J. Wysocki
2024-12-13 13:17 ` David Woodhouse
2024-12-13 11:12 ` David Woodhouse
2024-12-13 11:33 ` Ming Lei
2024-12-13 11:20 ` Peter Zijlstra
2024-12-13 13:13 ` Thomas Gleixner
2024-12-16 13:20 ` Thomas Gleixner [this message]
2024-12-16 17:41 ` [PATCH] sched: Prevent rescheduling when interrupts are disabled David Woodhouse
2024-12-12 11:12 ` Lockdep warnings on kexec (virtio_blk, hrtimers) Ming Lei
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87a5cv932f.ffs@tglx \
--to=tglx@linutronix.de \
--cc=bonzini@redhat.com \
--cc=dwmw2@infradead.org \
--cc=dyoung@redhat.com \
--cc=eperezma@redhat.com \
--cc=hpa@zytor.com \
--cc=jasowang@redhat.com \
--cc=jogness@linutronix.de \
--cc=kexec@lists.infradead.org \
--cc=linux-ext4@vger.kernel.org \
--cc=ming.lei@redhat.com \
--cc=mst@redhat.com \
--cc=peterz@infradead.org \
--cc=pmladek@suse.com \
--cc=sgarzare@redhat.com \
--cc=stefanha@redhat.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).