From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
stable@vger.kernel.org, Steven Rostedt <rostedt@goodmis.org>,
Daniel Wang <wonderfly@google.com>,
Peter Zijlstra <peterz@infradead.org>,
Andrew Morton <akpm@linux-foundation.org>,
Linus Torvalds <torvalds@linux-foundation.org>,
Alan Cox <gnomes@lxorguk.ukuu.org.uk>,
Jiri Slaby <jslaby@suse.com>, Peter Feiner <pfeiner@google.com>,
linux-serial@vger.kernel.org,
Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>,
Sergey Senozhatsky <sergey.senozhatsky@gmail.com>,
Petr Mladek <pmladek@suse.com>
Subject: [PATCH 4.19 30/46] panic: avoid deadlocks in re-entrant console drivers
Date: Fri, 28 Dec 2018 12:52:24 +0100 [thread overview]
Message-ID: <20181228113126.577353246@linuxfoundation.org> (raw)
In-Reply-To: <20181228113124.971620049@linuxfoundation.org>
4.19-stable review patch. If anyone has any objections, please let me know.
------------------
From: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>
commit c7c3f05e341a9a2bd1a92993d4f996cfd6e7348e upstream.
>From printk()/serial console point of view panic() is special, because
it may force CPU to re-enter printk() or/and serial console driver.
Therefore, some of serial consoles drivers are re-entrant. E.g. 8250:
serial8250_console_write()
{
if (port->sysrq)
locked = 0;
else if (oops_in_progress)
locked = spin_trylock_irqsave(&port->lock, flags);
else
spin_lock_irqsave(&port->lock, flags);
...
}
panic() does set oops_in_progress via bust_spinlocks(1), so in theory
we should be able to re-enter serial console driver from panic():
CPU0
<NMI>
uart_console_write()
serial8250_console_write() // if (oops_in_progress)
// spin_trylock_irqsave()
call_console_drivers()
console_unlock()
console_flush_on_panic()
bust_spinlocks(1) // oops_in_progress++
panic()
<NMI/>
spin_lock_irqsave(&port->lock, flags) // spin_lock_irqsave()
serial8250_console_write()
call_console_drivers()
console_unlock()
printk()
...
However, this does not happen and we deadlock in serial console on
port->lock spinlock. And the problem is that console_flush_on_panic()
called after bust_spinlocks(0):
void panic(const char *fmt, ...)
{
bust_spinlocks(1);
...
bust_spinlocks(0);
console_flush_on_panic();
...
}
bust_spinlocks(0) decrements oops_in_progress, so oops_in_progress
can go back to zero. Thus even re-entrant console drivers will simply
spin on port->lock spinlock. Given that port->lock may already be
locked either by a stopped CPU, or by the very same CPU we execute
panic() on (for instance, NMI panic() on printing CPU) the system
deadlocks and does not reboot.
Fix this by removing bust_spinlocks(0), so oops_in_progress is always
set in panic() now and, thus, re-entrant console drivers will trylock
the port->lock instead of spinning on it forever, when we call them
from console_flush_on_panic().
Link: http://lkml.kernel.org/r/20181025101036.6823-1-sergey.senozhatsky@gmail.com
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Daniel Wang <wonderfly@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Alan Cox <gnomes@lxorguk.ukuu.org.uk>
Cc: Jiri Slaby <jslaby@suse.com>
Cc: Peter Feiner <pfeiner@google.com>
Cc: linux-serial@vger.kernel.org
Cc: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>
Cc: stable@vger.kernel.org
Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Signed-off-by: Petr Mladek <pmladek@suse.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
kernel/panic.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
--- a/kernel/panic.c
+++ b/kernel/panic.c
@@ -14,6 +14,7 @@
#include <linux/kmsg_dump.h>
#include <linux/kallsyms.h>
#include <linux/notifier.h>
+#include <linux/vt_kern.h>
#include <linux/module.h>
#include <linux/random.h>
#include <linux/ftrace.h>
@@ -233,7 +234,10 @@ void panic(const char *fmt, ...)
if (_crash_kexec_post_notifiers)
__crash_kexec(NULL);
- bust_spinlocks(0);
+#ifdef CONFIG_VT
+ unblank_screen();
+#endif
+ console_unblank();
/*
* We may have ended up stopping the CPU holding the lock (in
next prev parent reply other threads:[~2018-12-28 12:14 UTC|newest]
Thread overview: 55+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-12-28 11:51 [PATCH 4.19 00/46] 4.19.13-stable review Greg Kroah-Hartman
2018-12-28 11:51 ` [PATCH 4.19 01/46] iomap: Revert "fs/iomap.c: get/put the page in iomap_page_create/release()" Greg Kroah-Hartman
2018-12-28 11:51 ` [PATCH 4.19 02/46] Revert "vfs: Allow userns root to call mknod on owned filesystems." Greg Kroah-Hartman
2018-12-28 11:51 ` [PATCH 4.19 03/46] USB: hso: Fix OOB memory access in hso_probe/hso_get_config_data Greg Kroah-Hartman
2018-12-28 11:51 ` [PATCH 4.19 04/46] xhci: Dont prevent USB2 bus suspend in state check intended for USB3 only Greg Kroah-Hartman
2018-12-28 11:51 ` [PATCH 4.19 05/46] USB: xhci: fix broken_suspend placement in struct xchi_hcd Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 06/46] USB: serial: option: add GosunCn ZTE WeLink ME3630 Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 07/46] USB: serial: option: add HP lt4132 Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 08/46] USB: serial: option: add Simcom SIM7500/SIM7600 (MBIM mode) Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 09/46] USB: serial: option: add Fibocom NL668 series Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 10/46] USB: serial: option: add Telit LN940 series Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 11/46] ubifs: Handle re-linking of inodes correctly while recovery Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 12/46] scsi: t10-pi: Return correct ref tag when queue has no integrity profile Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 13/46] scsi: sd: use mempool for discard special page Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 14/46] mmc: core: Reset HPI enabled state during re-init and in case of errors Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 15/46] mmc: core: Allow BKOPS and CACHE ctrl even if no HPI support Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 16/46] mmc: core: Use a minimum 1600ms timeout when enabling CACHE ctrl Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 17/46] mmc: omap_hsmmc: fix DMA API warning Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 18/46] gpio: max7301: fix driver for use with CONFIG_VMAP_STACK Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 19/46] gpiolib-acpi: Only defer request_irq for GpioInt ACPI event handlers Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 20/46] posix-timers: Fix division by zero bug Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 21/46] KVM: X86: Fix NULL deref in vcpu_scan_ioapic Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 22/46] kvm: x86: Add AMDs EX_CFG to the list of ignored MSRs Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 23/46] KVM: Fix UAF in nested posted interrupt processing Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 24/46] Drivers: hv: vmbus: Return -EINVAL for the sys files for unopened channels Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 25/46] futex: Cure exit race Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 26/46] x86/mtrr: Dont copy uninitialized gentry fields back to userspace Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 27/46] x86/mm: Fix decoy address handling vs 32-bit builds Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 28/46] x86/vdso: Pass --eh-frame-hdr to the linker Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 29/46] x86/intel_rdt: Ensure a CPU remains online for the regions pseudo-locking sequence Greg Kroah-Hartman
2018-12-28 11:52 ` Greg Kroah-Hartman [this message]
2018-12-28 11:52 ` [PATCH 4.19 31/46] mm: add mm_pxd_folded checks to pgtable_bytes accounting functions Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 32/46] mm: make the __PAGETABLE_PxD_FOLDED defines non-empty Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 33/46] mm: introduce mm_[p4d|pud|pmd]_folded Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 34/46] xfrm_user: fix freeing of xfrm states on acquire Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 35/46] rtlwifi: Fix leak of skb when processing C2H_BT_INFO Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 36/46] iwlwifi: mvm: dont send GEO_TX_POWER_LIMIT to old firmwares Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 37/46] Revert "mwifiex: restructure rx_reorder_tbl_lock usage" Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 38/46] iwlwifi: add new cards for 9560, 9462, 9461 and killer series Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 39/46] media: ov5640: Fix set format regression Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 40/46] mm, memory_hotplug: initialize struct pages for the full memory section Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 41/46] mm: thp: fix flags for pmd migration when split Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 42/46] mm, page_alloc: fix has_unmovable_pages for HugePages Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 43/46] mm: dont miss the last page because of round-off error Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 44/46] Input: elantech - disable elan-i2c for P52 and P72 Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 45/46] proc/sysctl: dont return ENOMEM on lookup when a table is unregistering Greg Kroah-Hartman
2018-12-28 11:52 ` [PATCH 4.19 46/46] drm/ioctl: Fix Spectre v1 vulnerabilities Greg Kroah-Hartman
2018-12-28 17:54 ` [PATCH 4.19 00/46] 4.19.13-stable review Dan Rue
2018-12-29 12:20 ` Greg Kroah-Hartman
2018-12-28 20:08 ` shuah
2018-12-29 9:55 ` Greg Kroah-Hartman
2018-12-28 21:29 ` Guenter Roeck
2018-12-29 12:20 ` Greg Kroah-Hartman
2018-12-29 9:01 ` Harsh Shandilya
2018-12-29 12:20 ` Greg Kroah-Hartman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20181228113126.577353246@linuxfoundation.org \
--to=gregkh@linuxfoundation.org \
--cc=akpm@linux-foundation.org \
--cc=gnomes@lxorguk.ukuu.org.uk \
--cc=jslaby@suse.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-serial@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=pfeiner@google.com \
--cc=pmladek@suse.com \
--cc=rostedt@goodmis.org \
--cc=sergey.senozhatsky.work@gmail.com \
--cc=sergey.senozhatsky@gmail.com \
--cc=stable@vger.kernel.org \
--cc=torvalds@linux-foundation.org \
--cc=wonderfly@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).