From: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>
To: John Ogness <john.ogness@linutronix.de>
Cc: Petr Mladek <pmladek@suse.com>, Nigel Croxon <ncroxon@redhat.com>,
"Theodore Y. Ts'o" <tytso@mit.edu>,
Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
Steven Rostedt <rostedt@goodmis.org>,
Sergey Senozhatsky <sergey.senozhatsky@gmail.com>,
dm-devel@redhat.com, Mikulas Patocka <mpatocka@redhat.com>,
linux-serial@vger.kernel.org
Subject: Re: Serial console is causing system lock-up
Date: Tue, 12 Mar 2019 17:59:34 +0900 [thread overview]
Message-ID: <20190312085934.GA1383@jagdpanzerIV> (raw)
In-Reply-To: <87a7i05wwi.fsf@linutronix.de>
On (03/12/19 09:17), John Ogness wrote:
> > wait M times (N - 1). Sounds quadratic.
>
> If these are critical messages, then we are _not allowed to drop any_!
> For critical messages printk must be synchronous. Thus for critical
> messages the situation you illustrated is appropriate.
>
> > 40) goto 10
> >
> > So I have some doubts regarding some of assumptions behind new printk
> > design. And the problem is not in prb_lock() unfairness. Current
> > printk design does look to me SMP-friendly; yes, it has unbound
> > printing loop; that can be addressed.
>
> Let us not forget, it deadlocked the machine. That's the reason this
> thread exists.
It didn't deadlock the machine. It was a typical soft lockup. Printing
CPU loop-ed in console_unlock() with preemption disabled; soft lockup
hrtimer was running on that CPU, but due to disabled preemption around
console_unlock() soft lockup's per-CPU kthread could not get scheduled
and could not update per-CPU touch_ts. Soft lockup hrtimer detected it:
[ 5128.552442] watchdog: BUG: soft lockup - CPU#9 stuck for 23s! [kworker/9:53:4131]
Along with that RCU was not able to get scheduled. Which was detected
by RCU stall detector:
[ 4891.199009] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[ 4891.221308] device-mapper: integrity: Checksum failed at sector 0x118d4f
[ 4891.251366] rcu: 9-....: (1923 ticks this GP) idle=7fa/1/0x4000000000000002 softirq=2190/2190 fqs=15013
[ 4891.251367] rcu: (detected by 16, t=60054 jiffies, g=24641, q=351)
[ 4891.311941] Sending NMI from CPU 16 to CPUs 9:
[..]
> 2. You seem unwilling to acknowledge the difference between emergency
> and informational messages. A message is either critical or it is
> not. If it is, it should be handled as such, regardless of
> interference, regardless if it means turning an SMP machine into a UP
> machine. If it is not critical, it should be sent along a
> non-interfering path so the the system is _not_ affected.
OK.
Let's move on then.
-ss
next prev parent reply other threads:[~2019-03-12 8:59 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-03-06 14:27 Serial console is causing system lock-up Mikulas Patocka
2019-03-06 15:22 ` Petr Mladek
2019-03-06 16:07 ` Mikulas Patocka
2019-03-06 16:30 ` Theodore Y. Ts'o
2019-03-06 17:11 ` Mikulas Patocka
2019-03-06 22:19 ` Steven Rostedt
2019-03-06 22:43 ` John Ogness
2019-03-07 2:22 ` Sergey Senozhatsky
2019-03-07 8:17 ` John Ogness
2019-03-07 8:25 ` Sergey Senozhatsky
2019-03-07 8:34 ` John Ogness
2019-03-07 9:17 ` Sergey Senozhatsky
2019-03-07 10:37 ` John Ogness
2019-03-07 12:26 ` Sergey Senozhatsky
2019-03-07 12:54 ` Mikulas Patocka
2019-03-07 14:21 ` John Ogness
2019-03-07 15:35 ` Petr Mladek
2019-03-12 2:32 ` Sergey Senozhatsky
2019-03-12 8:17 ` John Ogness
2019-03-12 8:59 ` Sergey Senozhatsky [this message]
2019-03-12 10:05 ` Mikulas Patocka
2019-03-12 13:19 ` John Ogness
2019-03-12 13:44 ` Petr Mladek
2019-03-12 12:08 ` Petr Mladek
2019-03-12 15:19 ` John Ogness
2019-03-13 2:38 ` Sergey Senozhatsky
2019-03-13 8:43 ` John Ogness
2019-03-14 10:30 ` Sergey Senozhatsky
2019-03-07 14:08 ` John Stoffel
2019-03-07 14:26 ` Mikulas Patocka
2019-03-08 1:22 ` Sergey Senozhatsky
2019-03-08 1:39 ` Sergey Senozhatsky
2019-03-08 2:36 ` John Ogness
2019-03-07 15:16 ` Petr Mladek
2019-03-07 1:56 ` Sergey Senozhatsky
2019-03-07 13:12 ` Mikulas Patocka
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190312085934.GA1383@jagdpanzerIV \
--to=sergey.senozhatsky.work@gmail.com \
--cc=dm-devel@redhat.com \
--cc=gregkh@linuxfoundation.org \
--cc=john.ogness@linutronix.de \
--cc=linux-serial@vger.kernel.org \
--cc=mpatocka@redhat.com \
--cc=ncroxon@redhat.com \
--cc=pmladek@suse.com \
--cc=rostedt@goodmis.org \
--cc=sergey.senozhatsky@gmail.com \
--cc=tytso@mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox