From: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>
To: Zygo Blaxell <uixjjji1@umail.furryterror.org>
Cc: pmladek@suse.com, tytso@mit.edu,
sergey.senozhatsky.work@gmail.com, arnd@arndb.de,
peterz@infradead.org, gregkh@linuxfoundation.org,
linux-kernel@vger.kernel.org, rostedt@goodmis.org,
linux-mm@kvack.org, Qian Cai <cai@lca.pw>,
catalin.marinas@arm.com, longman@redhat.com,
dan.j.williams@intel.com, will@kernel.org, tglx@linutronix.de,
linux-arm-kernel@lists.infradead.org
Subject: Re: dmesg -w regression in v5.4.22, bisected, was: Re: [PATCH] char/random: silence a lockdep splat with printk()
Date: Wed, 25 Mar 2020 11:35:06 +0900 [thread overview]
Message-ID: <20200325023506.GB241329@google.com> (raw)
In-Reply-To: <20200324151359.GF2693@hungrycats.org>
On (20/03/24 11:13), Zygo Blaxell wrote:
> On Wed, Nov 13, 2019 at 04:16:25PM -0500, Qian Cai wrote:
> > From: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>
> >
> > Sergey didn't like the locking order,
> >
> > uart_port->lock -> tty_port->lock
> >
> > uart_write (uart_port->lock)
> > __uart_start
> > pl011_start_tx
> > pl011_tx_chars
> > uart_write_wakeup
> > tty_port_tty_wakeup
> > tty_port_default
> > tty_port_tty_get (tty_port->lock)
> >
> > but those code is so old, and I have no clue how to de-couple it after
> > checking other locks in the splat. There is an onging effort to make all
> > printk() as deferred, so until that happens, workaround it for now as a
> > short-term fix.
>
> Starting with v5.4.22 I noticed 'dmesg -w' stopped working on some
> machines. dmesg will follow console output for a few seconds, then it
> stops. strace indicates dmesg is blocked in read() on the /dev/kmsg fd.
> If a new dmesg process starts, it gives messages for a few seconds,
> then also stops. rsyslog's kernel logging is similarly affected.
>
> Bisection points to this patch (now known as
> 1b710b1b10eff9d46666064ea25f079f70bc67a8 upstream). I can't reproduce
> the problem on a test VM, and some machines are running v5.4.22..v5.4.26
> with no dmesg problems. It seems there is some magic in the startup
> sequence of affected machines. This code isn't executed after RNG is
> seeded, so it would have to get its bad stuff done before that happens.
>
> Reverting commit 1b710b1b10eff9d46666064ea25f079f70bc67a8 fixes the
> dmesg regression on 5.4.26. It might put the original lockdep bug back,
> but on machines running stable kernels, I prefer randomly broken lockdep
> over repeatably broken dmesg.
This should fix the problem
https://lore.kernel.org/lkml/20200303113002.63089-1-sergey.senozhatsky@gmail.com
-ss
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
WARNING: multiple messages have this Message-ID (diff)
From: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>
To: Zygo Blaxell <uixjjji1@umail.furryterror.org>
Cc: Qian Cai <cai@lca.pw>,
tytso@mit.edu, arnd@arndb.de, gregkh@linuxfoundation.org,
sergey.senozhatsky.work@gmail.com, pmladek@suse.com,
rostedt@goodmis.org, catalin.marinas@arm.com, will@kernel.org,
dan.j.williams@intel.com, peterz@infradead.org,
longman@redhat.com, tglx@linutronix.de, linux-mm@kvack.org,
linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org
Subject: Re: dmesg -w regression in v5.4.22, bisected, was: Re: [PATCH] char/random: silence a lockdep splat with printk()
Date: Wed, 25 Mar 2020 11:35:06 +0900 [thread overview]
Message-ID: <20200325023506.GB241329@google.com> (raw)
In-Reply-To: <20200324151359.GF2693@hungrycats.org>
On (20/03/24 11:13), Zygo Blaxell wrote:
> On Wed, Nov 13, 2019 at 04:16:25PM -0500, Qian Cai wrote:
> > From: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>
> >
> > Sergey didn't like the locking order,
> >
> > uart_port->lock -> tty_port->lock
> >
> > uart_write (uart_port->lock)
> > __uart_start
> > pl011_start_tx
> > pl011_tx_chars
> > uart_write_wakeup
> > tty_port_tty_wakeup
> > tty_port_default
> > tty_port_tty_get (tty_port->lock)
> >
> > but those code is so old, and I have no clue how to de-couple it after
> > checking other locks in the splat. There is an onging effort to make all
> > printk() as deferred, so until that happens, workaround it for now as a
> > short-term fix.
>
> Starting with v5.4.22 I noticed 'dmesg -w' stopped working on some
> machines. dmesg will follow console output for a few seconds, then it
> stops. strace indicates dmesg is blocked in read() on the /dev/kmsg fd.
> If a new dmesg process starts, it gives messages for a few seconds,
> then also stops. rsyslog's kernel logging is similarly affected.
>
> Bisection points to this patch (now known as
> 1b710b1b10eff9d46666064ea25f079f70bc67a8 upstream). I can't reproduce
> the problem on a test VM, and some machines are running v5.4.22..v5.4.26
> with no dmesg problems. It seems there is some magic in the startup
> sequence of affected machines. This code isn't executed after RNG is
> seeded, so it would have to get its bad stuff done before that happens.
>
> Reverting commit 1b710b1b10eff9d46666064ea25f079f70bc67a8 fixes the
> dmesg regression on 5.4.26. It might put the original lockdep bug back,
> but on machines running stable kernels, I prefer randomly broken lockdep
> over repeatably broken dmesg.
This should fix the problem
https://lore.kernel.org/lkml/20200303113002.63089-1-sergey.senozhatsky@gmail.com
-ss
next prev parent reply other threads:[~2020-03-25 2:35 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-11-13 21:16 [PATCH] char/random: silence a lockdep splat with printk() Qian Cai
2019-11-13 21:16 ` Qian Cai
2019-12-03 18:46 ` Qian Cai
2019-12-03 18:46 ` Qian Cai
2019-12-05 1:00 ` Sergey Senozhatsky
2019-12-05 1:00 ` Sergey Senozhatsky
2019-12-17 1:52 ` Qian Cai
2019-12-17 1:52 ` Qian Cai
2020-01-02 15:42 ` Qian Cai
2020-01-02 15:42 ` Qian Cai
2020-01-02 17:07 ` Steven Rostedt
2020-01-02 17:07 ` Steven Rostedt
2020-01-02 17:16 ` Qian Cai
2020-01-02 17:16 ` Qian Cai
2020-01-02 18:00 ` Theodore Y. Ts'o
2020-01-02 18:00 ` Theodore Y. Ts'o
2020-01-07 21:07 ` Theodore Y. Ts'o
2020-01-07 21:07 ` Theodore Y. Ts'o
2020-03-24 15:13 ` dmesg -w regression in v5.4.22, bisected, was: " Zygo Blaxell
2020-03-24 15:13 ` Zygo Blaxell
2020-03-25 2:35 ` Sergey Senozhatsky [this message]
2020-03-25 2:35 ` Sergey Senozhatsky
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200325023506.GB241329@google.com \
--to=sergey.senozhatsky.work@gmail.com \
--cc=arnd@arndb.de \
--cc=cai@lca.pw \
--cc=catalin.marinas@arm.com \
--cc=dan.j.williams@intel.com \
--cc=gregkh@linuxfoundation.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=longman@redhat.com \
--cc=peterz@infradead.org \
--cc=pmladek@suse.com \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
--cc=tytso@mit.edu \
--cc=uixjjji1@umail.furryterror.org \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.