public inbox for linux-serial@vger.kernel.org
 help / color / mirror / Atom feed
From: John Ogness <john.ogness@linutronix.de>
To: Steven Rostedt <rostedt@goodmis.org>
Cc: Petr Mladek <pmladek@suse.com>, Nigel Croxon <ncroxon@redhat.com>,
	"Theodore Y. Ts'o" <tytso@mit.edu>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Sergey Senozhatsky <sergey.senozhatsky@gmail.com>,
	dm-devel@redhat.com, Mikulas Patocka <mpatocka@redhat.com>,
	linux-serial@vger.kernel.org
Subject: Re: Serial console is causing system lock-up
Date: Wed, 06 Mar 2019 23:43:45 +0100	[thread overview]
Message-ID: <87ftrzbp3y.fsf@linutronix.de> (raw)
In-Reply-To: <20190306171943.12345598@oasis.local.home> (Steven Rostedt's message of "Wed, 6 Mar 2019 17:19:43 -0500")

On 2019-03-06, Steven Rostedt <rostedt@goodmis.org> wrote:
>> This bug only happens if we select large logbuffer (millions of
>> characters). With smaller log buffer, there are messages "** X printk
>> messages dropped", but there's no lockup.
>> 
>> The kernel apparently puts 2 million characters into a console log
>> buffer, then takes some lock and than tries to write all of them to a
>> slow serial line.
>>
>> [...]
>>
>> The MD-RAID is supposed to recalculate data for the corrupted device
>> and bring it back to life. However, scrubbing the MD-RAID device
>> resulted in a lot of reads from the device with bad checksums, these
>> were reported to the log and killed the machine.
>> 
>> I made a patch to dm-integrity to rate-limit the error messages. But
>> anyway - killing the machine in case of too many log messages seems
>> bad.  If the log messages are produced faster than the kernel can
>> write them, the kernel should discard some of them, not kill itself.
>
> Sounds like another aurgment for the new printk design.

Assuming the bad checksum messages are considered an emergency (for
example, at least loglevel KERN_WARN), then the new printk design would
print those messages synchronously to the slow serial line in the
context of the driver as the driver is producing them.

There wouldn't be a lock-up, but it would definitely slow down the
driver. The situation of "messages being produced faster than the kernel
can write them" would never exist because the printk() call will only
return after the writing is completed. I am curious if that would be
acceptable here?

John Ogness

  reply	other threads:[~2019-03-06 22:43 UTC|newest]

Thread overview: 36+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-03-06 14:27 Serial console is causing system lock-up Mikulas Patocka
2019-03-06 15:22 ` Petr Mladek
2019-03-06 16:07   ` Mikulas Patocka
2019-03-06 16:30     ` Theodore Y. Ts'o
2019-03-06 17:11       ` Mikulas Patocka
2019-03-06 22:19         ` Steven Rostedt
2019-03-06 22:43           ` John Ogness [this message]
2019-03-07  2:22             ` Sergey Senozhatsky
2019-03-07  8:17               ` John Ogness
2019-03-07  8:25                 ` Sergey Senozhatsky
2019-03-07  8:34                   ` John Ogness
2019-03-07  9:17                     ` Sergey Senozhatsky
2019-03-07 10:37                       ` John Ogness
2019-03-07 12:26                         ` Sergey Senozhatsky
2019-03-07 12:54                           ` Mikulas Patocka
2019-03-07 14:21                           ` John Ogness
2019-03-07 15:35                             ` Petr Mladek
2019-03-12  2:32                             ` Sergey Senozhatsky
2019-03-12  8:17                               ` John Ogness
2019-03-12  8:59                                 ` Sergey Senozhatsky
2019-03-12 10:05                                 ` Mikulas Patocka
2019-03-12 13:19                                   ` John Ogness
2019-03-12 13:44                                     ` Petr Mladek
2019-03-12 12:08                                 ` Petr Mladek
2019-03-12 15:19                                   ` John Ogness
2019-03-13  2:38                                   ` Sergey Senozhatsky
2019-03-13  8:43                                     ` John Ogness
2019-03-14 10:30                                       ` Sergey Senozhatsky
2019-03-07 14:08             ` John Stoffel
2019-03-07 14:26               ` Mikulas Patocka
2019-03-08  1:22                 ` Sergey Senozhatsky
2019-03-08  1:39                   ` Sergey Senozhatsky
2019-03-08  2:36                     ` John Ogness
2019-03-07 15:16         ` Petr Mladek
2019-03-07  1:56     ` Sergey Senozhatsky
2019-03-07 13:12       ` Mikulas Patocka

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87ftrzbp3y.fsf@linutronix.de \
    --to=john.ogness@linutronix.de \
    --cc=dm-devel@redhat.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=linux-serial@vger.kernel.org \
    --cc=mpatocka@redhat.com \
    --cc=ncroxon@redhat.com \
    --cc=pmladek@suse.com \
    --cc=rostedt@goodmis.org \
    --cc=sergey.senozhatsky@gmail.com \
    --cc=tytso@mit.edu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox