public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
From: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
To: Petr Mladek <pmladek@suse.com>
Cc: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>,
	Sergey Senozhatsky <sergey.senozhatsky@gmail.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Steven Rostedt <rostedt@goodmis.org>,
	Peter Hurley <peter@hurleysoftware.com>,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH 2/2] printk: always report lost messages on serial console
Date: Wed, 4 Jan 2017 22:34:48 +0900	[thread overview]
Message-ID: <20170104133448.GA384@tigerII.localdomain> (raw)
In-Reply-To: <20170104105235.GP14894@pathway.suse.cz>

On (01/04/17 11:52), Petr Mladek wrote:
[..]
> > this is from the real serial logs I'm looking at right now. we attach
> > "bad news" to 'critical' messages only:
> > 
> > ...
> > [   32.941061] bc00: b65dc0d8 b65dc6d0 ae1fbc7c b65c11c5 b563c9c4 00000001 b65dc6d0 b0a62f74
> > ** 150 printk messages dropped ** [   32.941614] cee0: 00000081 ae1fcef0 00000038 00000000 b5369000 ae1fcf00 ae1fd0f0 00000000
> > ** 75 printk messages dropped ** [   32.941892] d860: 00056608 af203848 00000000 0004088c 000000d0 00000000 00000000 00000000
> > ** 12 printk messages dropped ** [   32.941940] ..
> > ** 2 printk messages dropped ** [   32.941951] ..
> > ** 10 printk messages dropped ** [   32.941992] ..
> > ** 1 printk messages dropped ** [   32.941999] ..
> > ...
> 
> Do you see how useless the above messages are, please?

what... these lost messages were of extreme importance. I can't tell
the exactly the loglevel, but I'm sure it was at least pr_err() level.
these were like really important messages, unlike the ones that got
suppressed/filtered-out.

along with these lost messages that were supposed to be printed, I have
regions of lost kernel messages (?) with no reports of lost messages from
console_unlock() (!). and that's the only thing I'm fixing here. and the
only thing we can fix. no permutation of console_unlock() lines will make
the buffer bigger or printk flooding CPUs nicer or serial console driver
faster.

once we unlocked the logbuf lock in console_unlock() we lost the race
against the printk() flooding CPUs. all, or some, of the remaining
messages, no matter how important and critical, will be drained. the
next time we lock the logbuf again the logbuf will not be the same.
the only meaningful thing we can print now is "XXX printk messages
dropped". we don't know what was in those messages and no one will ever
find out. it's gone.

and the options here are
	"print 1 random message out of XXX or XXXX lost messages"
vs
	"print 1 random message out of XXX or XXXX lost messages"

and that's really it. except that in the second case we shuffle
console_unlock() lines with no gain. and I'm not sure I understand
why we even discuss it. both options are absolutely equally terrible,
because the root cause of the problem is in completely different
place.

	-ss

  reply	other threads:[~2017-01-04 13:39 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-12-24 14:09 [PATCH 0/2] printk: always report dropped messages Sergey Senozhatsky
2016-12-24 14:09 ` [PATCH 1/2] printk: drop call_console_drivers() unused param Sergey Senozhatsky
2017-01-03 10:22   ` Petr Mladek
2017-01-09 13:49     ` Petr Mladek
2016-12-24 14:09 ` [PATCH 2/2] printk: always report lost messages on serial console Sergey Senozhatsky
2017-01-03 14:55   ` Petr Mladek
2017-01-03 15:47     ` Sergey Senozhatsky
2017-01-03 16:53       ` Petr Mladek
2017-01-04  2:46         ` Sergey Senozhatsky
2017-01-04 10:52           ` Petr Mladek
2017-01-04 13:34             ` Sergey Senozhatsky [this message]
2017-01-04 15:26               ` Petr Mladek
2017-01-05  2:30                 ` Sergey Senozhatsky
2017-01-05 10:53                   ` Petr Mladek
2017-01-09 16:56   ` Petr Mladek
2017-01-10  8:49     ` Sergey Senozhatsky
2017-01-11 16:50       ` Petr Mladek
2017-01-13  5:11         ` Sergey Senozhatsky

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170104133448.GA384@tigerII.localdomain \
    --to=sergey.senozhatsky@gmail.com \
    --cc=akpm@linux-foundation.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=peter@hurleysoftware.com \
    --cc=pmladek@suse.com \
    --cc=rostedt@goodmis.org \
    --cc=sergey.senozhatsky.work@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox