All of lore.kernel.org
 help / color / mirror / Atom feed
From: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>
To: Steven Rostedt <rostedt@goodmis.org>
Cc: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>,
	"Qixuan.Wu" <qixuan.wu@linux.alibaba.com>,
	linux-kernel-owner <linux-kernel-owner@vger.kernel.org>,
	Petr Mladek <pmladek@suse.com>, Jan Kara <jack@suse.cz>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	Sergey Senozhatsky <sergey.senozhatsky@gmail.com>,
	"chenggang.qin" <chenggang.qin@linux.alibaba.com>,
	caijingxian <caijingxian@linux.alibaba.com>,
	"yuanliang.wyl" <yuanliang.wyl@alibaba-inc.com>,
	Tejun Heo <tj@kernel.org>
Subject: Re: Would you help to tell why async printk solution was not taken to upstream kernel ?
Date: Tue, 6 Mar 2018 10:52:22 +0900	[thread overview]
Message-ID: <20180306015222.GA6713@jagdpanzerIV> (raw)
In-Reply-To: <20180305155802.5c0f73fc@gandalf.local.home>

Hello Steven,

Let me Cc Tejun

On (03/05/18 15:58), Steven Rostedt wrote:
> On Mon, 5 Mar 2018 11:14:16 +0900
> Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> wrote:
> 
> > But I still think that it makes sense to change that "print it all" approach.
> > With more clear/explicit watchdog-dependent limits - we do direct printk for
> > 1/2 (or 2/3) of a current watchdog threshold value and offload if there is
> > more stuff in the logbuf. Implicit "logbuf size * console throughput" is
> > harder to understand. Disabling watchdog because of printk is a bit too much
> > of a compromise, probably.
> 
> If you know the baud rate, logbuf size * console throughput is actually
> trivial to calculate.
> 
> Let's see. CONFIG_LOG_BUF_SHIFT defaults to 18 (2^18 = 262144).
> Lets say we have a slow 9600 baud serial, which would give us:
>
>  262144 * 8 / 9600 = 219 (rounded up).
> 
> Thus, the worse case scenario would be 219 seconds to output the entire
> buffer. Add 10 seconds more for extra overhead, and then you have 229
> second watchdog that should never trigger because of a very slow
> console.
>
> (A more common 151200 baud modem would empty the buffer in 14 seconds).

Right. And when you register one more console (e.g. net console), you need
to re-calculate and re-adjust watchdog. When you set kernel log_buf_len
param (e.g. you might do log_buf_len=32G to store ftrace dumps from NMI)
you need to re-calculate and re-adjust watchdog, etc.

> > IOW, is logbuf worth of messages so critically important after all that we
> > are ready to jeopardize the system stability?
> 
> The stability is only in jeopardy if the watchdogs trigger, right?

Not limited to, watchdog threshold is at least deterministic.
Unlike, for instance, this guy

	rcu_read_lock()
	 printk()
	rcu_read_unlock()

It will block RCU grace periods. In the worst case this can become a
full-blown RCU stall and even OOM. In a less dramatic case this can
increase memory pressure, cause reclaimer activities, etc, which is not
a very good development, whether you have a small embedded device or a
server under high load, especially given that all you did was a bunch
of printks.

	-ss

  reply	other threads:[~2018-03-06  1:52 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <1eb584e2-a479-46dd-8a25-820da7a34e85.qixuan.wu@linux.alibaba.com>
2018-03-04 13:01 ` Would you help to tell why async printk solution was not taken to upstream kernel ? Sergey Senozhatsky
2018-03-04 15:08   ` Qixuan.Wu
2018-03-04 15:43     ` Steven Rostedt
2018-03-05  2:14       ` Sergey Senozhatsky
2018-03-05 20:45         ` Steven Rostedt
2018-03-06  2:00           ` Sergey Senozhatsky
2018-03-06  2:47             ` Steven Rostedt
2018-03-06  2:53               ` Sergey Senozhatsky
2018-03-06  3:16                 ` Steven Rostedt
2018-03-06  8:10                   ` Sergey Senozhatsky
2018-03-05 20:58         ` Steven Rostedt
2018-03-06  1:52           ` Sergey Senozhatsky [this message]
2018-03-06  2:43             ` Sergey Senozhatsky
2018-03-06  3:18               ` Steven Rostedt
2018-03-05  6:56       ` Qixuan.Wu
2018-03-05 13:29         ` Petr Mladek

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180306015222.GA6713@jagdpanzerIV \
    --to=sergey.senozhatsky.work@gmail.com \
    --cc=caijingxian@linux.alibaba.com \
    --cc=chenggang.qin@linux.alibaba.com \
    --cc=jack@suse.cz \
    --cc=linux-kernel-owner@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=pmladek@suse.com \
    --cc=qixuan.wu@linux.alibaba.com \
    --cc=rostedt@goodmis.org \
    --cc=sergey.senozhatsky@gmail.com \
    --cc=tj@kernel.org \
    --cc=yuanliang.wyl@alibaba-inc.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.