public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
From: John Ogness <john.ogness@linutronix.de>
To: Douglas Anderson <dianders@chromium.org>,
	Andrew Morton <akpm@linux-foundation.org>
Cc: Petr Mladek <pmladek@suse.com>, Li Zhe <lizhe.67@bytedance.com>,
	Pingfan Liu <kernelfans@gmail.com>,
	Lecopzer Chen <lecopzer.chen@mediatek.com>,
	Douglas Anderson <dianders@chromium.org>,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH 3/4] watchdog/hardlockup: Use printk_cpu_sync_get_irqsave() to serialize reporting
Date: Fri, 22 Dec 2023 10:48:49 +0106	[thread overview]
Message-ID: <87h6kak1ye.fsf@jogness.linutronix.de> (raw)
In-Reply-To: <20231220131534.3.I6ff691b3b40f0379bc860f80c6e729a0485b5247@changeid>

On 2023-12-20, Douglas Anderson <dianders@chromium.org> wrote:
> The interleaving problem was less bad with the "buddy" hardlockup
> detector. With "buddy" we always end up calling
> `trigger_single_cpu_backtrace(cpu)` on some CPU other than the running
> one. trigger_single_cpu_backtrace() always at least serializes the
> individual stack crawls because it eventually uses
> printk_cpu_sync_get_irqsave(). Unfortunately the fact that
> trigger_single_cpu_backtrace() eventually calls
> printk_cpu_sync_get_irqsave() (on a different CPU) means that we have
> to drop the "lock" before calling it and we can't fully serialize all
> printouts associated with a given hardlockup.

I think that is good enough. Otherwise there would need to be some kind
of CPU handshaking to ensure things are synchronized correctly in case
multiple CPUs have triggered the situation.

> However, we still do get
> the advantage of serializing the output of print_modules() and
> print_irqtrace_events().
>
> Aside from serializing hardlockups from each other, this change also
> has the advantage of serializing hardlockups and softlockups from each
> other if they happen to happen at the same time since they are both
> using the same "lock".
>
> Even though nobody is expected to hang while holding the lock
> associated with printk_cpu_sync_get_irqsave(), out of an abundance of
> caution, we don't call printk_cpu_sync_get_irqsave() until after we
> print out about the hardlockup. This makes extra sure that, even if
> printk_cpu_sync_get_irqsave() somehow never runs we at least print
> that we saw the hardlockup.

I agree with calling printk() before trying to acquire ownership of the
cpu_sync.

> This is different than the choice made for
> softlockup because hardlockup is really our last resort.
>
> Signed-off-by: Douglas Anderson <dianders@chromium.org>

Reviewed-by: John Ogness <john.ogness@linutronix.de>

  reply	other threads:[~2023-12-22  9:42 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-12-20 21:15 [PATCH 0/4] watchdog: Better handling of concurrent lockups Douglas Anderson
2023-12-20 21:15 ` [PATCH 1/4] watchdog/hardlockup: Adopt softlockup logic avoiding double-dumps Douglas Anderson
2023-12-20 21:15 ` [PATCH 2/4] watchdog/softlockup: Use printk_cpu_sync_get_irqsave() to serialize reporting Douglas Anderson
2023-12-22  7:13   ` lizhe.67
2023-12-22  9:30   ` John Ogness
2024-02-06 10:21     ` Petr Mladek
2023-12-20 21:15 ` [PATCH 3/4] watchdog/hardlockup: " Douglas Anderson
2023-12-22  9:42   ` John Ogness [this message]
2023-12-20 21:15 ` [PATCH 4/4] watchdog: If panicking and we dumped everything, don't re-enable dumping Douglas Anderson
2024-02-06 10:12 ` [PATCH 0/4] watchdog: Better handling of concurrent lockups Petr Mladek
2024-02-06 10:45   ` John Ogness
2024-02-06 19:31     ` Doug Anderson
2024-02-07 13:04     ` Petr Mladek

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87h6kak1ye.fsf@jogness.linutronix.de \
    --to=john.ogness@linutronix.de \
    --cc=akpm@linux-foundation.org \
    --cc=dianders@chromium.org \
    --cc=kernelfans@gmail.com \
    --cc=lecopzer.chen@mediatek.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lizhe.67@bytedance.com \
    --cc=pmladek@suse.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox