From: John Ogness <john.ogness@linutronix.de>
To: Douglas Anderson <dianders@chromium.org>,
Andrew Morton <akpm@linux-foundation.org>
Cc: Petr Mladek <pmladek@suse.com>, Li Zhe <lizhe.67@bytedance.com>,
Pingfan Liu <kernelfans@gmail.com>,
Lecopzer Chen <lecopzer.chen@mediatek.com>,
Douglas Anderson <dianders@chromium.org>,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH 3/4] watchdog/hardlockup: Use printk_cpu_sync_get_irqsave() to serialize reporting
Date: Fri, 22 Dec 2023 10:48:49 +0106 [thread overview]
Message-ID: <87h6kak1ye.fsf@jogness.linutronix.de> (raw)
In-Reply-To: <20231220131534.3.I6ff691b3b40f0379bc860f80c6e729a0485b5247@changeid>
On 2023-12-20, Douglas Anderson <dianders@chromium.org> wrote:
> The interleaving problem was less bad with the "buddy" hardlockup
> detector. With "buddy" we always end up calling
> `trigger_single_cpu_backtrace(cpu)` on some CPU other than the running
> one. trigger_single_cpu_backtrace() always at least serializes the
> individual stack crawls because it eventually uses
> printk_cpu_sync_get_irqsave(). Unfortunately the fact that
> trigger_single_cpu_backtrace() eventually calls
> printk_cpu_sync_get_irqsave() (on a different CPU) means that we have
> to drop the "lock" before calling it and we can't fully serialize all
> printouts associated with a given hardlockup.
I think that is good enough. Otherwise there would need to be some kind
of CPU handshaking to ensure things are synchronized correctly in case
multiple CPUs have triggered the situation.
> However, we still do get
> the advantage of serializing the output of print_modules() and
> print_irqtrace_events().
>
> Aside from serializing hardlockups from each other, this change also
> has the advantage of serializing hardlockups and softlockups from each
> other if they happen to happen at the same time since they are both
> using the same "lock".
>
> Even though nobody is expected to hang while holding the lock
> associated with printk_cpu_sync_get_irqsave(), out of an abundance of
> caution, we don't call printk_cpu_sync_get_irqsave() until after we
> print out about the hardlockup. This makes extra sure that, even if
> printk_cpu_sync_get_irqsave() somehow never runs we at least print
> that we saw the hardlockup.
I agree with calling printk() before trying to acquire ownership of the
cpu_sync.
> This is different than the choice made for
> softlockup because hardlockup is really our last resort.
>
> Signed-off-by: Douglas Anderson <dianders@chromium.org>
Reviewed-by: John Ogness <john.ogness@linutronix.de>
next prev parent reply other threads:[~2023-12-22 9:42 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-12-20 21:15 [PATCH 0/4] watchdog: Better handling of concurrent lockups Douglas Anderson
2023-12-20 21:15 ` [PATCH 1/4] watchdog/hardlockup: Adopt softlockup logic avoiding double-dumps Douglas Anderson
2023-12-20 21:15 ` [PATCH 2/4] watchdog/softlockup: Use printk_cpu_sync_get_irqsave() to serialize reporting Douglas Anderson
2023-12-22 7:13 ` lizhe.67
2023-12-22 9:30 ` John Ogness
2024-02-06 10:21 ` Petr Mladek
2023-12-20 21:15 ` [PATCH 3/4] watchdog/hardlockup: " Douglas Anderson
2023-12-22 9:42 ` John Ogness [this message]
2023-12-20 21:15 ` [PATCH 4/4] watchdog: If panicking and we dumped everything, don't re-enable dumping Douglas Anderson
2024-02-06 10:12 ` [PATCH 0/4] watchdog: Better handling of concurrent lockups Petr Mladek
2024-02-06 10:45 ` John Ogness
2024-02-06 19:31 ` Doug Anderson
2024-02-07 13:04 ` Petr Mladek
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87h6kak1ye.fsf@jogness.linutronix.de \
--to=john.ogness@linutronix.de \
--cc=akpm@linux-foundation.org \
--cc=dianders@chromium.org \
--cc=kernelfans@gmail.com \
--cc=lecopzer.chen@mediatek.com \
--cc=linux-kernel@vger.kernel.org \
--cc=lizhe.67@bytedance.com \
--cc=pmladek@suse.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox