From: Petr Mladek <pmladek@suse.com>
To: Aaron Tomlin <atomlin@atomlin.com>
Cc: john.ogness@linutronix.de, rostedt@goodmis.org,
senozhatsky@chromium.org, mingo@redhat.com, peterz@infradead.org,
juri.lelli@redhat.com, vincent.guittot@linaro.org,
dietmar.eggemann@arm.com, bsegall@google.com, mgorman@suse.de,
vschneid@redhat.com, kprateek.nayak@amd.com,
akpm@linux-foundation.org, baoquan.he@linux.dev, rppt@kernel.org,
pasha.tatashin@soleen.com, pratyush@kernel.org,
ruirui.yang@linux.dev, david@kernel.org, ljs@kernel.org,
liam@infradead.org, vbabka@kernel.org, surenb@google.com,
mhocko@suse.com, kees@kernel.org, anna-maria@linutronix.de,
frederic@kernel.org, tglx@kernel.org, mhiramat@kernel.org,
mathieu.desnoyers@efficios.com, brendan.jackman@linux.dev,
hannes@cmpxchg.org, ziy@nvidia.com, bigeasy@linutronix.de,
clrkwllms@kernel.org, linux-kernel@vger.kernel.org,
kexec@lists.infradead.org, linux-mm@kvack.org,
linux-trace-kernel@vger.kernel.org,
linux-rt-devel@lists.linux.dev
Subject: Re: [RFC PATCH v2 1/5] panic, sys_info: Fix duplicate and dropped all-CPU backtraces in panic
Date: Mon, 16 Nov 2026 16:30:42 +0100 [thread overview]
Message-ID: <avshouTNBIeM5qtP@pathway.suse.cz> (raw)
In-Reply-To: <20261103024453.922373-2-atomlin@atomlin.com>
On Mon 2026-11-02 21:44:49, Aaron Tomlin wrote:
> When sys_info() was factored out of kernel/panic.c by
> commit b76e89e50fc3 ("panic: generalize panic_print's function to show
> sys info"), __sys_info() gained an unconditional call to
> trigger_all_cpu_backtrace() when SYS_INFO_ALL_BT is set, while
> panic_other_cpus_shutdown() retained its own call to
> panic_trigger_all_cpu_backtrace().
>
> This causes two problems during a kernel panic:
>
> 1. vpanic() invokes panic_trigger_all_cpu_backtrace() inside
> panic_other_cpus_shutdown() before stopping secondary CPUs, and
> then invokes trigger_all_cpu_backtrace() a second time inside
> sys_info(panic_print) after secondary CPUs have already been
> shut down, stalling for up to 10 seconds waiting for offline
> CPUs to respond to NMIs. Furthermore, if SYS_INFO_ALL_BT is
> configured via the kernel.kernel_sys_info sysctl rather than
> panic_print, panic_other_cpus_shutdown() misses it prior to CPU
> shutdown.
>
> 2. If sys_info(SYS_INFO_ALL_BT) is invoked on the panicking CPU
> prior to panic_other_cpus_shutdown(), calling
> trigger_all_cpu_backtrace() directly bypasses
> panic_trigger_all_cpu_backtrace() and leaves
> panic_triggering_all_cpu_backtrace unset. Because panic_cpu is
> already active, vprintk_emit() silently drops all NMI backtrace
> messages originating from non-panic CPUs.
>
> Export panic_trigger_all_cpu_backtrace() with a panic_on_this_cpu() and
> one-shot guard so all-CPU backtraces are only emitted once before
> secondary CPUs are stopped, consult sys_info_get_mask(panic_print) in
> panic_other_cpus_shutdown(), record prior Oops stack dumps in
> panic_try_start(), and route SYS_INFO_ALL_BT in __sys_info()
> through panic_trigger_all_cpu_backtrace() on the panicking CPU.
>
> --- a/kernel/panic.c
> +++ b/kernel/panic.c
> @@ -451,13 +452,21 @@ bool panic_try_start(void)
> old_cpu = PANIC_CPU_INVALID;
> this_cpu = raw_smp_processor_id();
>
> - return atomic_try_cmpxchg(&panic_cpu, &old_cpu, this_cpu);
> + if (!atomic_try_cmpxchg(&panic_cpu, &old_cpu, this_cpu))
> + return false;
> +
> + if (atomic_read(&panic_redirect_cpu) == PANIC_CPU_INVALID &&
> + (test_taint(TAINT_DIE) || oops_in_progress))
> + panic_this_cpu_backtrace_printed = true;
This would be pretty hard to maintain. The complexity of the check
shows how many code paths affect the decision. I can't imagine how
to synchonize all the pieces this way.
We need another approach. I remember that I have already proposed
something some time ago.
The idea was to enhance sys_info() API to remember what has already
been printed. trigger_all_cpu_backtrace() and frieds should
not longer be called directly. They should be called only via
sys_info() API.
See https://lore.kernel.org/all/aj5tFiwhRqPkAkqU@pathway.suse.cz/
for more details.
I think that this would be a better way to go.
What do you think, please?
Best Regards,
Petr
next prev parent reply other threads:[~2026-11-16 15:30 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-03 2:44 [RFC PATCH v2 0/5] panic, crash_core, sys_info: Introduce crash_kexec_in_memory_sys_info Aaron Tomlin
2026-11-03 2:44 ` [RFC PATCH v2 1/5] panic, sys_info: Fix duplicate and dropped all-CPU backtraces in panic Aaron Tomlin
2026-11-16 15:30 ` Petr Mladek [this message]
2026-11-03 2:44 ` [RFC PATCH v2 2/5] mm/show_mem: Avoid blocking on zone->lock during Oops or panic Aaron Tomlin
2026-11-04 2:49 ` sashiko-bot
2026-11-04 21:24 ` Aaron Tomlin
2026-11-03 2:44 ` [RFC PATCH v2 3/5] timers: Avoid blocking on hrtimer_cpu_base lock " Aaron Tomlin
2026-11-04 2:49 ` sashiko-bot
2026-11-09 22:28 ` Thomas Gleixner
2026-11-16 14:40 ` Aaron Tomlin
2026-11-03 2:44 ` [RFC PATCH v2 4/5] printk, tracing: Support console flush suppression and tail pinning Aaron Tomlin
2026-11-03 14:55 ` Steven Rostedt
2026-11-03 15:49 ` Aaron Tomlin
2026-11-16 17:08 ` Petr Mladek
2026-11-03 2:44 ` [RFC PATCH v2 5/5] panic, crash_core, sys_info: Introduce crash_kexec_in_memory_sys_info Aaron Tomlin
2026-11-04 2:49 ` sashiko-bot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=avshouTNBIeM5qtP@pathway.suse.cz \
--to=pmladek@suse.com \
--cc=akpm@linux-foundation.org \
--cc=anna-maria@linutronix.de \
--cc=atomlin@atomlin.com \
--cc=baoquan.he@linux.dev \
--cc=bigeasy@linutronix.de \
--cc=brendan.jackman@linux.dev \
--cc=bsegall@google.com \
--cc=clrkwllms@kernel.org \
--cc=david@kernel.org \
--cc=dietmar.eggemann@arm.com \
--cc=frederic@kernel.org \
--cc=hannes@cmpxchg.org \
--cc=john.ogness@linutronix.de \
--cc=juri.lelli@redhat.com \
--cc=kees@kernel.org \
--cc=kexec@lists.infradead.org \
--cc=kprateek.nayak@amd.com \
--cc=liam@infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-rt-devel@lists.linux.dev \
--cc=linux-trace-kernel@vger.kernel.org \
--cc=ljs@kernel.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=mgorman@suse.de \
--cc=mhiramat@kernel.org \
--cc=mhocko@suse.com \
--cc=mingo@redhat.com \
--cc=pasha.tatashin@soleen.com \
--cc=peterz@infradead.org \
--cc=pratyush@kernel.org \
--cc=rostedt@goodmis.org \
--cc=rppt@kernel.org \
--cc=ruirui.yang@linux.dev \
--cc=senozhatsky@chromium.org \
--cc=surenb@google.com \
--cc=tglx@kernel.org \
--cc=vbabka@kernel.org \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
--cc=ziy@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox