From: Petr Mladek <pmladek@suse.com>
To: Aaron Tomlin <atomlin@atomlin.com>
Cc: john.ogness@linutronix.de, rostedt@goodmis.org,
senozhatsky@chromium.org, mingo@redhat.com, peterz@infradead.org,
juri.lelli@redhat.com, vincent.guittot@linaro.org,
dietmar.eggemann@arm.com, bsegall@google.com, mgorman@suse.de,
vschneid@redhat.com, kprateek.nayak@amd.com,
akpm@linux-foundation.org, baoquan.he@linux.dev, rppt@kernel.org,
pasha.tatashin@soleen.com, pratyush@kernel.org,
ruirui.yang@linux.dev, david@kernel.org, ljs@kernel.org,
liam@infradead.org, vbabka@kernel.org, surenb@google.com,
mhocko@suse.com, kees@kernel.org, anna-maria@linutronix.de,
frederic@kernel.org, tglx@kernel.org, mhiramat@kernel.org,
mathieu.desnoyers@efficios.com, brendan.jackman@linux.dev,
hannes@cmpxchg.org, ziy@nvidia.com, bigeasy@linutronix.de,
clrkwllms@kernel.org, linux-kernel@vger.kernel.org,
kexec@lists.infradead.org, linux-mm@kvack.org,
linux-trace-kernel@vger.kernel.org,
linux-rt-devel@lists.linux.dev
Subject: Re: [RFC PATCH v2 1/5] panic, sys_info: Fix duplicate and dropped all-CPU backtraces in panic
Date: Mon, 16 Nov 2026 16:30:42 +0100 [thread overview]
Message-ID: <avshouTNBIeM5qtP@pathway.suse.cz> (raw)
In-Reply-To: <20261103024453.922373-2-atomlin@atomlin.com>
On Mon 2026-11-02 21:44:49, Aaron Tomlin wrote:
> When sys_info() was factored out of kernel/panic.c by
> commit b76e89e50fc3 ("panic: generalize panic_print's function to show
> sys info"), __sys_info() gained an unconditional call to
> trigger_all_cpu_backtrace() when SYS_INFO_ALL_BT is set, while
> panic_other_cpus_shutdown() retained its own call to
> panic_trigger_all_cpu_backtrace().
>
> This causes two problems during a kernel panic:
>
> 1. vpanic() invokes panic_trigger_all_cpu_backtrace() inside
> panic_other_cpus_shutdown() before stopping secondary CPUs, and
> then invokes trigger_all_cpu_backtrace() a second time inside
> sys_info(panic_print) after secondary CPUs have already been
> shut down, stalling for up to 10 seconds waiting for offline
> CPUs to respond to NMIs. Furthermore, if SYS_INFO_ALL_BT is
> configured via the kernel.kernel_sys_info sysctl rather than
> panic_print, panic_other_cpus_shutdown() misses it prior to CPU
> shutdown.
>
> 2. If sys_info(SYS_INFO_ALL_BT) is invoked on the panicking CPU
> prior to panic_other_cpus_shutdown(), calling
> trigger_all_cpu_backtrace() directly bypasses
> panic_trigger_all_cpu_backtrace() and leaves
> panic_triggering_all_cpu_backtrace unset. Because panic_cpu is
> already active, vprintk_emit() silently drops all NMI backtrace
> messages originating from non-panic CPUs.
>
> Export panic_trigger_all_cpu_backtrace() with a panic_on_this_cpu() and
> one-shot guard so all-CPU backtraces are only emitted once before
> secondary CPUs are stopped, consult sys_info_get_mask(panic_print) in
> panic_other_cpus_shutdown(), record prior Oops stack dumps in
> panic_try_start(), and route SYS_INFO_ALL_BT in __sys_info()
> through panic_trigger_all_cpu_backtrace() on the panicking CPU.
>
> --- a/kernel/panic.c
> +++ b/kernel/panic.c
> @@ -451,13 +452,21 @@ bool panic_try_start(void)
> old_cpu = PANIC_CPU_INVALID;
> this_cpu = raw_smp_processor_id();
>
> - return atomic_try_cmpxchg(&panic_cpu, &old_cpu, this_cpu);
> + if (!atomic_try_cmpxchg(&panic_cpu, &old_cpu, this_cpu))
> + return false;
> +
> + if (atomic_read(&panic_redirect_cpu) == PANIC_CPU_INVALID &&
> + (test_taint(TAINT_DIE) || oops_in_progress))
> + panic_this_cpu_backtrace_printed = true;
This would be pretty hard to maintain. The complexity of the check
shows how many code paths affect the decision. I can't imagine how
to synchonize all the pieces this way.
We need another approach. I remember that I have already proposed
something some time ago.
The idea was to enhance sys_info() API to remember what has already
been printed. trigger_all_cpu_backtrace() and frieds should
not longer be called directly. They should be called only via
sys_info() API.
See https://lore.kernel.org/all/aj5tFiwhRqPkAkqU@pathway.suse.cz/
for more details.
I think that this would be a better way to go.
What do you think, please?
Best Regards,
Petr
next prev parent reply other threads:[~2026-11-16 15:30 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-03 2:44 [RFC PATCH v2 0/5] panic, crash_core, sys_info: Introduce crash_kexec_in_memory_sys_info Aaron Tomlin
2026-11-03 2:44 ` [RFC PATCH v2 1/5] panic, sys_info: Fix duplicate and dropped all-CPU backtraces in panic Aaron Tomlin
2026-11-04 2:49 ` sashiko-bot
2026-11-16 15:30 ` Petr Mladek [this message]
2026-11-03 2:44 ` [RFC PATCH v2 2/5] mm/show_mem: Avoid blocking on zone->lock during Oops or panic Aaron Tomlin
2026-11-04 2:49 ` sashiko-bot
2026-11-04 21:24 ` Aaron Tomlin
2026-11-03 2:44 ` [RFC PATCH v2 3/5] timers: Avoid blocking on hrtimer_cpu_base lock " Aaron Tomlin
2026-11-04 2:49 ` sashiko-bot
2026-11-09 22:28 ` Thomas Gleixner
2026-11-16 14:40 ` Aaron Tomlin
2026-11-03 2:44 ` [RFC PATCH v2 4/5] printk, tracing: Support console flush suppression and tail pinning Aaron Tomlin
2026-11-03 14:55 ` Steven Rostedt
2026-11-03 15:49 ` Aaron Tomlin
2026-11-04 2:49 ` sashiko-bot
2026-11-16 17:08 ` Petr Mladek
2026-11-16 20:00 ` Petr Mladek
2026-11-03 2:44 ` [RFC PATCH v2 5/5] panic, crash_core, sys_info: Introduce crash_kexec_in_memory_sys_info Aaron Tomlin
2026-11-04 2:49 ` sashiko-bot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=avshouTNBIeM5qtP@pathway.suse.cz \
--to=pmladek@suse.com \
--cc=akpm@linux-foundation.org \
--cc=anna-maria@linutronix.de \
--cc=atomlin@atomlin.com \
--cc=baoquan.he@linux.dev \
--cc=bigeasy@linutronix.de \
--cc=brendan.jackman@linux.dev \
--cc=bsegall@google.com \
--cc=clrkwllms@kernel.org \
--cc=david@kernel.org \
--cc=dietmar.eggemann@arm.com \
--cc=frederic@kernel.org \
--cc=hannes@cmpxchg.org \
--cc=john.ogness@linutronix.de \
--cc=juri.lelli@redhat.com \
--cc=kees@kernel.org \
--cc=kexec@lists.infradead.org \
--cc=kprateek.nayak@amd.com \
--cc=liam@infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-rt-devel@lists.linux.dev \
--cc=linux-trace-kernel@vger.kernel.org \
--cc=ljs@kernel.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=mgorman@suse.de \
--cc=mhiramat@kernel.org \
--cc=mhocko@suse.com \
--cc=mingo@redhat.com \
--cc=pasha.tatashin@soleen.com \
--cc=peterz@infradead.org \
--cc=pratyush@kernel.org \
--cc=rostedt@goodmis.org \
--cc=rppt@kernel.org \
--cc=ruirui.yang@linux.dev \
--cc=senozhatsky@chromium.org \
--cc=surenb@google.com \
--cc=tglx@kernel.org \
--cc=vbabka@kernel.org \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
--cc=ziy@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.