* [GIT PULL] runtime verification: Updates for 6.17
@ 2025-07-29 21:43 Steven Rostedt
2025-07-31 0:35 ` pr-tracker-bot
2025-08-16 12:04 ` Richard Weinberger
0 siblings, 2 replies; 8+ messages in thread
From: Steven Rostedt @ 2025-07-29 21:43 UTC (permalink / raw)
To: Linus Torvalds; +Cc: LKML, Gabriele Monaco, Nam Cao, John Kacur, Tomas Glozar
Linus,
Runtime verification changes for 6.17
- Added Linear temporal logic monitors for RT application
Real-time applications may have design flaws causing them to have
unexpected latency. For example, the applications may raise page faults, or
may be blocked trying to take a mutex without priority inheritance.
However, while attempting to implement DA monitors for these real-time
rules, deterministic automaton is found to be inappropriate as the
specification language. The automaton is complicated, hard to understand,
and error-prone.
For these cases, linear temporal logic is found to be more suitable. The
LTL is more concise and intuitive.
- Make printk_deferred() public
The new monitors needed access to printk_deferred(). Make them visible for
the entire kernel.
- Add a vpanic() to allow for va_list to be passed to panic.
- Add rtapp container monitor.
A collection of monitors that check for common problems with real-time
applications that cause unexpected latency.
- Add page fault tracepoints to risc-v
These tracepoints are necessary to for the RV monitor to run on risc-v.
- Fix the behaviour of the rv tool with -s and idle tasks.
- Allow the rv tool to gracefully terminate with SIGTERM
- Adjusts dot2c not to create lines over 100 columns
- Properly order nested monitors in the RV Kconfig file
- Return the registration error in all DA monitor instead of 0
- Update and add new sched collection monitors
Replace tss and sncid monitors with more complete sts:
Not only prove that switches occur in scheduling context and scheduling
needs interrupt disabled but also that each call to the scheduler
disables interrupts to (optionally) switch.
New monitor: nrp
Preemption requires need resched which is cleared by any switch
(includes a non optimal workaround for /nested/ preemptions)
New monitor: sssw
suspension requires setting the task to sleepable and, after the
switch occurs, the task requires a wakeup to come back to runnable
New monitor: opid
waking and need-resched operations occur with interrupts and
preemption disabled or in IRQ without explicitly disabling preemption
Please pull the latest trace-rv-6.17 tree, which can be found at:
git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace.git
trace-rv-6.17
Tag SHA1: 70f4131d758600306632a4263c3b3c592a726f22
Head SHA1: 614384533dfe99293a7ff1bce3d4389adadbb759
Gabriele Monaco (14):
tools/rv: Do not skip idle in trace
tools/rv: Stop gracefully also on SIGTERM
tools/dot2c: Fix generated files going over 100 column limit
verification/rvgen: Organise Kconfig entries for nested monitors
rv: Return init error when registering monitors
rv: Add da_handle_start_run_event_ to per-task monitors
rv: Remove trailing whitespace from tracepoint string
rv: Use strings in da monitors tracepoints
rv: Adjust monitor dependencies
rv: Retry when da monitor detects race conditions
sched: Adapt sched tracepoints for RV task model
rv: Replace tss and sncid monitors with more complete sts
rv: Add nrp and sssw per-task monitors
rv: Add opid per-cpu monitor
Nam Cao (34):
rv: Add #undef TRACE_INCLUDE_FILE
printk: Make vprintk_deferred() public
panic: Add vpanic()
rv: Let the reactors take care of buffers
rv: rename CONFIG_DA_MON_EVENTS to CONFIG_RV_MON_EVENTS
rv: Add support for LTL monitors
rv: Add rtapp container monitor
riscv: mm: Add page fault trace points
rv: Add rtapp_pagefault monitor
rv: Add rtapp_sleep monitor
rv: Add documentation for rtapp monitor
rv: Allow to configure the number of per-task monitor
objtool: Add vpanic() to the noreturn list
panic: Fix up description of vpanic()
rv/ltl: Do not execute the Buchi automaton twice on start condition
verification/dot2k: Make a separate dot2k_templates/Kconfig_container
verification/dot2k: Remove __buff_to_string()
verification/dot2k: Replace is_container() hack with subparsers
verification/dot2k: Prepare the frontend for LTL inclusion
Documentation/rv: Prepare monitor synthesis document for LTL inclusion
verification/rvgen: Restructure the templates files
verification/rvgen: Restructure the classes to prepare for LTL inclusion
verification/rvgen: Add support for linear temporal logic
Documentation/rv: Add documentation for linear temporal logic monitors
verification/rvgen: Support the 'next' operator
verification/rvgen: Generate each variable definition only once
verification/rvgen: Do not generate unused variables
rv: Remove unused field in struct rv_monitor_def
rv: Merge struct rv_monitor_def into struct rv_monitor
rv: Merge struct rv_reactor_def into struct rv_reactor
rv: Remove rv_reactor's reference counter
rv: Remove struct rv_monitor::reacting
rv: Fix wrong type cast in monitors_show()
rv: Fix wrong type cast in reactors_show() and monitor_reactor_show()
----
Documentation/trace/rv/da_monitor_synthesis.rst | 147 ------
Documentation/trace/rv/index.rst | 4 +-
Documentation/trace/rv/linear_temporal_logic.rst | 134 +++++
Documentation/trace/rv/monitor_rtapp.rst | 133 +++++
Documentation/trace/rv/monitor_sched.rst | 307 +++++++++--
Documentation/trace/rv/monitor_synthesis.rst | 271 ++++++++++
arch/riscv/mm/fault.c | 8 +
include/linux/panic.h | 3 +
include/linux/printk.h | 7 +
include/linux/rv.h | 86 +++-
include/linux/sched.h | 15 +-
include/rv/da_monitor.h | 172 +++----
include/rv/ltl_monitor.h | 186 +++++++
include/trace/events/sched.h | 12 +-
kernel/fork.c | 5 +-
kernel/panic.c | 18 +-
kernel/printk/internal.h | 1 -
kernel/sched/core.c | 13 +-
kernel/trace/rv/Kconfig | 43 +-
kernel/trace/rv/Makefile | 9 +-
kernel/trace/rv/monitors/{tss => nrp}/Kconfig | 12 +-
kernel/trace/rv/monitors/nrp/nrp.c | 138 +++++
kernel/trace/rv/monitors/nrp/nrp.h | 75 +++
kernel/trace/rv/monitors/nrp/nrp_trace.h | 15 +
kernel/trace/rv/monitors/opid/Kconfig | 19 +
kernel/trace/rv/monitors/opid/opid.c | 168 ++++++
kernel/trace/rv/monitors/opid/opid.h | 104 ++++
.../{sncid/sncid_trace.h => opid/opid_trace.h} | 8 +-
kernel/trace/rv/monitors/pagefault/Kconfig | 20 +
kernel/trace/rv/monitors/pagefault/pagefault.c | 88 ++++
kernel/trace/rv/monitors/pagefault/pagefault.h | 64 +++
.../trace/rv/monitors/pagefault/pagefault_trace.h | 14 +
kernel/trace/rv/monitors/rtapp/Kconfig | 11 +
kernel/trace/rv/monitors/rtapp/rtapp.c | 33 ++
kernel/trace/rv/monitors/rtapp/rtapp.h | 3 +
kernel/trace/rv/monitors/sched/Kconfig | 1 +
kernel/trace/rv/monitors/sched/sched.c | 3 +-
kernel/trace/rv/monitors/sco/sco.c | 7 +-
kernel/trace/rv/monitors/scpd/Kconfig | 2 +-
kernel/trace/rv/monitors/scpd/scpd.c | 7 +-
kernel/trace/rv/monitors/sleep/Kconfig | 22 +
kernel/trace/rv/monitors/sleep/sleep.c | 237 +++++++++
kernel/trace/rv/monitors/sleep/sleep.h | 257 ++++++++++
kernel/trace/rv/monitors/sleep/sleep_trace.h | 14 +
kernel/trace/rv/monitors/sncid/sncid.c | 96 ----
kernel/trace/rv/monitors/sncid/sncid.h | 49 --
kernel/trace/rv/monitors/snep/Kconfig | 2 +-
kernel/trace/rv/monitors/snep/snep.c | 7 +-
kernel/trace/rv/monitors/snep/snep.h | 14 +-
kernel/trace/rv/monitors/snroc/snroc.c | 3 +-
kernel/trace/rv/monitors/{sncid => sssw}/Kconfig | 10 +-
kernel/trace/rv/monitors/sssw/sssw.c | 116 +++++
kernel/trace/rv/monitors/sssw/sssw.h | 105 ++++
kernel/trace/rv/monitors/sssw/sssw_trace.h | 15 +
kernel/trace/rv/monitors/sts/Kconfig | 19 +
kernel/trace/rv/monitors/sts/sts.c | 156 ++++++
kernel/trace/rv/monitors/sts/sts.h | 117 +++++
.../monitors/{tss/tss_trace.h => sts/sts_trace.h} | 8 +-
kernel/trace/rv/monitors/tss/tss.c | 91 ----
kernel/trace/rv/monitors/tss/tss.h | 47 --
kernel/trace/rv/monitors/wip/Kconfig | 2 +-
kernel/trace/rv/monitors/wip/wip.c | 3 +-
kernel/trace/rv/monitors/wwnr/wwnr.c | 3 +-
kernel/trace/rv/reactor_panic.c | 8 +-
kernel/trace/rv/reactor_printk.c | 8 +-
kernel/trace/rv/rv.c | 220 ++++----
kernel/trace/rv/rv.h | 39 +-
kernel/trace/rv/rv_reactors.c | 138 ++---
kernel/trace/rv/rv_trace.h | 166 ++++--
tools/objtool/noreturns.h | 1 +
tools/verification/dot2/Makefile | 26 -
tools/verification/dot2/dot2k | 53 --
tools/verification/models/rtapp/pagefault.ltl | 1 +
tools/verification/models/rtapp/sleep.ltl | 22 +
tools/verification/models/sched/nrp.dot | 29 ++
tools/verification/models/sched/opid.dot | 35 ++
tools/verification/models/sched/sncid.dot | 18 -
tools/verification/models/sched/sssw.dot | 30 ++
tools/verification/models/sched/sts.dot | 38 ++
tools/verification/models/sched/tss.dot | 18 -
tools/verification/rv/src/in_kernel.c | 4 +-
tools/verification/rv/src/rv.c | 1 +
tools/verification/rvgen/.gitignore | 3 +
tools/verification/rvgen/Makefile | 27 +
tools/verification/rvgen/__main__.py | 67 +++
tools/verification/{dot2 => rvgen}/dot2c | 2 +-
.../verification/{dot2 => rvgen/rvgen}/automata.py | 0
tools/verification/rvgen/rvgen/container.py | 32 ++
tools/verification/{dot2 => rvgen/rvgen}/dot2c.py | 22 +-
tools/verification/rvgen/rvgen/dot2k.py | 129 +++++
.../{dot2/dot2k.py => rvgen/rvgen/generator.py} | 265 +++-------
tools/verification/rvgen/rvgen/ltl2ba.py | 566 +++++++++++++++++++++
tools/verification/rvgen/rvgen/ltl2k.py | 271 ++++++++++
.../rvgen/templates}/Kconfig | 0
.../rvgen/rvgen/templates/container/Kconfig | 5 +
.../rvgen/templates/container/main.c} | 3 +-
.../rvgen/templates/container/main.h} | 0
.../rvgen/templates/dot2k}/main.c | 3 +-
.../rvgen/templates/dot2k}/trace.h | 0
.../rvgen/rvgen/templates/ltl2k/main.c | 102 ++++
.../rvgen/rvgen/templates/ltl2k/trace.h | 14 +
101 files changed, 4860 insertions(+), 1265 deletions(-)
delete mode 100644 Documentation/trace/rv/da_monitor_synthesis.rst
create mode 100644 Documentation/trace/rv/linear_temporal_logic.rst
create mode 100644 Documentation/trace/rv/monitor_rtapp.rst
create mode 100644 Documentation/trace/rv/monitor_synthesis.rst
create mode 100644 include/rv/ltl_monitor.h
rename kernel/trace/rv/monitors/{tss => nrp}/Kconfig (51%)
create mode 100644 kernel/trace/rv/monitors/nrp/nrp.c
create mode 100644 kernel/trace/rv/monitors/nrp/nrp.h
create mode 100644 kernel/trace/rv/monitors/nrp/nrp_trace.h
create mode 100644 kernel/trace/rv/monitors/opid/Kconfig
create mode 100644 kernel/trace/rv/monitors/opid/opid.c
create mode 100644 kernel/trace/rv/monitors/opid/opid.h
rename kernel/trace/rv/monitors/{sncid/sncid_trace.h => opid/opid_trace.h} (66%)
create mode 100644 kernel/trace/rv/monitors/pagefault/Kconfig
create mode 100644 kernel/trace/rv/monitors/pagefault/pagefault.c
create mode 100644 kernel/trace/rv/monitors/pagefault/pagefault.h
create mode 100644 kernel/trace/rv/monitors/pagefault/pagefault_trace.h
create mode 100644 kernel/trace/rv/monitors/rtapp/Kconfig
create mode 100644 kernel/trace/rv/monitors/rtapp/rtapp.c
create mode 100644 kernel/trace/rv/monitors/rtapp/rtapp.h
create mode 100644 kernel/trace/rv/monitors/sleep/Kconfig
create mode 100644 kernel/trace/rv/monitors/sleep/sleep.c
create mode 100644 kernel/trace/rv/monitors/sleep/sleep.h
create mode 100644 kernel/trace/rv/monitors/sleep/sleep_trace.h
delete mode 100644 kernel/trace/rv/monitors/sncid/sncid.c
delete mode 100644 kernel/trace/rv/monitors/sncid/sncid.h
rename kernel/trace/rv/monitors/{sncid => sssw}/Kconfig (58%)
create mode 100644 kernel/trace/rv/monitors/sssw/sssw.c
create mode 100644 kernel/trace/rv/monitors/sssw/sssw.h
create mode 100644 kernel/trace/rv/monitors/sssw/sssw_trace.h
create mode 100644 kernel/trace/rv/monitors/sts/Kconfig
create mode 100644 kernel/trace/rv/monitors/sts/sts.c
create mode 100644 kernel/trace/rv/monitors/sts/sts.h
rename kernel/trace/rv/monitors/{tss/tss_trace.h => sts/sts_trace.h} (67%)
delete mode 100644 kernel/trace/rv/monitors/tss/tss.c
delete mode 100644 kernel/trace/rv/monitors/tss/tss.h
delete mode 100644 tools/verification/dot2/Makefile
delete mode 100644 tools/verification/dot2/dot2k
create mode 100644 tools/verification/models/rtapp/pagefault.ltl
create mode 100644 tools/verification/models/rtapp/sleep.ltl
create mode 100644 tools/verification/models/sched/nrp.dot
create mode 100644 tools/verification/models/sched/opid.dot
delete mode 100644 tools/verification/models/sched/sncid.dot
create mode 100644 tools/verification/models/sched/sssw.dot
create mode 100644 tools/verification/models/sched/sts.dot
delete mode 100644 tools/verification/models/sched/tss.dot
create mode 100644 tools/verification/rvgen/.gitignore
create mode 100644 tools/verification/rvgen/Makefile
create mode 100644 tools/verification/rvgen/__main__.py
rename tools/verification/{dot2 => rvgen}/dot2c (97%)
rename tools/verification/{dot2 => rvgen/rvgen}/automata.py (100%)
create mode 100644 tools/verification/rvgen/rvgen/container.py
rename tools/verification/{dot2 => rvgen/rvgen}/dot2c.py (92%)
create mode 100644 tools/verification/rvgen/rvgen/dot2k.py
rename tools/verification/{dot2/dot2k.py => rvgen/rvgen/generator.py} (51%)
create mode 100644 tools/verification/rvgen/rvgen/ltl2ba.py
create mode 100644 tools/verification/rvgen/rvgen/ltl2k.py
rename tools/verification/{dot2/dot2k_templates => rvgen/rvgen/templates}/Kconfig (100%)
create mode 100644 tools/verification/rvgen/rvgen/templates/container/Kconfig
rename tools/verification/{dot2/dot2k_templates/main_container.c => rvgen/rvgen/templates/container/main.c} (92%)
rename tools/verification/{dot2/dot2k_templates/main_container.h => rvgen/rvgen/templates/container/main.h} (100%)
rename tools/verification/{dot2/dot2k_templates => rvgen/rvgen/templates/dot2k}/main.c (96%)
rename tools/verification/{dot2/dot2k_templates => rvgen/rvgen/templates/dot2k}/trace.h (100%)
create mode 100644 tools/verification/rvgen/rvgen/templates/ltl2k/main.c
create mode 100644 tools/verification/rvgen/rvgen/templates/ltl2k/trace.h
---------------------------
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [GIT PULL] runtime verification: Updates for 6.17
2025-07-29 21:43 [GIT PULL] runtime verification: Updates for 6.17 Steven Rostedt
@ 2025-07-31 0:35 ` pr-tracker-bot
2025-08-16 12:04 ` Richard Weinberger
1 sibling, 0 replies; 8+ messages in thread
From: pr-tracker-bot @ 2025-07-31 0:35 UTC (permalink / raw)
To: Steven Rostedt
Cc: Linus Torvalds, LKML, Gabriele Monaco, Nam Cao, John Kacur,
Tomas Glozar
The pull request you sent on Tue, 29 Jul 2025 17:43:32 -0400:
> git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace.git trace-rv-6.17
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/4ff261e725d7376c12e745fdbe8a33cd6dbd5a83
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [GIT PULL] runtime verification: Updates for 6.17
2025-07-29 21:43 [GIT PULL] runtime verification: Updates for 6.17 Steven Rostedt
2025-07-31 0:35 ` pr-tracker-bot
@ 2025-08-16 12:04 ` Richard Weinberger
2025-08-16 12:08 ` Richard Weinberger
2025-08-16 13:43 ` Russell King (Oracle)
1 sibling, 2 replies; 8+ messages in thread
From: Richard Weinberger @ 2025-08-16 12:04 UTC (permalink / raw)
To: Steven Rostedt, rmk+kernel
Cc: Linus Torvalds, LKML, Gabriele Monaco, Nam Cao, John Kacur,
Tomas Glozar, linux-arm-kernel
[Adding ARM folks]
On Tue, Jul 29, 2025 at 11:43 PM Steven Rostedt <rostedt@goodmis.org> wrote:
> Linus,
>
> Runtime verification changes for 6.17
[...]
> - Add a vpanic() to allow for va_list to be passed to panic.
This change is causing a regression on ARM32.
panic() no longer shows a proper stack trace.
With this change:
[ 2.943690] Kernel panic - not syncing: VFS: Unable to mount root
fs on unknown-block(0,0)
[ 2.950101] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted
6.17.0-rc1 #176 NONE
[ 2.950509] Hardware name: Generic DT based system
[ 2.950865] Call trace:
[ 2.951722] unwind_backtrace from show_stack+0x18/0x1c
[ 2.953111] show_stack from dump_stack_lvl+0x54/0x68
[ 2.953312] dump_stack_lvl from vpanic+0xf8/0x388
[ 2.953526] vpanic from __do_trace_suspend_resume+0x0/0x50
With commit 3f045de7f5 "panic: Add vpanic()" reverted:
[ 2.680077] Kernel panic - not syncing: VFS: Unable to mount root
fs on unknown-block(0,0)
[ 2.690241] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted
6.17.0-rc1-dirty #177 NONE
[ 2.690832] Hardware name: Generic DT based system
[ 2.691339] Call trace:
[ 2.692530] unwind_backtrace from show_stack+0x18/0x1c
[ 2.694283] show_stack from dump_stack_lvl+0x54/0x68
[ 2.694536] dump_stack_lvl from panic+0x110/0x31c
[ 2.694944] panic from mount_root_generic+0x1fc/0x2ac
[ 2.695399] mount_root_generic from prepare_namespace+0x200/0x258
[ 2.695723] prepare_namespace from kernel_init+0x20/0x140
[ 2.695974] kernel_init from ret_from_fork+0x14/0x28
The change itself looks fine, so I suspect it uncovers a deeper
problem with panic().
Maybe it changes the stack layout enough such that dump_stack() on
ARM32 is no longer
able to walk it? I'm using gcc 15.1.1, if this matters.
To reproduce the issue using qemu, build a multi_v7_defconfig zImage and run:
qemu-system-arm -m 512 -M virt -kernel arch/arm/boot/zImage
-no-reboot -nographic -append "console=ttyAMA0 panic=1"
It will fail to mount a rootfs and panic.
--
Thanks,
//richard
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [GIT PULL] runtime verification: Updates for 6.17
2025-08-16 12:04 ` Richard Weinberger
@ 2025-08-16 12:08 ` Richard Weinberger
2025-08-16 13:43 ` Russell King (Oracle)
1 sibling, 0 replies; 8+ messages in thread
From: Richard Weinberger @ 2025-08-16 12:08 UTC (permalink / raw)
To: Steven Rostedt, Russell King - ARM Linux
Cc: Linus Torvalds, LKML, Gabriele Monaco, Nam Cao, John Kacur,
Tomas Glozar, linux-arm-kernel
[Re-sending with Russel's correct mail address]
On Sat, Aug 16, 2025 at 2:04 PM Richard Weinberger
<richard.weinberger@gmail.com> wrote:
>
> [Adding ARM folks]
>
> On Tue, Jul 29, 2025 at 11:43 PM Steven Rostedt <rostedt@goodmis.org> wrote:
> > Linus,
> >
> > Runtime verification changes for 6.17
>
> [...]
>
> > - Add a vpanic() to allow for va_list to be passed to panic.
>
> This change is causing a regression on ARM32.
> panic() no longer shows a proper stack trace.
>
> With this change:
> [ 2.943690] Kernel panic - not syncing: VFS: Unable to mount root
> fs on unknown-block(0,0)
> [ 2.950101] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted
> 6.17.0-rc1 #176 NONE
> [ 2.950509] Hardware name: Generic DT based system
> [ 2.950865] Call trace:
> [ 2.951722] unwind_backtrace from show_stack+0x18/0x1c
> [ 2.953111] show_stack from dump_stack_lvl+0x54/0x68
> [ 2.953312] dump_stack_lvl from vpanic+0xf8/0x388
> [ 2.953526] vpanic from __do_trace_suspend_resume+0x0/0x50
>
> With commit 3f045de7f5 "panic: Add vpanic()" reverted:
> [ 2.680077] Kernel panic - not syncing: VFS: Unable to mount root
> fs on unknown-block(0,0)
> [ 2.690241] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted
> 6.17.0-rc1-dirty #177 NONE
> [ 2.690832] Hardware name: Generic DT based system
> [ 2.691339] Call trace:
> [ 2.692530] unwind_backtrace from show_stack+0x18/0x1c
> [ 2.694283] show_stack from dump_stack_lvl+0x54/0x68
> [ 2.694536] dump_stack_lvl from panic+0x110/0x31c
> [ 2.694944] panic from mount_root_generic+0x1fc/0x2ac
> [ 2.695399] mount_root_generic from prepare_namespace+0x200/0x258
> [ 2.695723] prepare_namespace from kernel_init+0x20/0x140
> [ 2.695974] kernel_init from ret_from_fork+0x14/0x28
>
> The change itself looks fine, so I suspect it uncovers a deeper
> problem with panic().
> Maybe it changes the stack layout enough such that dump_stack() on
> ARM32 is no longer
> able to walk it? I'm using gcc 15.1.1, if this matters.
>
> To reproduce the issue using qemu, build a multi_v7_defconfig zImage and run:
> qemu-system-arm -m 512 -M virt -kernel arch/arm/boot/zImage
> -no-reboot -nographic -append "console=ttyAMA0 panic=1"
> It will fail to mount a rootfs and panic.
>
> --
> Thanks,
> //richard
--
Thanks,
//richard
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [GIT PULL] runtime verification: Updates for 6.17
2025-08-16 12:04 ` Richard Weinberger
2025-08-16 12:08 ` Richard Weinberger
@ 2025-08-16 13:43 ` Russell King (Oracle)
2025-08-16 19:26 ` Richard Weinberger
1 sibling, 1 reply; 8+ messages in thread
From: Russell King (Oracle) @ 2025-08-16 13:43 UTC (permalink / raw)
To: Richard Weinberger
Cc: Steven Rostedt, Linus Torvalds, LKML, Gabriele Monaco, Nam Cao,
John Kacur, Tomas Glozar, linux-arm-kernel
On Sat, Aug 16, 2025 at 02:04:34PM +0200, Richard Weinberger wrote:
> [Adding ARM folks]
>
> On Tue, Jul 29, 2025 at 11:43 PM Steven Rostedt <rostedt@goodmis.org> wrote:
> > Linus,
> >
> > Runtime verification changes for 6.17
>
> [...]
>
> > - Add a vpanic() to allow for va_list to be passed to panic.
>
> This change is causing a regression on ARM32.
> panic() no longer shows a proper stack trace.
>
> With this change:
> [ 2.943690] Kernel panic - not syncing: VFS: Unable to mount root
> fs on unknown-block(0,0)
> [ 2.950101] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted
> 6.17.0-rc1 #176 NONE
> [ 2.950509] Hardware name: Generic DT based system
> [ 2.950865] Call trace:
> [ 2.951722] unwind_backtrace from show_stack+0x18/0x1c
> [ 2.953111] show_stack from dump_stack_lvl+0x54/0x68
> [ 2.953312] dump_stack_lvl from vpanic+0xf8/0x388
> [ 2.953526] vpanic from __do_trace_suspend_resume+0x0/0x50
Please lookup the value of __do_trace_suspend_resume in System.map.
>
> With commit 3f045de7f5 "panic: Add vpanic()" reverted:
> [ 2.680077] Kernel panic - not syncing: VFS: Unable to mount root
> fs on unknown-block(0,0)
> [ 2.690241] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted
> 6.17.0-rc1-dirty #177 NONE
> [ 2.690832] Hardware name: Generic DT based system
> [ 2.691339] Call trace:
> [ 2.692530] unwind_backtrace from show_stack+0x18/0x1c
> [ 2.694283] show_stack from dump_stack_lvl+0x54/0x68
> [ 2.694536] dump_stack_lvl from panic+0x110/0x31c
> [ 2.694944] panic from mount_root_generic+0x1fc/0x2ac
Also please lookup mount_root_generic.
> Maybe it changes the stack layout enough such that dump_stack() on
> ARM32 is no longer
> able to walk it? I'm using gcc 15.1.1, if this matters.
Also please check whether you're using frame pointers or the unwinder
(CONFIG_UNWINDER_FRAME_POINTER or CONFIG_ARM_UNWIND).
Thanks.
--
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [GIT PULL] runtime verification: Updates for 6.17
2025-08-16 13:43 ` Russell King (Oracle)
@ 2025-08-16 19:26 ` Richard Weinberger
2025-08-18 8:41 ` Richard Weinberger
0 siblings, 1 reply; 8+ messages in thread
From: Richard Weinberger @ 2025-08-16 19:26 UTC (permalink / raw)
To: Russell King (Oracle)
Cc: Steven Rostedt, Linus Torvalds, LKML, Gabriele Monaco, Nam Cao,
John Kacur, Tomas Glozar, linux-arm-kernel
On Sat, Aug 16, 2025 at 3:44 PM Russell King (Oracle)
<linux@armlinux.org.uk> wrote:
> > With this change:
> > [ 2.943690] Kernel panic - not syncing: VFS: Unable to mount root
> > fs on unknown-block(0,0)
> > [ 2.950101] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted
> > 6.17.0-rc1 #176 NONE
> > [ 2.950509] Hardware name: Generic DT based system
> > [ 2.950865] Call trace:
> > [ 2.951722] unwind_backtrace from show_stack+0x18/0x1c
> > [ 2.953111] show_stack from dump_stack_lvl+0x54/0x68
> > [ 2.953312] dump_stack_lvl from vpanic+0xf8/0x388
> > [ 2.953526] vpanic from __do_trace_suspend_resume+0x0/0x50
>
> Please lookup the value of __do_trace_suspend_resume in System.map.
Hm, I see it twice in System.map, both times non-global.
$ grep __do_trace_suspend_resume System.map
c0303098 t __do_trace_suspend_resume
c03c5758 t __do_trace_suspend_resume
>
> >
> > With commit 3f045de7f5 "panic: Add vpanic()" reverted:
> > [ 2.680077] Kernel panic - not syncing: VFS: Unable to mount root
> > fs on unknown-block(0,0)
> > [ 2.690241] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted
> > 6.17.0-rc1-dirty #177 NONE
> > [ 2.690832] Hardware name: Generic DT based system
> > [ 2.691339] Call trace:
> > [ 2.692530] unwind_backtrace from show_stack+0x18/0x1c
> > [ 2.694283] show_stack from dump_stack_lvl+0x54/0x68
> > [ 2.694536] dump_stack_lvl from panic+0x110/0x31c
> > [ 2.694944] panic from mount_root_generic+0x1fc/0x2ac
>
> Also please lookup mount_root_generic.
$ grep mount_root_generic System.map
c1c01814 T mount_root_generic
>
> > Maybe it changes the stack layout enough such that dump_stack() on
> > ARM32 is no longer
> > able to walk it? I'm using gcc 15.1.1, if this matters.
In the meanwhile I tried also with gcc 8.5, same problem.
> Also please check whether you're using frame pointers or the unwinder
> (CONFIG_UNWINDER_FRAME_POINTER or CONFIG_ARM_UNWIND).
With CONFIG_UNWINDER_FRAME_POINTER the stack trace is sane,
so only CONFIG_ARM_UNWIND is broken.
I kind of expected it the other way around...
--
Thanks,
//richard
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [GIT PULL] runtime verification: Updates for 6.17
2025-08-16 19:26 ` Richard Weinberger
@ 2025-08-18 8:41 ` Richard Weinberger
2025-09-02 20:51 ` Richard Weinberger
0 siblings, 1 reply; 8+ messages in thread
From: Richard Weinberger @ 2025-08-18 8:41 UTC (permalink / raw)
To: Russell King (Oracle)
Cc: Steven Rostedt, Linus Torvalds, LKML, Gabriele Monaco, Nam Cao,
John Kacur, Tomas Glozar, linux-arm-kernel
On Sat, Aug 16, 2025 at 9:26 PM Richard Weinberger
<richard.weinberger@gmail.com> wrote:
> > Also please check whether you're using frame pointers or the unwinder
> > (CONFIG_UNWINDER_FRAME_POINTER or CONFIG_ARM_UNWIND).
>
> With CONFIG_UNWINDER_FRAME_POINTER the stack trace is sane,
> so only CONFIG_ARM_UNWIND is broken.
> I kind of expected it the other way around...
I found time to gather more details, maybe it rings a bell...
The problem is related to va_list. As soon as va_list is passed as
function argument the
unwinder is no longer able to unwind correctly.
Unwinding vpanic() does not lead to a correct frame anymore.
Maybe because of va_list the function arguments are passed via stack?
The unwind table entries for panic() and vpanic() are:
0xc0302e20 <vpanic>: 0x808489b0
Compact model index: 0
0x84 0x89 pop {r4, r7, r11, r14}
0xb0 finish
0xc0303190 <panic>: @0xc1c63ebc
Compact model index: 1
0x02 vsp = vsp + 12
0x84 0x00 pop {r14}
0xb1 0x0f pop {r0, r1, r2, r3}
0xb0 finish
Disassembly of vpanic and panic:
c0303558 <vpanic>:
* @args: Arguments for the format string
*
* Display a message, then perform cleanups. This function never returns.
*/
void vpanic(const char *fmt, va_list args)
{
c0303558: e92d4890 push {r4, r7, fp, lr}
c030355c: e52de004 push {lr} @ (str lr, [sp, #-4]!)
c0303560: eb00a54f bl c032caa4 <__gnu_mcount_nc>
static char buf[1024];
long i, i_next = 0, len;
int state = 0;
int old_cpu, this_cpu;
bool _crash_kexec_post_notifiers = crash_kexec_post_notifiers;
c0303564: e3084464 movw r4, #33892 @ 0x8464
c0303568: e34c4222 movt r4, #49698 @ 0xc222
{
c030356c: e1a05000 mov r5, r0
c0303570: e1a07001 mov r7, r1
bool _crash_kexec_post_notifiers = crash_kexec_post_notifiers;
c0303574: e5d4602c ldrb r6, [r4, #44] @ 0x2c
c0303578: e3560001 cmp r6, #1
c030357c: 9a000002 bls c030358c <vpanic+0x34>
c0303580: e59f0324 ldr r0, [pc, #804] @ c03038ac
<vpanic+0x354>
c0303584: e1a01006 mov r1, r6
c0303588: eb17f5b4 bl c0900c60
<__ubsan_handle_load_invalid_value>
if (panic_on_warn) {
c030358c: e3043f08 movw r3, #20232 @ 0x4f08
c0303590: e34c31f0 movt r3, #49648 @ 0xc1f0
c0303594: e5932000 ldr r2, [r3]
c0303598: e3520000 cmp r2, #0
* This thread may hit another WARN() in the panic path.
* Resetting this prevents additional WARN() from panicking the
* system on this thread. Other threads are blocked by the
* panic_mutex in panic().
*/
panic_on_warn = 0;
c030359c: 13a02000 movne r2, #0
c03035a0: 15832000 strne r2, [r3]
}
#define arch_local_irq_disable arch_local_irq_disable
static inline void arch_local_irq_disable(void)
{
asm volatile(
c03035a4: f10c0080 cpsid i
* Use the __builtin helper when available - this results in better
* code, especially when using GCC in combination with the per-task
* stack protector, as the compiler will recognize that it needs to
* load the TLS register only once in every function.
*/
cur = __builtin_thread_pointer();
c03035a8: ee1d3f70 mrc 15, 0, r3, cr13, cr0, {3}
* so go ahead.
* `old_cpu == this_cpu' means we came from nmi_panic() which sets
* panic_cpu to this CPU. In this case, this is also the 1st CPU.
*/
old_cpu = PANIC_CPU_INVALID;
this_cpu = raw_smp_processor_id();
c03035ac: e5932008 ldr r2, [r3, #8]
{
#if defined(arch_atomic_cmpxchg)
return arch_atomic_cmpxchg(v, old, new);
#elif defined(arch_atomic_cmpxchg_relaxed)
int ret;
__atomic_pre_full_fence();
c03035b0: f57ff05b dmb ish
#if __LINUX_ARM_ARCH__ >= 7 && defined(CONFIG_SMP)
#define ARCH_HAS_PREFETCHW
static inline void prefetchw(const void *ptr)
{
__asm__ __volatile__(
c03035b4: e59f32f4 ldr r3, [pc, #756] @ c03038b0
<vpanic+0x358>
c03035b8: f593f000 pldw [r3]
unsigned long res;
prefetchw(&ptr->counter);
do {
__asm__ __volatile__("@ atomic_cmpxchg\n"
c03035bc: e3e01000 mvn r1, #0
c03035c0: e1930f9f ldrex r0, [r3]
c03035c4: e3a0c000 mov ip, #0
c03035c8: e1300001 teq r0, r1
c03035cc: 0183cf92 strexeq ip, r2, [r3]
"teq %1, %4\n"
"strexeq %0, %5, [%3]\n"
: "=&r" (res), "=&r" (oldval), "+Qo" (ptr->counter)
: "r" (&ptr->counter), "Ir" (old), "r" (new)
: "cc");
} while (res);
c03035d0: e35c0000 cmp ip, #0
c03035d4: 1afffff9 bne c03035c0 <vpanic+0x68>
ret = arch_atomic_cmpxchg_relaxed(v, old, new);
__atomic_post_full_fence();
c03035d8: f57ff05b dmb ish
/* atomic_try_cmpxchg updates old_cpu on failure */
if (atomic_try_cmpxchg(&panic_cpu, &old_cpu, this_cpu)) {
/* go ahead */
} else if (old_cpu != this_cpu)
c03035dc: e1520000 cmp r2, r0
c03035e0: 13700001 cmnne r0, #1
c03035e4: 0a000000 beq c03035ec <vpanic+0x94>
panic_smp_self_stop();
c03035e8: eb00a127 bl c032ba8c <panic_smp_self_stop>
console_verbose();
c03035ec: eb034d54 bl c03d6b44 <console_verbose>
bust_spinlocks(1);
c03035f0: e3a00001 mov r0, #1
c03035f4: eb15167b bl c0848fe8 <bust_spinlocks>
len = vscnprintf(buf, sizeof(buf), fmt, args);
c03035f8: e59f02b4 ldr r0, [pc, #692] @ c03038b4
<vpanic+0x35c>
c03035fc: e1a03007 mov r3, r7
c0303600: e1a02005 mov r2, r5
c0303604: e3a01b01 mov r1, #1024 @ 0x400
bool _crash_kexec_post_notifiers = crash_kexec_post_notifiers;
c0303608: e2066001 and r6, r6, #1
len = vscnprintf(buf, sizeof(buf), fmt, args);
c030360c: eb45cce9 bl c14769b8 <vscnprintf>
if (len && buf[len - 1] == '\n')
c0303610: e3500000 cmp r0, #0
c0303614: 0a000011 beq c0303660 <vpanic+0x108>
c0303618: e2405001 sub r5, r0, #1
c030361c: e3550b01 cmp r5, #1024 @ 0x400
c0303620: 3a000002 bcc c0303630 <vpanic+0xd8>
c0303624: e59f028c ldr r0, [pc, #652] @ c03038b8
<vpanic+0x360>
c0303628: e1a01005 mov r1, r5
c030362c: eb17f56c bl c0900be4 <__ubsan_handle_out_of_bounds>
c0303630: e0843005 add r3, r4, r5
c0303634: e5d33030 ldrb r3, [r3, #48] @ 0x30
c0303638: e353000a cmp r3, #10
c030363c: 1a000007 bne c0303660 <vpanic+0x108>
buf[len - 1] = '\0';
c0303640: e3550b01 cmp r5, #1024 @ 0x400
c0303644: 3a000002 bcc c0303654 <vpanic+0xfc>
c0303648: e59f026c ldr r0, [pc, #620] @ c03038bc
<vpanic+0x364>
c030364c: e1a01005 mov r1, r5
c0303650: eb17f563 bl c0900be4 <__ubsan_handle_out_of_bounds>
c0303654: e0845005 add r5, r4, r5
c0303658: e3a03000 mov r3, #0
c030365c: e5c53030 strb r3, [r5, #48] @ 0x30
pr_emerg("Kernel panic - not syncing: %s\n", buf);
c0303660: e59f124c ldr r1, [pc, #588] @ c03038b4
<vpanic+0x35c>
c0303664: e30e07b4 movw r0, #59316 @ 0xe7b4
c0303668: e34c019d movt r0, #49565 @ 0xc19d
c030366c: eb0003c7 bl c0304590 <_printk>
/*
* Unlike the bitops with the '__' prefix above, this one *is* atomic,
* so `volatile` must always stay here with no cast-aways. See
* `Documentation/atomic_bitops.txt` for the details.
*/
return 1UL & (addr[BIT_WORD(nr)] >> (nr & (BITS_PER_LONG-1)));
c0303670: e5943008 ldr r3, [r4, #8]
#ifdef CONFIG_DEBUG_BUGVERBOSE
/*
* Avoid nested stack-dumping if a panic occurs during oops processing
*/
if (!test_taint(TAINT_DIE) && oops_in_progress <= 1)
c0303674: e3130080 tst r3, #128 @ 0x80
c0303678: 1a000005 bne c0303694 <vpanic+0x13c>
c030367c: e30a3284 movw r3, #41604 @ 0xa284
c0303680: e34c3222 movt r3, #49698 @ 0xc222
c0303684: e5933000 ldr r3, [r3]
c0303688: e3530001 cmp r3, #1
c030368c: ca000000 bgt c0303694 <vpanic+0x13c>
dump_stack();
c0303690: eb006216 bl c031bef0 <dump_stack>
* If we want to run this after calling panic_notifiers, pass
* the "crash_kexec_post_notifiers" option to the kernel.
*
* Bypass the panic_cpu check and call __crash_kexec directly.
*/
if (!_crash_kexec_post_notifiers)
c0303694: e3560000 cmp r6, #0
c0303698: 1a000001 bne c03036a4 <vpanic+0x14c>
__crash_kexec(NULL);
c030369c: e1a00006 mov r0, r6
c03036a0: eb050e94 bl c04470f8 <__crash_kexec>
if (panic_print & SYS_INFO_ALL_CPU_BT) {
c03036a4: e5943000 ldr r3, [r4]
c03036a8: e3130040 tst r3, #64 @ 0x40
c03036ac: 0a000007 beq c03036d0 <vpanic+0x178>
* to allow calling code to fall back to some other mechanism:
*/
#ifdef arch_trigger_cpumask_backtrace
static inline bool trigger_all_cpu_backtrace(void)
{
arch_trigger_cpumask_backtrace(cpu_online_mask, -1);
c03036b0: e3e01000 mvn r1, #0
c03036b4: e3040f14 movw r0, #20244 @ 0x4f14
c03036b8: e34c01f0 movt r0, #49648 @ 0xc1f0
panic_triggering_all_cpu_backtrace = true;
c03036bc: e3a03001 mov r3, #1
c03036c0: e5c43430 strb r3, [r4, #1072] @ 0x430
c03036c4: eb00a103 bl c032bad8
<arch_trigger_cpumask_backtrace>
panic_triggering_all_cpu_backtrace = false;
c03036c8: e3a03000 mov r3, #0
c03036cc: e5c43430 strb r3, [r4, #1072] @ 0x430
if (!crash_kexec)
c03036d0: e3560000 cmp r6, #0
c03036d4: 1a000001 bne c03036e0 <vpanic+0x188>
smp_send_stop();
c03036d8: eb00a0b0 bl c032b9a0 <smp_send_stop>
c03036dc: ea000000 b c03036e4 <vpanic+0x18c>
crash_smp_send_stop();
c03036e0: eb00a7e6 bl c032d680 <crash_smp_send_stop>
panic_other_cpus_shutdown(_crash_kexec_post_notifiers);
printk_legacy_allow_panic_sync();
c03036e4: eb035931 bl c03d9bb0
<printk_legacy_allow_panic_sync>
/*
* Run any panic handlers, including those that might need to
* add information to the kmsg dump output.
*/
atomic_notifier_call_chain(&panic_notifier_list, 0, buf);
c03036e8: e59f21c4 ldr r2, [pc, #452] @ c03038b4
<vpanic+0x35c>
c03036ec: e3a01000 mov r1, #0
c03036f0: e59f01c8 ldr r0, [pc, #456] @ c03038c0
<vpanic+0x368>
c03036f4: eb024681 bl c0395100 <atomic_notifier_call_chain>
sys_info(panic_print);
c03036f8: e5940000 ldr r0, [r4]
c03036fc: eb45b2ff bl c1470300 <sys_info>
kmsg_dump_desc(KMSG_DUMP_PANIC, buf);
c0303700: e59f11ac ldr r1, [pc, #428] @ c03038b4
<vpanic+0x35c>
c0303704: e3a00001 mov r0, #1
c0303708: eb036048 bl c03db830 <kmsg_dump_desc>
* Note: since some panic_notifiers can make crashed kernel
* more unstable, it can increase risks of the kdump failure too.
*
* Bypass the panic_cpu check and call __crash_kexec directly.
*/
if (_crash_kexec_post_notifiers)
c030370c: e3560000 cmp r6, #0
c0303710: 0a000001 beq c030371c <vpanic+0x1c4>
__crash_kexec(NULL);
c0303714: e3a00000 mov r0, #0
c0303718: eb050e76 bl c04470f8 <__crash_kexec>
console_unblank();
c030371c: eb035ede bl c03db29c <console_unblank>
* buffer. Try to acquire the lock then release it regardless of the
* result. The release will also print the buffers out. Locks debug
* should be disabled to avoid reporting bad unlock balance when
* panic() is not being callled from OOPS.
*/
debug_locks_off();
c0303720: eb151570 bl c0848ce8 <debug_locks_off>
console_flush_on_panic(CONSOLE_FLUSH_PENDING);
c0303724: e3a00000 mov r0, #0
c0303728: eb035f7a bl c03db518 <console_flush_on_panic>
if ((panic_print & SYS_INFO_PANIC_CONSOLE_REPLAY) ||
c030372c: e5943000 ldr r3, [r4]
c0303730: e3130020 tst r3, #32
c0303734: 0a000002 beq c0303744 <vpanic+0x1ec>
panic_console_replay)
console_flush_on_panic(CONSOLE_REPLAY_ALL);
c0303738: e3a00001 mov r0, #1
c030373c: eb035f75 bl c03db518 <console_flush_on_panic>
c0303740: ea000007 b c0303764 <vpanic+0x20c>
if ((panic_print & SYS_INFO_PANIC_CONSOLE_REPLAY) ||
c0303744: e5d4543c ldrb r5, [r4, #1084] @ 0x43c
c0303748: e3550001 cmp r5, #1
c030374c: 9a000002 bls c030375c <vpanic+0x204>
c0303750: e59f016c ldr r0, [pc, #364] @ c03038c4
<vpanic+0x36c>
c0303754: e1a01005 mov r1, r5
c0303758: eb17f540 bl c0900c60
<__ubsan_handle_load_invalid_value>
c030375c: e3150001 tst r5, #1
c0303760: 1afffff4 bne c0303738 <vpanic+0x1e0>
if (!panic_blink)
c0303764: e5943440 ldr r3, [r4, #1088] @ 0x440
c0303768: e30e8b08 movw r8, #60168 @ 0xeb08
c030376c: e34c81b7 movt r8, #49591 @ 0xc1b7
panic_blink = no_blink;
if (panic_timeout > 0) {
c0303770: e5941444 ldr r1, [r4, #1092] @ 0x444
if (!panic_blink)
c0303774: e3530000 cmp r3, #0
panic_blink = no_blink;
c0303778: 03053010 movweq r3, #20496 @ 0x5010
c030377c: 034c3036 movteq r3, #49206 @ 0xc036
c0303780: 05843440 streq r3, [r4, #1088] @ 0x440
if (panic_timeout > 0) {
c0303784: e3510000 cmp r1, #0
int state = 0;
c0303788: d3a05000 movle r5, #0
long i, i_next = 0, len;
c030378c: d1a07005 movle r7, r5
if (panic_timeout > 0) {
c0303790: da00001c ble c0303808 <vpanic+0x2b0>
int state = 0;
c0303794: e3a05000 mov r5, #0
touch_nmi_watchdog();
if (i >= i_next) {
i += panic_blink(state ^= 1);
i_next = i + 3600 / PANIC_BLINK_SPD;
}
mdelay(PANIC_TIMER_STEP);
c0303798: e30cbb60 movw fp, #52064 @ 0xcb60
c030379c: e340bccc movt fp, #3276 @ 0xccc
long i, i_next = 0, len;
c03037a0: e1a07005 mov r7, r5
for (i = 0; i < panic_timeout * 1000; i += PANIC_TIMER_STEP) {
c03037a4: e1a06005 mov r6, r5
c03037a8: e3a0affa mov sl, #1000 @ 0x3e8
pr_emerg("Rebooting in %d seconds..\n", panic_timeout);
c03037ac: e30e07d8 movw r0, #59352 @ 0xe7d8
c03037b0: e34c019d movt r0, #49565 @ 0xc19d
c03037b4: eb000375 bl c0304590 <_printk>
for (i = 0; i < panic_timeout * 1000; i += PANIC_TIMER_STEP) {
c03037b8: e5941444 ldr r1, [r4, #1092] @ 0x444
c03037bc: e003019a mul r3, sl, r1
c03037c0: e1530006 cmp r3, r6
c03037c4: da00000f ble c0303808 <vpanic+0x2b0>
if (i >= i_next) {
c03037c8: e1560007 cmp r6, r7
c03037cc: ba000005 blt c03037e8 <vpanic+0x290>
i += panic_blink(state ^= 1);
c03037d0: e2255001 eor r5, r5, #1
c03037d4: e5943440 ldr r3, [r4, #1088] @ 0x440
c03037d8: e1a00005 mov r0, r5
c03037dc: e12fff33 blx r3
c03037e0: e0866000 add r6, r6, r0
i_next = i + 3600 / PANIC_BLINK_SPD;
c03037e4: e28670c8 add r7, r6, #200 @ 0xc8
{
c03037e8: e3a09064 mov r9, #100 @ 0x64
mdelay(PANIC_TIMER_STEP);
c03037ec: e5983004 ldr r3, [r8, #4]
c03037f0: e1a0000b mov r0, fp
c03037f4: e12fff33 blx r3
c03037f8: e2599001 subs r9, r9, #1
c03037fc: 1afffffa bne c03037ec <vpanic+0x294>
for (i = 0; i < panic_timeout * 1000; i += PANIC_TIMER_STEP) {
c0303800: e2866064 add r6, r6, #100 @ 0x64
c0303804: eaffffeb b c03037b8 <vpanic+0x260>
}
}
if (panic_timeout != 0) {
c0303808: e3510000 cmp r1, #0
c030380c: 0a000007 beq c0303830 <vpanic+0x2d8>
/*
* This will not be a clean reboot, with everything
* shutting down. But if there is a chance of
* rebooting the system it will be rebooted.
*/
if (panic_reboot_mode != REBOOT_UNDEFINED)
c0303810: e3083194 movw r3, #33172 @ 0x8194
c0303814: e34c31f2 movt r3, #49650 @ 0xc1f2
c0303818: e5932000 ldr r2, [r3]
c030381c: e3720001 cmn r2, #1
reboot_mode = panic_reboot_mode;
c0303820: 13083198 movwne r3, #33176 @ 0x8198
c0303824: 134c31f2 movtne r3, #49650 @ 0xc1f2
c0303828: 15832000 strne r2, [r3]
emergency_restart();
c030382c: eb024cbd bl c0396b28 <emergency_restart>
}
#endif
#if defined(CONFIG_S390)
disabled_wait();
#endif
pr_emerg("---[ end Kernel panic - not syncing: %s ]---\n", buf);
c0303830: e59f107c ldr r1, [pc, #124] @ c03038b4
<vpanic+0x35c>
c0303834: e30e07f8 movw r0, #59384 @ 0xe7f8
c0303838: e34c019d movt r0, #49565 @ 0xc19d
c030383c: eb000353 bl c0304590 <_printk>
/* Do not scroll important messages printed above */
suppress_printk = 1;
c0303840: e3043f70 movw r3, #20336 @ 0x4f70
c0303844: e34c31f0 movt r3, #49648 @ 0xc1f0
/*
* The final messages may not have been printed if in a context that
* defers printing (such as NMI) and irq_work is not available.
* Explicitly flush the kernel log buffer one last time.
*/
console_flush_on_panic(CONSOLE_FLUSH_PENDING);
c0303848: e3a00000 mov r0, #0
suppress_printk = 1;
c030384c: e3a02001 mov r2, #1
c0303850: e5832000 str r2, [r3]
console_flush_on_panic(CONSOLE_FLUSH_PENDING);
c0303854: eb035f2f bl c03db518 <console_flush_on_panic>
nbcon_atomic_flush_unsafe();
c0303858: eb03684e bl c03dd998 <nbcon_atomic_flush_unsafe>
asm volatile(
c030385c: f1080080 cpsie i
touch_softlockup_watchdog();
if (i >= i_next) {
i += panic_blink(state ^= 1);
i_next = i + 3600 / PANIC_BLINK_SPD;
}
mdelay(PANIC_TIMER_STEP);
c0303860: e30cab60 movw sl, #52064 @ 0xcb60
c0303864: e340accc movt sl, #3276 @ 0xccc
for (i = 0; ; i += PANIC_TIMER_STEP) {
c0303868: e3a06000 mov r6, #0
if (i >= i_next) {
c030386c: e1560007 cmp r6, r7
c0303870: ba000005 blt c030388c <vpanic+0x334>
i += panic_blink(state ^= 1);
c0303874: e2255001 eor r5, r5, #1
c0303878: e5943440 ldr r3, [r4, #1088] @ 0x440
c030387c: e1a00005 mov r0, r5
c0303880: e12fff33 blx r3
c0303884: e0866000 add r6, r6, r0
i_next = i + 3600 / PANIC_BLINK_SPD;
c0303888: e28670c8 add r7, r6, #200 @ 0xc8
for (i = 0; ; i += PANIC_TIMER_STEP) {
c030388c: e3a09064 mov r9, #100 @ 0x64
mdelay(PANIC_TIMER_STEP);
c0303890: e5983004 ldr r3, [r8, #4]
c0303894: e1a0000a mov r0, sl
c0303898: e12fff33 blx r3
c030389c: e2599001 subs r9, r9, #1
c03038a0: 1afffffa bne c0303890 <vpanic+0x338>
for (i = 0; ; i += PANIC_TIMER_STEP) {
c03038a4: e2866064 add r6, r6, #100 @ 0x64
touch_softlockup_watchdog();
c03038a8: eaffffef b c030386c <vpanic+0x314>
c03038ac: c1f231bc .word 0xc1f231bc
c03038b0: c1f231cc .word 0xc1f231cc
c03038b4: c2228494 .word 0xc2228494
c03038b8: c1f231d0 .word 0xc1f231d0
c03038bc: c1f231e4 .word 0xc1f231e4
c03038c0: c2228898 .word 0xc2228898
c03038c4: c1f231f8 .word 0xc1f231f8
c03038c8 <panic>:
}
EXPORT_SYMBOL(vpanic);
/* Identical to vpanic(), except it takes variadic arguments instead
of va_list */
void panic(const char *fmt, ...)
{
c03038c8: e92d000f push {r0, r1, r2, r3}
c03038cc: e52de004 push {lr} @ (str lr, [sp, #-4]!)
c03038d0: e24dd00c sub sp, sp, #12
c03038d4: e52de004 push {lr} @ (str lr, [sp, #-4]!)
c03038d8: eb00a471 bl c032caa4 <__gnu_mcount_nc>
c03038dc: ee1d3f70 mrc 15, 0, r3, cr13, cr0, {3}
va_list args;
va_start(args, fmt);
c03038e0: e28d1014 add r1, sp, #20
vpanic(fmt, args);
c03038e4: e59d0010 ldr r0, [sp, #16]
{
c03038e8: e5932508 ldr r2, [r3, #1288] @ 0x508
c03038ec: e58d2004 str r2, [sp, #4]
c03038f0: e3a02000 mov r2, #0
va_start(args, fmt);
c03038f4: e58d1000 str r1, [sp]
vpanic(fmt, args);
c03038f8: ebffff16 bl c0303558 <vpanic>
--
Thanks,
//richard
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [GIT PULL] runtime verification: Updates for 6.17
2025-08-18 8:41 ` Richard Weinberger
@ 2025-09-02 20:51 ` Richard Weinberger
0 siblings, 0 replies; 8+ messages in thread
From: Richard Weinberger @ 2025-09-02 20:51 UTC (permalink / raw)
To: Russell King (Oracle)
Cc: Steven Rostedt, Linus Torvalds, LKML, Gabriele Monaco, Nam Cao,
John Kacur, Tomas Glozar, linux-arm-kernel
On Mon, Aug 18, 2025 at 10:41 AM Richard Weinberger
<richard.weinberger@gmail.com> wrote:
>
> On Sat, Aug 16, 2025 at 9:26 PM Richard Weinberger
> <richard.weinberger@gmail.com> wrote:
> > > Also please check whether you're using frame pointers or the unwinder
> > > (CONFIG_UNWINDER_FRAME_POINTER or CONFIG_ARM_UNWIND).
> >
> > With CONFIG_UNWINDER_FRAME_POINTER the stack trace is sane,
> > so only CONFIG_ARM_UNWIND is broken.
> > I kind of expected it the other way around...
>
> I found time to gather more details, maybe it rings a bell...
>
> The problem is related to va_list. As soon as va_list is passed as
> function argument the
> unwinder is no longer able to unwind correctly.
> Unwinding vpanic() does not lead to a correct frame anymore.
> Maybe because of va_list the function arguments are passed via stack?
Time to post an update, there is a problem in the ARM unwinder.
When the last instruction of a function is BL, LR points outside of
the function,
this confuses the unwinder.
This branch contains two WIP patches from Russel and myself to address
the issue:
https://git.kernel.org/pub/scm/linux/kernel/git/rw/misc.git/log/?h=arm_unwind_fix
--
Thanks,
//richard
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2025-09-02 20:51 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-07-29 21:43 [GIT PULL] runtime verification: Updates for 6.17 Steven Rostedt
2025-07-31 0:35 ` pr-tracker-bot
2025-08-16 12:04 ` Richard Weinberger
2025-08-16 12:08 ` Richard Weinberger
2025-08-16 13:43 ` Russell King (Oracle)
2025-08-16 19:26 ` Richard Weinberger
2025-08-18 8:41 ` Richard Weinberger
2025-09-02 20:51 ` Richard Weinberger
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).