[GIT PULL] runtime verification: Updates for 6.17

[GIT PULL] runtime verification: Updates for 6.17

[Steven Rostedt]

Linus,

Runtime verification changes for 6.17

- Added Linear temporal logic monitors for RT application

  Real-time applications may have design flaws causing them to have
  unexpected latency. For example, the applications may raise page faults, or
  may be blocked trying to take a mutex without priority inheritance.

  However, while attempting to implement DA monitors for these real-time
  rules, deterministic automaton is found to be inappropriate as the
  specification language. The automaton is complicated, hard to understand,
  and error-prone.

  For these cases, linear temporal logic is found to be more suitable. The
  LTL is more concise and intuitive.

- Make printk_deferred() public

  The new monitors needed access to printk_deferred(). Make them visible for
  the entire kernel.

- Add a vpanic() to allow for va_list to be passed to panic.

- Add rtapp container monitor.

  A collection of monitors that check for common problems with real-time
  applications that cause unexpected latency.

- Add page fault tracepoints to risc-v

  These tracepoints are necessary to for the RV monitor to run on risc-v.

- Fix the behaviour of the rv tool with -s and idle tasks.

- Allow the rv tool to gracefully terminate with SIGTERM

- Adjusts dot2c not to create lines over 100 columns

- Properly order nested monitors in the RV Kconfig file

- Return the registration error in all DA monitor instead of 0

- Update and add new sched collection monitors

  Replace tss and sncid monitors with more complete sts:
  Not only prove that switches occur in scheduling context and scheduling
  needs interrupt disabled but also that each call to the scheduler
  disables interrupts to (optionally) switch.

  New monitor: nrp
   Preemption requires need resched which is cleared by any switch
   (includes a non optimal workaround for /nested/ preemptions)

  New monitor: sssw
   suspension requires setting the task to sleepable and, after the
   switch occurs, the task requires a wakeup to come back to runnable

  New monitor: opid
   waking and need-resched operations occur with interrupts and
   preemption disabled or in IRQ without explicitly disabling preemption


Please pull the latest trace-rv-6.17 tree, which can be found at:


  git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace.git
trace-rv-6.17

Tag SHA1: 70f4131d758600306632a4263c3b3c592a726f22
Head SHA1: 614384533dfe99293a7ff1bce3d4389adadbb759


Gabriele Monaco (14):
      tools/rv: Do not skip idle in trace
      tools/rv: Stop gracefully also on SIGTERM
      tools/dot2c: Fix generated files going over 100 column limit
      verification/rvgen: Organise Kconfig entries for nested monitors
      rv: Return init error when registering monitors
      rv: Add da_handle_start_run_event_ to per-task monitors
      rv: Remove trailing whitespace from tracepoint string
      rv: Use strings in da monitors tracepoints
      rv: Adjust monitor dependencies
      rv: Retry when da monitor detects race conditions
      sched: Adapt sched tracepoints for RV task model
      rv: Replace tss and sncid monitors with more complete sts
      rv: Add nrp and sssw per-task monitors
      rv: Add opid per-cpu monitor

Nam Cao (34):
      rv: Add #undef TRACE_INCLUDE_FILE
      printk: Make vprintk_deferred() public
      panic: Add vpanic()
      rv: Let the reactors take care of buffers
      rv: rename CONFIG_DA_MON_EVENTS to CONFIG_RV_MON_EVENTS
      rv: Add support for LTL monitors
      rv: Add rtapp container monitor
      riscv: mm: Add page fault trace points
      rv: Add rtapp_pagefault monitor
      rv: Add rtapp_sleep monitor
      rv: Add documentation for rtapp monitor
      rv: Allow to configure the number of per-task monitor
      objtool: Add vpanic() to the noreturn list
      panic: Fix up description of vpanic()
      rv/ltl: Do not execute the Buchi automaton twice on start condition
      verification/dot2k: Make a separate dot2k_templates/Kconfig_container
      verification/dot2k: Remove __buff_to_string()
      verification/dot2k: Replace is_container() hack with subparsers
      verification/dot2k: Prepare the frontend for LTL inclusion
      Documentation/rv: Prepare monitor synthesis document for LTL inclusion
      verification/rvgen: Restructure the templates files
      verification/rvgen: Restructure the classes to prepare for LTL inclusion
      verification/rvgen: Add support for linear temporal logic
      Documentation/rv: Add documentation for linear temporal logic monitors
      verification/rvgen: Support the 'next' operator
      verification/rvgen: Generate each variable definition only once
      verification/rvgen: Do not generate unused variables
      rv: Remove unused field in struct rv_monitor_def
      rv: Merge struct rv_monitor_def into struct rv_monitor
      rv: Merge struct rv_reactor_def into struct rv_reactor
      rv: Remove rv_reactor's reference counter
      rv: Remove struct rv_monitor::reacting
      rv: Fix wrong type cast in monitors_show()
      rv: Fix wrong type cast in reactors_show() and monitor_reactor_show()

----
 Documentation/trace/rv/da_monitor_synthesis.rst    | 147 ------
 Documentation/trace/rv/index.rst                   |   4 +-
 Documentation/trace/rv/linear_temporal_logic.rst   | 134 +++++
 Documentation/trace/rv/monitor_rtapp.rst           | 133 +++++
 Documentation/trace/rv/monitor_sched.rst           | 307 +++++++++--
 Documentation/trace/rv/monitor_synthesis.rst       | 271 ++++++++++
 arch/riscv/mm/fault.c                              |   8 +
 include/linux/panic.h                              |   3 +
 include/linux/printk.h                             |   7 +
 include/linux/rv.h                                 |  86 +++-
 include/linux/sched.h                              |  15 +-
 include/rv/da_monitor.h                            | 172 +++----
 include/rv/ltl_monitor.h                           | 186 +++++++
 include/trace/events/sched.h                       |  12 +-
 kernel/fork.c                                      |   5 +-
 kernel/panic.c                                     |  18 +-
 kernel/printk/internal.h                           |   1 -
 kernel/sched/core.c                                |  13 +-
 kernel/trace/rv/Kconfig                            |  43 +-
 kernel/trace/rv/Makefile                           |   9 +-
 kernel/trace/rv/monitors/{tss => nrp}/Kconfig      |  12 +-
 kernel/trace/rv/monitors/nrp/nrp.c                 | 138 +++++
 kernel/trace/rv/monitors/nrp/nrp.h                 |  75 +++
 kernel/trace/rv/monitors/nrp/nrp_trace.h           |  15 +
 kernel/trace/rv/monitors/opid/Kconfig              |  19 +
 kernel/trace/rv/monitors/opid/opid.c               | 168 ++++++
 kernel/trace/rv/monitors/opid/opid.h               | 104 ++++
 .../{sncid/sncid_trace.h => opid/opid_trace.h}     |   8 +-
 kernel/trace/rv/monitors/pagefault/Kconfig         |  20 +
 kernel/trace/rv/monitors/pagefault/pagefault.c     |  88 ++++
 kernel/trace/rv/monitors/pagefault/pagefault.h     |  64 +++
 .../trace/rv/monitors/pagefault/pagefault_trace.h  |  14 +
 kernel/trace/rv/monitors/rtapp/Kconfig             |  11 +
 kernel/trace/rv/monitors/rtapp/rtapp.c             |  33 ++
 kernel/trace/rv/monitors/rtapp/rtapp.h             |   3 +
 kernel/trace/rv/monitors/sched/Kconfig             |   1 +
 kernel/trace/rv/monitors/sched/sched.c             |   3 +-
 kernel/trace/rv/monitors/sco/sco.c                 |   7 +-
 kernel/trace/rv/monitors/scpd/Kconfig              |   2 +-
 kernel/trace/rv/monitors/scpd/scpd.c               |   7 +-
 kernel/trace/rv/monitors/sleep/Kconfig             |  22 +
 kernel/trace/rv/monitors/sleep/sleep.c             | 237 +++++++++
 kernel/trace/rv/monitors/sleep/sleep.h             | 257 ++++++++++
 kernel/trace/rv/monitors/sleep/sleep_trace.h       |  14 +
 kernel/trace/rv/monitors/sncid/sncid.c             |  96 ----
 kernel/trace/rv/monitors/sncid/sncid.h             |  49 --
 kernel/trace/rv/monitors/snep/Kconfig              |   2 +-
 kernel/trace/rv/monitors/snep/snep.c               |   7 +-
 kernel/trace/rv/monitors/snep/snep.h               |  14 +-
 kernel/trace/rv/monitors/snroc/snroc.c             |   3 +-
 kernel/trace/rv/monitors/{sncid => sssw}/Kconfig   |  10 +-
 kernel/trace/rv/monitors/sssw/sssw.c               | 116 +++++
 kernel/trace/rv/monitors/sssw/sssw.h               | 105 ++++
 kernel/trace/rv/monitors/sssw/sssw_trace.h         |  15 +
 kernel/trace/rv/monitors/sts/Kconfig               |  19 +
 kernel/trace/rv/monitors/sts/sts.c                 | 156 ++++++
 kernel/trace/rv/monitors/sts/sts.h                 | 117 +++++
 .../monitors/{tss/tss_trace.h => sts/sts_trace.h}  |   8 +-
 kernel/trace/rv/monitors/tss/tss.c                 |  91 ----
 kernel/trace/rv/monitors/tss/tss.h                 |  47 --
 kernel/trace/rv/monitors/wip/Kconfig               |   2 +-
 kernel/trace/rv/monitors/wip/wip.c                 |   3 +-
 kernel/trace/rv/monitors/wwnr/wwnr.c               |   3 +-
 kernel/trace/rv/reactor_panic.c                    |   8 +-
 kernel/trace/rv/reactor_printk.c                   |   8 +-
 kernel/trace/rv/rv.c                               | 220 ++++----
 kernel/trace/rv/rv.h                               |  39 +-
 kernel/trace/rv/rv_reactors.c                      | 138 ++---
 kernel/trace/rv/rv_trace.h                         | 166 ++++--
 tools/objtool/noreturns.h                          |   1 +
 tools/verification/dot2/Makefile                   |  26 -
 tools/verification/dot2/dot2k                      |  53 --
 tools/verification/models/rtapp/pagefault.ltl      |   1 +
 tools/verification/models/rtapp/sleep.ltl          |  22 +
 tools/verification/models/sched/nrp.dot            |  29 ++
 tools/verification/models/sched/opid.dot           |  35 ++
 tools/verification/models/sched/sncid.dot          |  18 -
 tools/verification/models/sched/sssw.dot           |  30 ++
 tools/verification/models/sched/sts.dot            |  38 ++
 tools/verification/models/sched/tss.dot            |  18 -
 tools/verification/rv/src/in_kernel.c              |   4 +-
 tools/verification/rv/src/rv.c                     |   1 +
 tools/verification/rvgen/.gitignore                |   3 +
 tools/verification/rvgen/Makefile                  |  27 +
 tools/verification/rvgen/__main__.py               |  67 +++
 tools/verification/{dot2 => rvgen}/dot2c           |   2 +-
 .../verification/{dot2 => rvgen/rvgen}/automata.py |   0
 tools/verification/rvgen/rvgen/container.py        |  32 ++
 tools/verification/{dot2 => rvgen/rvgen}/dot2c.py  |  22 +-
 tools/verification/rvgen/rvgen/dot2k.py            | 129 +++++
 .../{dot2/dot2k.py => rvgen/rvgen/generator.py}    | 265 +++-------
 tools/verification/rvgen/rvgen/ltl2ba.py           | 566 +++++++++++++++++++++
 tools/verification/rvgen/rvgen/ltl2k.py            | 271 ++++++++++
 .../rvgen/templates}/Kconfig                       |   0
 .../rvgen/rvgen/templates/container/Kconfig        |   5 +
 .../rvgen/templates/container/main.c}              |   3 +-
 .../rvgen/templates/container/main.h}              |   0
 .../rvgen/templates/dot2k}/main.c                  |   3 +-
 .../rvgen/templates/dot2k}/trace.h                 |   0
 .../rvgen/rvgen/templates/ltl2k/main.c             | 102 ++++
 .../rvgen/rvgen/templates/ltl2k/trace.h            |  14 +
 101 files changed, 4860 insertions(+), 1265 deletions(-)
 delete mode 100644 Documentation/trace/rv/da_monitor_synthesis.rst
 create mode 100644 Documentation/trace/rv/linear_temporal_logic.rst
 create mode 100644 Documentation/trace/rv/monitor_rtapp.rst
 create mode 100644 Documentation/trace/rv/monitor_synthesis.rst
 create mode 100644 include/rv/ltl_monitor.h
 rename kernel/trace/rv/monitors/{tss => nrp}/Kconfig (51%)
 create mode 100644 kernel/trace/rv/monitors/nrp/nrp.c
 create mode 100644 kernel/trace/rv/monitors/nrp/nrp.h
 create mode 100644 kernel/trace/rv/monitors/nrp/nrp_trace.h
 create mode 100644 kernel/trace/rv/monitors/opid/Kconfig
 create mode 100644 kernel/trace/rv/monitors/opid/opid.c
 create mode 100644 kernel/trace/rv/monitors/opid/opid.h
 rename kernel/trace/rv/monitors/{sncid/sncid_trace.h => opid/opid_trace.h} (66%)
 create mode 100644 kernel/trace/rv/monitors/pagefault/Kconfig
 create mode 100644 kernel/trace/rv/monitors/pagefault/pagefault.c
 create mode 100644 kernel/trace/rv/monitors/pagefault/pagefault.h
 create mode 100644 kernel/trace/rv/monitors/pagefault/pagefault_trace.h
 create mode 100644 kernel/trace/rv/monitors/rtapp/Kconfig
 create mode 100644 kernel/trace/rv/monitors/rtapp/rtapp.c
 create mode 100644 kernel/trace/rv/monitors/rtapp/rtapp.h
 create mode 100644 kernel/trace/rv/monitors/sleep/Kconfig
 create mode 100644 kernel/trace/rv/monitors/sleep/sleep.c
 create mode 100644 kernel/trace/rv/monitors/sleep/sleep.h
 create mode 100644 kernel/trace/rv/monitors/sleep/sleep_trace.h
 delete mode 100644 kernel/trace/rv/monitors/sncid/sncid.c
 delete mode 100644 kernel/trace/rv/monitors/sncid/sncid.h
 rename kernel/trace/rv/monitors/{sncid => sssw}/Kconfig (58%)
 create mode 100644 kernel/trace/rv/monitors/sssw/sssw.c
 create mode 100644 kernel/trace/rv/monitors/sssw/sssw.h
 create mode 100644 kernel/trace/rv/monitors/sssw/sssw_trace.h
 create mode 100644 kernel/trace/rv/monitors/sts/Kconfig
 create mode 100644 kernel/trace/rv/monitors/sts/sts.c
 create mode 100644 kernel/trace/rv/monitors/sts/sts.h
 rename kernel/trace/rv/monitors/{tss/tss_trace.h => sts/sts_trace.h} (67%)
 delete mode 100644 kernel/trace/rv/monitors/tss/tss.c
 delete mode 100644 kernel/trace/rv/monitors/tss/tss.h
 delete mode 100644 tools/verification/dot2/Makefile
 delete mode 100644 tools/verification/dot2/dot2k
 create mode 100644 tools/verification/models/rtapp/pagefault.ltl
 create mode 100644 tools/verification/models/rtapp/sleep.ltl
 create mode 100644 tools/verification/models/sched/nrp.dot
 create mode 100644 tools/verification/models/sched/opid.dot
 delete mode 100644 tools/verification/models/sched/sncid.dot
 create mode 100644 tools/verification/models/sched/sssw.dot
 create mode 100644 tools/verification/models/sched/sts.dot
 delete mode 100644 tools/verification/models/sched/tss.dot
 create mode 100644 tools/verification/rvgen/.gitignore
 create mode 100644 tools/verification/rvgen/Makefile
 create mode 100644 tools/verification/rvgen/__main__.py
 rename tools/verification/{dot2 => rvgen}/dot2c (97%)
 rename tools/verification/{dot2 => rvgen/rvgen}/automata.py (100%)
 create mode 100644 tools/verification/rvgen/rvgen/container.py
 rename tools/verification/{dot2 => rvgen/rvgen}/dot2c.py (92%)
 create mode 100644 tools/verification/rvgen/rvgen/dot2k.py
 rename tools/verification/{dot2/dot2k.py => rvgen/rvgen/generator.py} (51%)
 create mode 100644 tools/verification/rvgen/rvgen/ltl2ba.py
 create mode 100644 tools/verification/rvgen/rvgen/ltl2k.py
 rename tools/verification/{dot2/dot2k_templates => rvgen/rvgen/templates}/Kconfig (100%)
 create mode 100644 tools/verification/rvgen/rvgen/templates/container/Kconfig
 rename tools/verification/{dot2/dot2k_templates/main_container.c => rvgen/rvgen/templates/container/main.c} (92%)
 rename tools/verification/{dot2/dot2k_templates/main_container.h => rvgen/rvgen/templates/container/main.h} (100%)
 rename tools/verification/{dot2/dot2k_templates => rvgen/rvgen/templates/dot2k}/main.c (96%)
 rename tools/verification/{dot2/dot2k_templates => rvgen/rvgen/templates/dot2k}/trace.h (100%)
 create mode 100644 tools/verification/rvgen/rvgen/templates/ltl2k/main.c
 create mode 100644 tools/verification/rvgen/rvgen/templates/ltl2k/trace.h
---------------------------

Re: [GIT PULL] runtime verification: Updates for 6.17

[pr-tracker-bot]

The pull request you sent on Tue, 29 Jul 2025 17:43:32 -0400:

> git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace.git trace-rv-6.17

has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/4ff261e725d7376c12e745fdbe8a33cd6dbd5a83

Thank you!

-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html

Re: [GIT PULL] runtime verification: Updates for 6.17

[Richard Weinberger]

[Adding ARM folks]

On Tue, Jul 29, 2025 at 11:43 PM Steven Rostedt <rostedt@goodmis.org> wrote:
> Linus,
>
> Runtime verification changes for 6.17

[...]

> - Add a vpanic() to allow for va_list to be passed to panic.

This change is causing a regression on ARM32.
panic() no longer shows a proper stack trace.

With this change:
[    2.943690] Kernel panic - not syncing: VFS: Unable to mount root
fs on unknown-block(0,0)
[    2.950101] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted
6.17.0-rc1 #176 NONE
[    2.950509] Hardware name: Generic DT based system
[    2.950865] Call trace:
[    2.951722]  unwind_backtrace from show_stack+0x18/0x1c
[    2.953111]  show_stack from dump_stack_lvl+0x54/0x68
[    2.953312]  dump_stack_lvl from vpanic+0xf8/0x388
[    2.953526]  vpanic from __do_trace_suspend_resume+0x0/0x50

With commit  3f045de7f5 "panic: Add vpanic()" reverted:
[    2.680077] Kernel panic - not syncing: VFS: Unable to mount root
fs on unknown-block(0,0)
[    2.690241] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted
6.17.0-rc1-dirty #177 NONE
[    2.690832] Hardware name: Generic DT based system
[    2.691339] Call trace:
[    2.692530]  unwind_backtrace from show_stack+0x18/0x1c
[    2.694283]  show_stack from dump_stack_lvl+0x54/0x68
[    2.694536]  dump_stack_lvl from panic+0x110/0x31c
[    2.694944]  panic from mount_root_generic+0x1fc/0x2ac
[    2.695399]  mount_root_generic from prepare_namespace+0x200/0x258
[    2.695723]  prepare_namespace from kernel_init+0x20/0x140
[    2.695974]  kernel_init from ret_from_fork+0x14/0x28

The change itself looks fine, so I suspect it uncovers a deeper
problem with panic().
Maybe it changes the stack layout enough such that dump_stack() on
ARM32 is no longer
able to walk it? I'm using gcc 15.1.1, if this matters.

To reproduce the issue using qemu, build a multi_v7_defconfig zImage and run:
qemu-system-arm -m 512 -M virt -kernel arch/arm/boot/zImage
-no-reboot -nographic -append "console=ttyAMA0 panic=1"
It will fail to mount a rootfs and panic.

-- 
Thanks,
//richard

Re: [GIT PULL] runtime verification: Updates for 6.17

[Richard Weinberger]

[Re-sending with Russel's correct mail address]

On Sat, Aug 16, 2025 at 2:04 PM Richard Weinberger
<richard.weinberger@gmail.com> wrote:
>
> [Adding ARM folks]
>
> On Tue, Jul 29, 2025 at 11:43 PM Steven Rostedt <rostedt@goodmis.org> wrote:
> > Linus,
> >
> > Runtime verification changes for 6.17
>
> [...]
>
> > - Add a vpanic() to allow for va_list to be passed to panic.
>
> This change is causing a regression on ARM32.
> panic() no longer shows a proper stack trace.
>
> With this change:
> [    2.943690] Kernel panic - not syncing: VFS: Unable to mount root
> fs on unknown-block(0,0)
> [    2.950101] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted
> 6.17.0-rc1 #176 NONE
> [    2.950509] Hardware name: Generic DT based system
> [    2.950865] Call trace:
> [    2.951722]  unwind_backtrace from show_stack+0x18/0x1c
> [    2.953111]  show_stack from dump_stack_lvl+0x54/0x68
> [    2.953312]  dump_stack_lvl from vpanic+0xf8/0x388
> [    2.953526]  vpanic from __do_trace_suspend_resume+0x0/0x50
>
> With commit  3f045de7f5 "panic: Add vpanic()" reverted:
> [    2.680077] Kernel panic - not syncing: VFS: Unable to mount root
> fs on unknown-block(0,0)
> [    2.690241] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted
> 6.17.0-rc1-dirty #177 NONE
> [    2.690832] Hardware name: Generic DT based system
> [    2.691339] Call trace:
> [    2.692530]  unwind_backtrace from show_stack+0x18/0x1c
> [    2.694283]  show_stack from dump_stack_lvl+0x54/0x68
> [    2.694536]  dump_stack_lvl from panic+0x110/0x31c
> [    2.694944]  panic from mount_root_generic+0x1fc/0x2ac
> [    2.695399]  mount_root_generic from prepare_namespace+0x200/0x258
> [    2.695723]  prepare_namespace from kernel_init+0x20/0x140
> [    2.695974]  kernel_init from ret_from_fork+0x14/0x28
>
> The change itself looks fine, so I suspect it uncovers a deeper
> problem with panic().
> Maybe it changes the stack layout enough such that dump_stack() on
> ARM32 is no longer
> able to walk it? I'm using gcc 15.1.1, if this matters.
>
> To reproduce the issue using qemu, build a multi_v7_defconfig zImage and run:
> qemu-system-arm -m 512 -M virt -kernel arch/arm/boot/zImage
> -no-reboot -nographic -append "console=ttyAMA0 panic=1"
> It will fail to mount a rootfs and panic.
>
> --
> Thanks,
> //richard



-- 
Thanks,
//richard

Re: [GIT PULL] runtime verification: Updates for 6.17

[Russell King (Oracle)]

On Sat, Aug 16, 2025 at 02:04:34PM +0200, Richard Weinberger wrote:
> [Adding ARM folks]
> 
> On Tue, Jul 29, 2025 at 11:43 PM Steven Rostedt <rostedt@goodmis.org> wrote:
> > Linus,
> >
> > Runtime verification changes for 6.17
> 
> [...]
> 
> > - Add a vpanic() to allow for va_list to be passed to panic.
> 
> This change is causing a regression on ARM32.
> panic() no longer shows a proper stack trace.
> 
> With this change:
> [    2.943690] Kernel panic - not syncing: VFS: Unable to mount root
> fs on unknown-block(0,0)
> [    2.950101] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted
> 6.17.0-rc1 #176 NONE
> [    2.950509] Hardware name: Generic DT based system
> [    2.950865] Call trace:
> [    2.951722]  unwind_backtrace from show_stack+0x18/0x1c
> [    2.953111]  show_stack from dump_stack_lvl+0x54/0x68
> [    2.953312]  dump_stack_lvl from vpanic+0xf8/0x388
> [    2.953526]  vpanic from __do_trace_suspend_resume+0x0/0x50

Please lookup the value of __do_trace_suspend_resume in System.map.

> 
> With commit  3f045de7f5 "panic: Add vpanic()" reverted:
> [    2.680077] Kernel panic - not syncing: VFS: Unable to mount root
> fs on unknown-block(0,0)
> [    2.690241] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted
> 6.17.0-rc1-dirty #177 NONE
> [    2.690832] Hardware name: Generic DT based system
> [    2.691339] Call trace:
> [    2.692530]  unwind_backtrace from show_stack+0x18/0x1c
> [    2.694283]  show_stack from dump_stack_lvl+0x54/0x68
> [    2.694536]  dump_stack_lvl from panic+0x110/0x31c
> [    2.694944]  panic from mount_root_generic+0x1fc/0x2ac

Also please lookup mount_root_generic.

> Maybe it changes the stack layout enough such that dump_stack() on
> ARM32 is no longer
> able to walk it? I'm using gcc 15.1.1, if this matters.

Also please check whether you're using frame pointers or the unwinder
(CONFIG_UNWINDER_FRAME_POINTER or CONFIG_ARM_UNWIND).

Thanks.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!

Re: [GIT PULL] runtime verification: Updates for 6.17

[Richard Weinberger]

On Sat, Aug 16, 2025 at 3:44 PM Russell King (Oracle)
<linux@armlinux.org.uk> wrote:
> > With this change:
> > [    2.943690] Kernel panic - not syncing: VFS: Unable to mount root
> > fs on unknown-block(0,0)
> > [    2.950101] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted
> > 6.17.0-rc1 #176 NONE
> > [    2.950509] Hardware name: Generic DT based system
> > [    2.950865] Call trace:
> > [    2.951722]  unwind_backtrace from show_stack+0x18/0x1c
> > [    2.953111]  show_stack from dump_stack_lvl+0x54/0x68
> > [    2.953312]  dump_stack_lvl from vpanic+0xf8/0x388
> > [    2.953526]  vpanic from __do_trace_suspend_resume+0x0/0x50
>
> Please lookup the value of __do_trace_suspend_resume in System.map.

Hm, I see it twice in System.map, both times non-global.
$  grep __do_trace_suspend_resume System.map
c0303098 t __do_trace_suspend_resume
c03c5758 t __do_trace_suspend_resume

>
> >
> > With commit  3f045de7f5 "panic: Add vpanic()" reverted:
> > [    2.680077] Kernel panic - not syncing: VFS: Unable to mount root
> > fs on unknown-block(0,0)
> > [    2.690241] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted
> > 6.17.0-rc1-dirty #177 NONE
> > [    2.690832] Hardware name: Generic DT based system
> > [    2.691339] Call trace:
> > [    2.692530]  unwind_backtrace from show_stack+0x18/0x1c
> > [    2.694283]  show_stack from dump_stack_lvl+0x54/0x68
> > [    2.694536]  dump_stack_lvl from panic+0x110/0x31c
> > [    2.694944]  panic from mount_root_generic+0x1fc/0x2ac
>
> Also please lookup mount_root_generic.

$ grep mount_root_generic System.map
c1c01814 T mount_root_generic

>
> > Maybe it changes the stack layout enough such that dump_stack() on
> > ARM32 is no longer
> > able to walk it? I'm using gcc 15.1.1, if this matters.

In the meanwhile I tried also with gcc 8.5, same problem.

> Also please check whether you're using frame pointers or the unwinder
> (CONFIG_UNWINDER_FRAME_POINTER or CONFIG_ARM_UNWIND).

With CONFIG_UNWINDER_FRAME_POINTER the stack trace is sane,
so only CONFIG_ARM_UNWIND is broken.
I kind of expected it the other way around...

-- 
Thanks,
//richard

Re: [GIT PULL] runtime verification: Updates for 6.17

[Richard Weinberger]

On Sat, Aug 16, 2025 at 9:26 PM Richard Weinberger
<richard.weinberger@gmail.com> wrote:
> > Also please check whether you're using frame pointers or the unwinder
> > (CONFIG_UNWINDER_FRAME_POINTER or CONFIG_ARM_UNWIND).
>
> With CONFIG_UNWINDER_FRAME_POINTER the stack trace is sane,
> so only CONFIG_ARM_UNWIND is broken.
> I kind of expected it the other way around...

I found time to gather more details, maybe it rings a bell...

The problem is related to va_list. As soon as va_list is passed as
function argument the
unwinder is no longer able to unwind correctly.
Unwinding vpanic() does not lead to a correct frame anymore.
Maybe because of va_list the function arguments are passed via stack?

The unwind table entries for panic() and vpanic() are:
0xc0302e20 <vpanic>: 0x808489b0
 Compact model index: 0
 0x84 0x89 pop {r4, r7, r11, r14}
 0xb0      finish

0xc0303190 <panic>: @0xc1c63ebc
 Compact model index: 1
 0x02      vsp = vsp + 12
 0x84 0x00 pop {r14}
 0xb1 0x0f pop {r0, r1, r2, r3}
 0xb0      finish

Disassembly of vpanic and panic:

c0303558 <vpanic>:
* @args: Arguments for the format string
*
* Display a message, then perform cleanups. This function never returns.
*/
void vpanic(const char *fmt, va_list args)
{
c0303558:       e92d4890        push    {r4, r7, fp, lr}
c030355c:       e52de004        push    {lr}            @ (str lr, [sp, #-4]!)
c0303560:       eb00a54f        bl      c032caa4 <__gnu_mcount_nc>
       static char buf[1024];
       long i, i_next = 0, len;
       int state = 0;
       int old_cpu, this_cpu;
       bool _crash_kexec_post_notifiers = crash_kexec_post_notifiers;
c0303564:       e3084464        movw    r4, #33892      @ 0x8464
c0303568:       e34c4222        movt    r4, #49698      @ 0xc222
{
c030356c:       e1a05000        mov     r5, r0
c0303570:       e1a07001        mov     r7, r1
       bool _crash_kexec_post_notifiers = crash_kexec_post_notifiers;
c0303574:       e5d4602c        ldrb    r6, [r4, #44]   @ 0x2c
c0303578:       e3560001        cmp     r6, #1
c030357c:       9a000002        bls     c030358c <vpanic+0x34>
c0303580:       e59f0324        ldr     r0, [pc, #804]  @ c03038ac
<vpanic+0x354>
c0303584:       e1a01006        mov     r1, r6
c0303588:       eb17f5b4        bl      c0900c60
<__ubsan_handle_load_invalid_value>

       if (panic_on_warn) {
c030358c:       e3043f08        movw    r3, #20232      @ 0x4f08
c0303590:       e34c31f0        movt    r3, #49648      @ 0xc1f0
c0303594:       e5932000        ldr     r2, [r3]
c0303598:       e3520000        cmp     r2, #0
                * This thread may hit another WARN() in the panic path.
                * Resetting this prevents additional WARN() from panicking the
                * system on this thread.  Other threads are blocked by the
                * panic_mutex in panic().
                */
               panic_on_warn = 0;
c030359c:       13a02000        movne   r2, #0
c03035a0:       15832000        strne   r2, [r3]
}

#define arch_local_irq_disable arch_local_irq_disable
static inline void arch_local_irq_disable(void)
{
       asm volatile(
c03035a4:       f10c0080        cpsid   i
        * Use the __builtin helper when available - this results in better
        * code, especially when using GCC in combination with the per-task
        * stack protector, as the compiler will recognize that it needs to
        * load the TLS register only once in every function.
        */
       cur = __builtin_thread_pointer();
c03035a8:       ee1d3f70        mrc     15, 0, r3, cr13, cr0, {3}
        * so go ahead.
        * `old_cpu == this_cpu' means we came from nmi_panic() which sets
        * panic_cpu to this CPU.  In this case, this is also the 1st CPU.
        */
       old_cpu = PANIC_CPU_INVALID;
       this_cpu = raw_smp_processor_id();
c03035ac:       e5932008        ldr     r2, [r3, #8]
{
#if defined(arch_atomic_cmpxchg)
       return arch_atomic_cmpxchg(v, old, new);
#elif defined(arch_atomic_cmpxchg_relaxed)
       int ret;
       __atomic_pre_full_fence();
c03035b0:       f57ff05b        dmb     ish

#if __LINUX_ARM_ARCH__ >= 7 && defined(CONFIG_SMP)
#define ARCH_HAS_PREFETCHW
static inline void prefetchw(const void *ptr)
{
       __asm__ __volatile__(
c03035b4:       e59f32f4        ldr     r3, [pc, #756]  @ c03038b0
<vpanic+0x358>
c03035b8:       f593f000        pldw    [r3]
       unsigned long res;

       prefetchw(&ptr->counter);

       do {
               __asm__ __volatile__("@ atomic_cmpxchg\n"
c03035bc:       e3e01000        mvn     r1, #0
c03035c0:       e1930f9f        ldrex   r0, [r3]
c03035c4:       e3a0c000        mov     ip, #0
c03035c8:       e1300001        teq     r0, r1
c03035cc:       0183cf92        strexeq ip, r2, [r3]
               "teq    %1, %4\n"
               "strexeq %0, %5, [%3]\n"
                   : "=&r" (res), "=&r" (oldval), "+Qo" (ptr->counter)
                   : "r" (&ptr->counter), "Ir" (old), "r" (new)
                   : "cc");
       } while (res);
c03035d0:       e35c0000        cmp     ip, #0
c03035d4:       1afffff9        bne     c03035c0 <vpanic+0x68>
       ret = arch_atomic_cmpxchg_relaxed(v, old, new);
       __atomic_post_full_fence();
c03035d8:       f57ff05b        dmb     ish

       /* atomic_try_cmpxchg updates old_cpu on failure */
       if (atomic_try_cmpxchg(&panic_cpu, &old_cpu, this_cpu)) {
               /* go ahead */
       } else if (old_cpu != this_cpu)
c03035dc:       e1520000        cmp     r2, r0
c03035e0:       13700001        cmnne   r0, #1
c03035e4:       0a000000        beq     c03035ec <vpanic+0x94>
               panic_smp_self_stop();
c03035e8:       eb00a127        bl      c032ba8c <panic_smp_self_stop>

       console_verbose();
c03035ec:       eb034d54        bl      c03d6b44 <console_verbose>
       bust_spinlocks(1);
c03035f0:       e3a00001        mov     r0, #1
c03035f4:       eb15167b        bl      c0848fe8 <bust_spinlocks>
       len = vscnprintf(buf, sizeof(buf), fmt, args);
c03035f8:       e59f02b4        ldr     r0, [pc, #692]  @ c03038b4
<vpanic+0x35c>
c03035fc:       e1a03007        mov     r3, r7
c0303600:       e1a02005        mov     r2, r5
c0303604:       e3a01b01        mov     r1, #1024       @ 0x400
       bool _crash_kexec_post_notifiers = crash_kexec_post_notifiers;
c0303608:       e2066001        and     r6, r6, #1
       len = vscnprintf(buf, sizeof(buf), fmt, args);
c030360c:       eb45cce9        bl      c14769b8 <vscnprintf>

       if (len && buf[len - 1] == '\n')
c0303610:       e3500000        cmp     r0, #0
c0303614:       0a000011        beq     c0303660 <vpanic+0x108>
c0303618:       e2405001        sub     r5, r0, #1
c030361c:       e3550b01        cmp     r5, #1024       @ 0x400
c0303620:       3a000002        bcc     c0303630 <vpanic+0xd8>
c0303624:       e59f028c        ldr     r0, [pc, #652]  @ c03038b8
<vpanic+0x360>
c0303628:       e1a01005        mov     r1, r5
c030362c:       eb17f56c        bl      c0900be4 <__ubsan_handle_out_of_bounds>
c0303630:       e0843005        add     r3, r4, r5
c0303634:       e5d33030        ldrb    r3, [r3, #48]   @ 0x30
c0303638:       e353000a        cmp     r3, #10
c030363c:       1a000007        bne     c0303660 <vpanic+0x108>
               buf[len - 1] = '\0';
c0303640:       e3550b01        cmp     r5, #1024       @ 0x400
c0303644:       3a000002        bcc     c0303654 <vpanic+0xfc>
c0303648:       e59f026c        ldr     r0, [pc, #620]  @ c03038bc
<vpanic+0x364>
c030364c:       e1a01005        mov     r1, r5
c0303650:       eb17f563        bl      c0900be4 <__ubsan_handle_out_of_bounds>
c0303654:       e0845005        add     r5, r4, r5
c0303658:       e3a03000        mov     r3, #0
c030365c:       e5c53030        strb    r3, [r5, #48]   @ 0x30

       pr_emerg("Kernel panic - not syncing: %s\n", buf);
c0303660:       e59f124c        ldr     r1, [pc, #588]  @ c03038b4
<vpanic+0x35c>
c0303664:       e30e07b4        movw    r0, #59316      @ 0xe7b4
c0303668:       e34c019d        movt    r0, #49565      @ 0xc19d
c030366c:       eb0003c7        bl      c0304590 <_printk>
       /*
        * Unlike the bitops with the '__' prefix above, this one *is* atomic,
        * so `volatile` must always stay here with no cast-aways. See
        * `Documentation/atomic_bitops.txt` for the details.
        */
       return 1UL & (addr[BIT_WORD(nr)] >> (nr & (BITS_PER_LONG-1)));
c0303670:       e5943008        ldr     r3, [r4, #8]
#ifdef CONFIG_DEBUG_BUGVERBOSE
       /*
        * Avoid nested stack-dumping if a panic occurs during oops processing
        */
       if (!test_taint(TAINT_DIE) && oops_in_progress <= 1)
c0303674:       e3130080        tst     r3, #128        @ 0x80
c0303678:       1a000005        bne     c0303694 <vpanic+0x13c>
c030367c:       e30a3284        movw    r3, #41604      @ 0xa284
c0303680:       e34c3222        movt    r3, #49698      @ 0xc222
c0303684:       e5933000        ldr     r3, [r3]
c0303688:       e3530001        cmp     r3, #1
c030368c:       ca000000        bgt     c0303694 <vpanic+0x13c>
               dump_stack();
c0303690:       eb006216        bl      c031bef0 <dump_stack>
        * If we want to run this after calling panic_notifiers, pass
        * the "crash_kexec_post_notifiers" option to the kernel.
        *
        * Bypass the panic_cpu check and call __crash_kexec directly.
        */
       if (!_crash_kexec_post_notifiers)
c0303694:       e3560000        cmp     r6, #0
c0303698:       1a000001        bne     c03036a4 <vpanic+0x14c>
               __crash_kexec(NULL);
c030369c:       e1a00006        mov     r0, r6
c03036a0:       eb050e94        bl      c04470f8 <__crash_kexec>
       if (panic_print & SYS_INFO_ALL_CPU_BT) {
c03036a4:       e5943000        ldr     r3, [r4]
c03036a8:       e3130040        tst     r3, #64 @ 0x40
c03036ac:       0a000007        beq     c03036d0 <vpanic+0x178>
* to allow calling code to fall back to some other mechanism:
*/
#ifdef arch_trigger_cpumask_backtrace
static inline bool trigger_all_cpu_backtrace(void)
{
       arch_trigger_cpumask_backtrace(cpu_online_mask, -1);
c03036b0:       e3e01000        mvn     r1, #0
c03036b4:       e3040f14        movw    r0, #20244      @ 0x4f14
c03036b8:       e34c01f0        movt    r0, #49648      @ 0xc1f0
               panic_triggering_all_cpu_backtrace = true;
c03036bc:       e3a03001        mov     r3, #1
c03036c0:       e5c43430        strb    r3, [r4, #1072] @ 0x430
c03036c4:       eb00a103        bl      c032bad8
<arch_trigger_cpumask_backtrace>
               panic_triggering_all_cpu_backtrace = false;
c03036c8:       e3a03000        mov     r3, #0
c03036cc:       e5c43430        strb    r3, [r4, #1072] @ 0x430
       if (!crash_kexec)
c03036d0:       e3560000        cmp     r6, #0
c03036d4:       1a000001        bne     c03036e0 <vpanic+0x188>
               smp_send_stop();
c03036d8:       eb00a0b0        bl      c032b9a0 <smp_send_stop>
c03036dc:       ea000000        b       c03036e4 <vpanic+0x18c>
               crash_smp_send_stop();
c03036e0:       eb00a7e6        bl      c032d680 <crash_smp_send_stop>

       panic_other_cpus_shutdown(_crash_kexec_post_notifiers);

       printk_legacy_allow_panic_sync();
c03036e4:       eb035931        bl      c03d9bb0
<printk_legacy_allow_panic_sync>

       /*
        * Run any panic handlers, including those that might need to
        * add information to the kmsg dump output.
        */
       atomic_notifier_call_chain(&panic_notifier_list, 0, buf);
c03036e8:       e59f21c4        ldr     r2, [pc, #452]  @ c03038b4
<vpanic+0x35c>
c03036ec:       e3a01000        mov     r1, #0
c03036f0:       e59f01c8        ldr     r0, [pc, #456]  @ c03038c0
<vpanic+0x368>
c03036f4:       eb024681        bl      c0395100 <atomic_notifier_call_chain>

       sys_info(panic_print);
c03036f8:       e5940000        ldr     r0, [r4]
c03036fc:       eb45b2ff        bl      c1470300 <sys_info>

       kmsg_dump_desc(KMSG_DUMP_PANIC, buf);
c0303700:       e59f11ac        ldr     r1, [pc, #428]  @ c03038b4
<vpanic+0x35c>
c0303704:       e3a00001        mov     r0, #1
c0303708:       eb036048        bl      c03db830 <kmsg_dump_desc>
        * Note: since some panic_notifiers can make crashed kernel
        * more unstable, it can increase risks of the kdump failure too.
        *
        * Bypass the panic_cpu check and call __crash_kexec directly.
        */
       if (_crash_kexec_post_notifiers)
c030370c:       e3560000        cmp     r6, #0
c0303710:       0a000001        beq     c030371c <vpanic+0x1c4>
               __crash_kexec(NULL);
c0303714:       e3a00000        mov     r0, #0
c0303718:       eb050e76        bl      c04470f8 <__crash_kexec>

       console_unblank();
c030371c:       eb035ede        bl      c03db29c <console_unblank>
        * buffer.  Try to acquire the lock then release it regardless of the
        * result.  The release will also print the buffers out.  Locks debug
        * should be disabled to avoid reporting bad unlock balance when
        * panic() is not being callled from OOPS.
        */
       debug_locks_off();
c0303720:       eb151570        bl      c0848ce8 <debug_locks_off>
       console_flush_on_panic(CONSOLE_FLUSH_PENDING);
c0303724:       e3a00000        mov     r0, #0
c0303728:       eb035f7a        bl      c03db518 <console_flush_on_panic>

       if ((panic_print & SYS_INFO_PANIC_CONSOLE_REPLAY) ||
c030372c:       e5943000        ldr     r3, [r4]
c0303730:       e3130020        tst     r3, #32
c0303734:       0a000002        beq     c0303744 <vpanic+0x1ec>
               panic_console_replay)
               console_flush_on_panic(CONSOLE_REPLAY_ALL);
c0303738:       e3a00001        mov     r0, #1
c030373c:       eb035f75        bl      c03db518 <console_flush_on_panic>
c0303740:       ea000007        b       c0303764 <vpanic+0x20c>
       if ((panic_print & SYS_INFO_PANIC_CONSOLE_REPLAY) ||
c0303744:       e5d4543c        ldrb    r5, [r4, #1084] @ 0x43c
c0303748:       e3550001        cmp     r5, #1
c030374c:       9a000002        bls     c030375c <vpanic+0x204>
c0303750:       e59f016c        ldr     r0, [pc, #364]  @ c03038c4
<vpanic+0x36c>
c0303754:       e1a01005        mov     r1, r5
c0303758:       eb17f540        bl      c0900c60
<__ubsan_handle_load_invalid_value>
c030375c:       e3150001        tst     r5, #1
c0303760:       1afffff4        bne     c0303738 <vpanic+0x1e0>

       if (!panic_blink)
c0303764:       e5943440        ldr     r3, [r4, #1088] @ 0x440
c0303768:       e30e8b08        movw    r8, #60168      @ 0xeb08
c030376c:       e34c81b7        movt    r8, #49591      @ 0xc1b7
               panic_blink = no_blink;

       if (panic_timeout > 0) {
c0303770:       e5941444        ldr     r1, [r4, #1092] @ 0x444
       if (!panic_blink)
c0303774:       e3530000        cmp     r3, #0
               panic_blink = no_blink;
c0303778:       03053010        movweq  r3, #20496      @ 0x5010
c030377c:       034c3036        movteq  r3, #49206      @ 0xc036
c0303780:       05843440        streq   r3, [r4, #1088] @ 0x440
       if (panic_timeout > 0) {
c0303784:       e3510000        cmp     r1, #0
       int state = 0;
c0303788:       d3a05000        movle   r5, #0
       long i, i_next = 0, len;
c030378c:       d1a07005        movle   r7, r5
       if (panic_timeout > 0) {
c0303790:       da00001c        ble     c0303808 <vpanic+0x2b0>
       int state = 0;
c0303794:       e3a05000        mov     r5, #0
                       touch_nmi_watchdog();
                       if (i >= i_next) {
                               i += panic_blink(state ^= 1);
                               i_next = i + 3600 / PANIC_BLINK_SPD;
                       }
                       mdelay(PANIC_TIMER_STEP);
c0303798:       e30cbb60        movw    fp, #52064      @ 0xcb60
c030379c:       e340bccc        movt    fp, #3276       @ 0xccc
       long i, i_next = 0, len;
c03037a0:       e1a07005        mov     r7, r5
               for (i = 0; i < panic_timeout * 1000; i += PANIC_TIMER_STEP) {
c03037a4:       e1a06005        mov     r6, r5
c03037a8:       e3a0affa        mov     sl, #1000       @ 0x3e8
               pr_emerg("Rebooting in %d seconds..\n", panic_timeout);
c03037ac:       e30e07d8        movw    r0, #59352      @ 0xe7d8
c03037b0:       e34c019d        movt    r0, #49565      @ 0xc19d
c03037b4:       eb000375        bl      c0304590 <_printk>
               for (i = 0; i < panic_timeout * 1000; i += PANIC_TIMER_STEP) {
c03037b8:       e5941444        ldr     r1, [r4, #1092] @ 0x444
c03037bc:       e003019a        mul     r3, sl, r1
c03037c0:       e1530006        cmp     r3, r6
c03037c4:       da00000f        ble     c0303808 <vpanic+0x2b0>
                       if (i >= i_next) {
c03037c8:       e1560007        cmp     r6, r7
c03037cc:       ba000005        blt     c03037e8 <vpanic+0x290>
                               i += panic_blink(state ^= 1);
c03037d0:       e2255001        eor     r5, r5, #1
c03037d4:       e5943440        ldr     r3, [r4, #1088] @ 0x440
c03037d8:       e1a00005        mov     r0, r5
c03037dc:       e12fff33        blx     r3
c03037e0:       e0866000        add     r6, r6, r0
                               i_next = i + 3600 / PANIC_BLINK_SPD;
c03037e4:       e28670c8        add     r7, r6, #200    @ 0xc8
{
c03037e8:       e3a09064        mov     r9, #100        @ 0x64
                       mdelay(PANIC_TIMER_STEP);
c03037ec:       e5983004        ldr     r3, [r8, #4]
c03037f0:       e1a0000b        mov     r0, fp
c03037f4:       e12fff33        blx     r3
c03037f8:       e2599001        subs    r9, r9, #1
c03037fc:       1afffffa        bne     c03037ec <vpanic+0x294>
               for (i = 0; i < panic_timeout * 1000; i += PANIC_TIMER_STEP) {
c0303800:       e2866064        add     r6, r6, #100    @ 0x64
c0303804:       eaffffeb        b       c03037b8 <vpanic+0x260>
               }
       }
       if (panic_timeout != 0) {
c0303808:       e3510000        cmp     r1, #0
c030380c:       0a000007        beq     c0303830 <vpanic+0x2d8>
               /*
                * This will not be a clean reboot, with everything
                * shutting down.  But if there is a chance of
                * rebooting the system it will be rebooted.
                */
               if (panic_reboot_mode != REBOOT_UNDEFINED)
c0303810:       e3083194        movw    r3, #33172      @ 0x8194
c0303814:       e34c31f2        movt    r3, #49650      @ 0xc1f2
c0303818:       e5932000        ldr     r2, [r3]
c030381c:       e3720001        cmn     r2, #1
                       reboot_mode = panic_reboot_mode;
c0303820:       13083198        movwne  r3, #33176      @ 0x8198
c0303824:       134c31f2        movtne  r3, #49650      @ 0xc1f2
c0303828:       15832000        strne   r2, [r3]
               emergency_restart();
c030382c:       eb024cbd        bl      c0396b28 <emergency_restart>
       }
#endif
#if defined(CONFIG_S390)
       disabled_wait();
#endif
       pr_emerg("---[ end Kernel panic - not syncing: %s ]---\n", buf);
c0303830:       e59f107c        ldr     r1, [pc, #124]  @ c03038b4
<vpanic+0x35c>
c0303834:       e30e07f8        movw    r0, #59384      @ 0xe7f8
c0303838:       e34c019d        movt    r0, #49565      @ 0xc19d
c030383c:       eb000353        bl      c0304590 <_printk>

       /* Do not scroll important messages printed above */
       suppress_printk = 1;
c0303840:       e3043f70        movw    r3, #20336      @ 0x4f70
c0303844:       e34c31f0        movt    r3, #49648      @ 0xc1f0
       /*
        * The final messages may not have been printed if in a context that
        * defers printing (such as NMI) and irq_work is not available.
        * Explicitly flush the kernel log buffer one last time.
        */
       console_flush_on_panic(CONSOLE_FLUSH_PENDING);
c0303848:       e3a00000        mov     r0, #0
       suppress_printk = 1;
c030384c:       e3a02001        mov     r2, #1
c0303850:       e5832000        str     r2, [r3]
       console_flush_on_panic(CONSOLE_FLUSH_PENDING);
c0303854:       eb035f2f        bl      c03db518 <console_flush_on_panic>
       nbcon_atomic_flush_unsafe();
c0303858:       eb03684e        bl      c03dd998 <nbcon_atomic_flush_unsafe>
       asm volatile(
c030385c:       f1080080        cpsie   i
               touch_softlockup_watchdog();
               if (i >= i_next) {
                       i += panic_blink(state ^= 1);
                       i_next = i + 3600 / PANIC_BLINK_SPD;
               }
               mdelay(PANIC_TIMER_STEP);
c0303860:       e30cab60        movw    sl, #52064      @ 0xcb60
c0303864:       e340accc        movt    sl, #3276       @ 0xccc
       for (i = 0; ; i += PANIC_TIMER_STEP) {
c0303868:       e3a06000        mov     r6, #0
               if (i >= i_next) {
c030386c:       e1560007        cmp     r6, r7
c0303870:       ba000005        blt     c030388c <vpanic+0x334>
                       i += panic_blink(state ^= 1);
c0303874:       e2255001        eor     r5, r5, #1
c0303878:       e5943440        ldr     r3, [r4, #1088] @ 0x440
c030387c:       e1a00005        mov     r0, r5
c0303880:       e12fff33        blx     r3
c0303884:       e0866000        add     r6, r6, r0
                       i_next = i + 3600 / PANIC_BLINK_SPD;
c0303888:       e28670c8        add     r7, r6, #200    @ 0xc8
       for (i = 0; ; i += PANIC_TIMER_STEP) {
c030388c:       e3a09064        mov     r9, #100        @ 0x64
               mdelay(PANIC_TIMER_STEP);
c0303890:       e5983004        ldr     r3, [r8, #4]
c0303894:       e1a0000a        mov     r0, sl
c0303898:       e12fff33        blx     r3
c030389c:       e2599001        subs    r9, r9, #1
c03038a0:       1afffffa        bne     c0303890 <vpanic+0x338>
       for (i = 0; ; i += PANIC_TIMER_STEP) {
c03038a4:       e2866064        add     r6, r6, #100    @ 0x64
               touch_softlockup_watchdog();
c03038a8:       eaffffef        b       c030386c <vpanic+0x314>
c03038ac:       c1f231bc        .word   0xc1f231bc
c03038b0:       c1f231cc        .word   0xc1f231cc
c03038b4:       c2228494        .word   0xc2228494
c03038b8:       c1f231d0        .word   0xc1f231d0
c03038bc:       c1f231e4        .word   0xc1f231e4
c03038c0:       c2228898        .word   0xc2228898
c03038c4:       c1f231f8        .word   0xc1f231f8

c03038c8 <panic>:
}
EXPORT_SYMBOL(vpanic);

/* Identical to vpanic(), except it takes variadic arguments instead
of va_list */
void panic(const char *fmt, ...)
{
c03038c8:       e92d000f        push    {r0, r1, r2, r3}
c03038cc:       e52de004        push    {lr}            @ (str lr, [sp, #-4]!)
c03038d0:       e24dd00c        sub     sp, sp, #12
c03038d4:       e52de004        push    {lr}            @ (str lr, [sp, #-4]!)
c03038d8:       eb00a471        bl      c032caa4 <__gnu_mcount_nc>
c03038dc:       ee1d3f70        mrc     15, 0, r3, cr13, cr0, {3}
       va_list args;

       va_start(args, fmt);
c03038e0:       e28d1014        add     r1, sp, #20
       vpanic(fmt, args);
c03038e4:       e59d0010        ldr     r0, [sp, #16]
{
c03038e8:       e5932508        ldr     r2, [r3, #1288] @ 0x508
c03038ec:       e58d2004        str     r2, [sp, #4]
c03038f0:       e3a02000        mov     r2, #0
       va_start(args, fmt);
c03038f4:       e58d1000        str     r1, [sp]
       vpanic(fmt, args);
c03038f8:       ebffff16        bl      c0303558 <vpanic>

-- 
Thanks,
//richard

Re: [GIT PULL] runtime verification: Updates for 6.17

[Richard Weinberger]

On Mon, Aug 18, 2025 at 10:41 AM Richard Weinberger
<richard.weinberger@gmail.com> wrote:
>
> On Sat, Aug 16, 2025 at 9:26 PM Richard Weinberger
> <richard.weinberger@gmail.com> wrote:
> > > Also please check whether you're using frame pointers or the unwinder
> > > (CONFIG_UNWINDER_FRAME_POINTER or CONFIG_ARM_UNWIND).
> >
> > With CONFIG_UNWINDER_FRAME_POINTER the stack trace is sane,
> > so only CONFIG_ARM_UNWIND is broken.
> > I kind of expected it the other way around...
>
> I found time to gather more details, maybe it rings a bell...
>
> The problem is related to va_list. As soon as va_list is passed as
> function argument the
> unwinder is no longer able to unwind correctly.
> Unwinding vpanic() does not lead to a correct frame anymore.
> Maybe because of va_list the function arguments are passed via stack?

Time to post an update, there is a problem in the ARM unwinder.
When the last instruction of a function is BL, LR points outside of
the function,
this confuses the unwinder.

This branch contains two WIP patches from Russel and myself to address
the issue:
https://git.kernel.org/pub/scm/linux/kernel/git/rw/misc.git/log/?h=arm_unwind_fix

-- 
Thanks,
//richard