[RFC PATCH v3 00/17] s390: SFrame user space unwinding

Jens Remus
2025-12-08T17:16:38Z

This RFC series adds s390 support for unwinding of user space using
SFrame. It is based on Josh's, Steven's, and my work (see prerequisites
below). The generic unwind user (sframe) frameworks are extended to
enable support for a few s390-particularities (see patches 9-12),
including unwinding of user space using back chain (see patches 15-17).
The latter could be broken apart as a separate patch series.

Changes in RFC v3:
- Rebase on and include my unwind user cleanup series v4, which includes
a simplification of unwind_user_word_size() on x86. (Linus)
- Implement unwinding of user space using s390 back chain using unwind
user fp instead of introducing a new unwind user backchain. (Josh)

Changes in RFC v2:
- Rebased on latest "unwind user" enhancements from Peter Zijlstra and
my latest "unwind user sframe" series v12.
- Incorporated RFC v1 review feedback.
- No new config options (except for unwind user backchain).

Motivation:

On s390 unwinding using frame pointer (FP) is unsupported, because of
lack of proper s390 64-bit (s390x) ABI specification and compiler
support. The ABI does only specify a "preferred" FP register. Both GCC
and Clang, regardless of compiler option -fno-omit-frame-pointer, setup
the preferred FP register as late as possible, which usually is after
static stack allocation, so that the CFA cannot be deduced from the FP
without any further data, such as provided by DWARF CFI or SFrame.

In theory there is a s390-specific alternative of unwinding using
back chain (compiler option -mbackchain), but this has its own
limitations. Ubuntu is currently the only distribution that that
builds user space with back chain.

As a consequence the Kernel stack tracer cannot unwind user space
(except if it is built with back chain). Recording call graphs of user
space using perf is limited to stack dump sampling (i.e. perf record
--call-graph dwarf), which generates a fairly large amount of data and
has limitations.

Initial testing of recording call graphs using perf using the s390
support for SFrame provided by RFC v1 of this series shows that
data size notably improves:

perf record data size is greatly reduced (smaller perf.data):

SFrame (--call-graph fp):
# perf record -F 9999 --call-graph fp objdump -wdWF objdump
[ perf record: Woken up 9 times to write data ]
[ perf record: Captured and wrote 2.498 MB perf.data (10891 samples) ]

Stack sampling (--call-graph dwarf) with a default stack size of 8192:
# perf record -F 9999 --call-graph dwarf objdump -wdWF objdump
[ perf record: Woken up 270 times to write data ]
[ perf record: Captured and wrote 67.467 MB perf.data (8241 samples) ]

Prerequirements:

This RFC series applies on top of the latest unwind user sframe series
"[PATCH v12 00/13] unwind_deferred: Implement sframe handling":
https://lore.kernel.org/all/20251119132323.1281768-1-jremus@linux.ibm.com/

It depends on binutils 2.45 to build executables and libraries
(e.g. vDSO) with SFrame on s390, including the latest SFrame V2 with
PC-relative FDE encoding. Optionally a binutils mainline build is
required for SFrame V2 with outermost frame indication.

The unwind user sframe series depends on a Glibc patch from Josh, that
adds support for the prctls introduced in the Kernel:
https://lore.kernel.org/all/20250122023517.lmztuocecdjqzfhc@jpoimboe/
Note that Josh's Glibc patch needs to be adjusted for the updated prctl
numbers from "[PATCH v12 13/13] unwind_user/sframe: Add prctl()
interface for registering .sframe sections":
https://lore.kernel.org/all/20251119132323.1281768-14-jremus@linux.ibm.com/

Overview:

Patches 1-4 originate from my "[PATCH v4 0/3] unwind_user: Cleanups"
series and can be ignored here (please review in the respective series):
https://lore.kernel.org/all/20251208160352.1363040-1-jremus@linux.ibm.com/

Patch 5 aligns asm/dwarf.h to x86 asm/dwarf2.h.

Patch 6 replicates Josh's x86 patch "x86/asm: Avoid emitting DWARF
CFI for non-VDSO" for s390.

Patch 7 changes the build of the vDSO on s390 to keep the function
symbols for stack tracing purposes.

Patch 8 replicates Josh's patch "x86/vdso: Enable sframe generation
in VDSO" for s390. It enables generation of SFrame stack trace
information (.sframe section) for the vDSO if the assembler supports it.

Patches 9-12 enable Josh's generic unwind user (sframe) frameworks to
support the following s390 particularities:

- Patch 9 adds support for architectures that define their CFA as SP at
callsite + offset.

- Patch 10 adds support support for architectures that pass the return
address (RA) in a register instead of on the stack and that do not
necessarily save the RA on the stack (or in another register) in the
topmost frame (e.g. in the prologue or in leaf functions).

- Patch 11 adds support for architectures that save RA/FP in other
registers instead of on the stack, e.g. in leaf functions.

- Patch 12 adds support for architectures that store the CFA offset
from CFA base register (e.g. SP or FP) in SFrame encoded. For
instance on s390 the CFA offset is stored adjusted by -160 and
then scaled down by 8 to enable and improve the use of signed 8-bit
SFrame offsets (i.e. CFA, RA, and FP offset).

Patch 13 introduces frame_pointer() in ptrace on s390, which is a
prerequisite for enabling unwind user.

Patch 14 adds support for unwinding of user space using SFrame on
s390. It leverages the extensions of the generic unwind user (sframe)
frameworks from patches 8-11.

Patches 15-16 enable unwind user (fp) to support the following s390
back chain particularities:

- Patch 15 introduces FP/RA location unknown, which enables s390
back chain unwinding, which cannot unwind FP.

- Patch 16 enables sophisticated architecture-specific initialization
of the FP frame, which enables s390 back chain unwinding to provide
dynamic information.

Patch 17 adds support for unwinding of user space using back chain on
s390. Main reasons to support back chain on s390 are:
- With Ubuntu there is a major distribution that builds user space with
back chain.
- Java JREs, such as OpenJDK, do maintain the back chain in jitted code.

Limitations:

Unwinding of user space using back chain cannot - by design - restore
the FP. Therefore unwinding of subsequent frames using e.g. SFrame may
fail, if the FP is the CFA base register.

Thanks and regards,
Jens

Jens Remus (17):
unwind_user: Enhance comments on get CFA, FP, and RA
unwind_user/fp: Use dummies instead of ifdef
x86/unwind_user: Guard unwind_user_word_size() by UNWIND_USER
x86/unwind_user: Simplify unwind_user_word_size()
s390: asm/dwarf.h should only be included in assembly files
s390/vdso: Avoid emitting DWARF CFI for non-vDSO
s390/vdso: Keep function symbols in vDSO
s390/vdso: Enable SFrame generation in vDSO
unwind_user: Enable archs that define CFA = SP_callsite + offset
unwind_user: Enable archs that pass RA in a register
unwind_user: Enable archs that save RA/FP in other registers
unwind_user/sframe: Enable archs with encoded SFrame CFA offsets
s390/ptrace: Provide frame_pointer()
s390/unwind_user/sframe: Enable HAVE_UNWIND_USER_SFRAME
unwind_user: Introduce FP/RA location unknown
unwind_user/fp: Use arch-specific helper to initialize FP frame
s390/unwind_user/fp: Enable back chain unwinding of user space

arch/Kconfig | 3 +
arch/s390/Kconfig | 2 +
arch/s390/include/asm/dwarf.h | 53 ++++--
arch/s390/include/asm/ptrace.h | 18 +-
arch/s390/include/asm/unwind_user.h | 183 +++++++++++++++++++++
arch/s390/include/asm/unwind_user_sframe.h | 33 ++++
arch/s390/kernel/vdso64/Makefile | 9 +-
arch/s390/kernel/vdso64/vdso64.lds.S | 9 +
arch/x86/include/asm/unwind_user.h | 66 ++++++--
include/asm-generic/Kbuild | 1 +
include/asm-generic/unwind_user_sframe.h | 35 ++++
include/linux/unwind_user.h | 30 +++-
include/linux/unwind_user_types.h | 20 ++-
kernel/unwind/sframe.c | 13 +-
kernel/unwind/sframe.h | 14 ++
kernel/unwind/user.c | 83 ++++++----
16 files changed, 493 insertions(+), 79 deletions(-)
create mode 100644 arch/s390/include/asm/unwind_user.h
create mode 100644 arch/s390/include/asm/unwind_user_sframe.h
create mode 100644 include/asm-generic/unwind_user_sframe.h

--
2.51.0