Re: [PATCH v5 8/8] unwind: arm64: Use sframe to unwind interrupt frames

Re: [PATCH v5 8/8] unwind: arm64: Use sframe to unwind interrupt frames

[Mark Rutland]

On Mon, May 04, 2026 at 10:47:26AM +0200, Jens Remus wrote:
> Hello Mark,
> 
> I mostly have comments regarding your the SFrame related remarks.

Thanks for this; I have a few more questions and comments below.

> On 5/1/2026 6:46 PM, Mark Rutland wrote:
> > Thanks for putting this together. I think this is looking pretty good.
> > However, there are some things that aren't quite right and need some
> > work, which I've commented on below.
> 
> > (2) To make unwinding generally possible, we'll need to annotate some
> >     assembly functions as unwindable. We'll need to do that for string
> >     routines under lib/, and probably some crypto code, but we don't
> >     need to do that for most code in head.S, entry.S, etc.
> > 
> >     The vast majority of relevant assembly functions are leaf functions
> >     (where the return address is never moved out of the LR), so we can
> >     probably get away with a simple annotation for those that avoids the
> >     need for open-coded CFI directives everywhere.
> 
> Wrapping them in .cfi_startproc ... .cfi_endproc should do.  For instance
> by extending SYM_FUNC_START() and SYM_FUNC_END() or introducing flavors
> that do.  Or where you thinking of something else?

I was expecting we'd do something like that, either with distinct
versions, or some entirely separate annotation.

We can't override SYM_FUNC_START() or SYM_FUNC_END() since those are
also used for non-leaf functions. The bulk of the work is going to be
making sure we only annotate leaf functions specifically, which will
require some human analysis.

> > On Tue, Apr 28, 2026 at 06:36:43PM +0000, Dylan Hatch wrote:
> >> diff --git a/arch/arm64/include/asm/stacktrace/common.h b/arch/arm64/include/asm/stacktrace/common.h
> 
> >> @@ -21,6 +21,8 @@ struct stack_info {
> >>   *
> >>   * @fp:          The fp value in the frame record (or the real fp)
> >>   * @pc:          The lr value in the frame record (or the real lr)
> >> + * @sp:          The sp value at the call site of the current function.
> >> + * @unreliable:  Stacktrace is unreliable.
> >>   *
> >>   * @stack:       The stack currently being unwound.
> >>   * @stacks:      An array of stacks which can be unwound.
> >> @@ -29,7 +31,11 @@ struct stack_info {
> >>  struct unwind_state {
> >>  	unsigned long fp;
> >>  	unsigned long pc;
> >> +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME
> >> +	unsigned long sp;
> >> +#endif
> > 
> > As this is only used by the kernel unwinder (and not the hyp unwinder),
> > this should live in struct kunwind_state in stacktrace.c.
> > 
> > That said, for unwinding across exception boundaries we should not need
> > this, as the SP value will be in the pt_regs. If we only use SFrame for
> > the exception boundary case, we can remove this entirely. I would
> > strongly prefer that we do that.

> >> +	/* Get the Canonical Frame Address (CFA) */
> >> +	switch (frame.cfa.rule) {
> >> +	case UNWIND_CFA_RULE_SP_OFFSET:
> >> +		cfa = state->common.sp;
> 
> IIUC you suggest this to be changed as follows?
> 
> 		cfa = regs->regs[31];

I was suggesting:

		cfa = regs->sp;

Note: arm64's struct pt_regs has:

	union {
		struct user_pt_regs user_regs;
		struct {
			u64 regs[31];
			u64 sp;
			u64 pc;
			u64 pstate;
		};
	};	

... so regs->regs[31] would be an out-of-bounds array access.

[...]

> >> +	case UNWIND_CFA_RULE_REG_OFFSET:
> >> +	case UNWIND_CFA_RULE_REG_OFFSET_DEREF:
> >> +		/* regs only available in topmost/interrupt frame */
> >> +		if (!regs || frame.cfa.regnum > 30)
> >> +			return -EINVAL;
> >> +		cfa = regs->regs[frame.cfa.regnum];
> >> +		break;
> > 
> > Do we ever expect to see UNWIND_CFA_RULE_REG_OFFSET or
> > UNWIND_CFA_RULE_REG_OFFSET_DEREF in practice for kernel code?
> 
> No.  Those can only occur with SFrame V3 flexible FDE, which are
> currently not generated by GNU assembler for arm64/aarch64, and thus
> could be omitted in the arm64-specific kernel sframe unwinder:
> 
> https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=gas/config/tc-aarch64.h;hb=binutils-2_46#l342

Ok.

Do we know whether there are currently cases on aarch64 that cannot be
encoded in SFrame (without flexible FDE), or whether SFrame without
flexible FDE is sufficient for arm64 as-is? ... or do we have
counter-examples today?

Looking at:

  https://sourceware.org/binutils/docs/sframe-spec.html#Flexible-FDE-Type-Interpretation-1

For arm64 I'm not sure whether we'd encounter the DRAP or stack
realignment cases within the kernel (perhaps with SVE?), nor whether the
Register-based RA/FP Locations cases would apply if we assume that we
continue to use frame records.

[... ]

> I must admit that while reviewing I thought it would be future-proof to
> have support for rules that can only be represented with SFrame V3
> flexible FDE, even if they are currently not used on arm64.  Ideally
> kunwind_next_sframe() could be made common, if another architecture
> would implement kernel unwinding using sframe.

While I understand that principle, I think that for now it would be
better to keep this arch-specific and minimal:

* We have arch-specific concerns (e.g. the FRAME_META_TYPE_FINAL
  frames), and factoring that into generic code is going to be painful
  to adapt (which we're likely to need to do in the near future), and to
  maintain going forwards. Keeping that arch-specific for now will make
  it easier/quicker to get to a stable state.

* Code which isn't used is liable to be wrong or made wrong by accident.
  For example, with all the SP cases I mentioned in my initial reply.

We can certainly look at making that more generic in future, but for now
I'd prefer to omit the code that cannot be used (and have some sort of
build or boot/module-load time check that SFrame only has elements that
we expect), and make sure that we thoroughly test the cases that do
exist in practice.

Do we expect SFrame V3 flexible FDE to be generated by toolchains in the
near future?

[...]

> >> +	/* CFA alignment 8 bytes */
> >> +	if (cfa & 0x7)
> >> +		return -EINVAL;
> > 
> > If the CFA is the SP upon entry to the function, then per AAPCS64 rules
> > it should be aligned to 16 bytes. Otherwise, where has this 8 byte
> > alignment requirement come from? Does SFrame mandate that?
> 
> That originates from the common unwind user logic (see
> kernel/unwind/user.c, unwind_user_next_common()), which currently
> assumes 8-byte/4-byte SP alignment for all 64-bit/32-bit architectures.
> 
> So checking for 16-byte alignment here would make sense.

Just to confirm, am I correct to understand that the SFrame definition
of CFA is intended to be the same as the DWARF definition of CFA, and so
for arm64 the CFA is the SP when the function is called?

That's the case for DWARF on arm64:

  https://github.com/ARM-software/abi-aa/releases/download/2025Q4/aadwarf64.pdf
  https://github.com/ARM-software/abi-aa/blob/daa7a94ca55973736c0e434a67a6e4bbcd35d7fa/aadwarf64/aadwarf64.rst

| The CFA is the value of the stack pointer (sp) at the call site in the
| previous frame.

I couldn't find an explciit statement to that effect in:

  https://sourceware.org/binutils/docs/sframe-spec.html

... but I guess that is implied, given the other bits inherited from
DWARF.

I see that the documented behaviour for CFA on AMD64 and s390x are
consistent with their DWARF behaviour.

> >> +
> >> +	/* Get the Return Address (RA) */
> >> +	switch (frame.ra.rule) {
> >> +	case UNWIND_RULE_RETAIN:
> >> +		/* regs only available in topmost/interrupt frame */
> >> +		if (!regs)
> >> +			return -EINVAL;
> >> +		ra = regs->regs[30];
> >> +		source = KUNWIND_SOURCE_REGS_LR;
> >> +		break;
> >> +	/* UNWIND_USER_RULE_CFA_OFFSET not implemented on purpose */
> 
> Nit: s/UNWIND_USER_RULE_CFA_OFFSET/UNWIND_RULE_CFA_OFFSET/
> 
> > 
> > It would be better for the comment to say *why* that's not implemented.
> > 
> > I assume that's because UNWIND_USER_RULE_CFA_OFFSET would mean that the return
> > address is a stack address, and that's obviously not legitimate.
> 
> That and SFrame V3 currently cannot represent FP/RA as CFA + offset
> (i.e. UNWIND_RULE_CFA_OFFSET; .cfi_val_offset FP/RA).
> 
> The comment originates from the common unwind user logic (see
> kernel/unwind/user.c).  I am open to improve that.  What about:
> 
> 	/*
> 	 * UNWIND_RULE_CFA_OFFSET not implemented on purpose, as a stack
> 	 * address cannot be a legitimate return address value.  It is
> 	 * also not used (e.g. not represented in sframe).
> 	 */

I'd go with something simpler, e.g.

	/*
	 * UNWIND_RULE_CFA_OFFSET doesn't make sense for RA.
	 * The return address cannot legitimately be a stack addres.
	 */

[...]

> > 
> > I don't think we expect UNWIND_RULE_REG_OFFSET unless that's sometimes used
> > instead of UNWIND_RULE_RETAIN to express that the return address is in x30
> > (with zero offset).
> 
> No.  Unless there would be nonsense .cfi_register 30, 30, which would
> require SFrame V3 flexible FDE to be represented.

Ok.

> @Indu:  We may consider to treat .cfi_register <reg>, <reg> (for FP/RA)
> like .cfi_restore <reg> in the GNU assembler?
> 
> > Similarly, if the address is on the stack it should be in a frame
> > record. Would we ever expect UNWIND_RULE_REG_OFFSET_DEREF rather than
> > UNWIND_RULE_CFA_OFFSET_DEREF?
> 
> No.  See above (SFrame V3 flexible FDE).

Ok.

> >> +	default:
> >> +		WARN_ON_ONCE(1);
> >> +		return -EINVAL;
> >> +	}
> >> +
> >> +	/* Get the Frame Pointer (FP) */
> >> +	switch (frame.fp.rule) {
> >> +	case UNWIND_RULE_RETAIN:
> >> +		fp = state->common.fp;
> >> +		break;
> >> +	/* UNWIND_USER_RULE_CFA_OFFSET not implemented on purpose */
> > 
> > As for RA, the comment should explain why that's not implemented.
> 
> I am open to improve the comment in the the common unwind user logic.
> What about:
> 
> 	/*
> 	 * UNWIND_RULE_CFA_OFFSET not implemented on purpose, as it is
> 	 * not used (e.g. not represented in sframe).
> 	 */

For me, this wording raises more questions, e.g.

* Does 'not used' mean that toolchains don't use that, or that the spec
  doesn't permit that?

* Does 'not represented' mean that this is not represntable, or that
  toolchains currently don't generate SFrame with the appropriate
  elements.

IIUC you're saying that this *is* representable with flexible FDE, but
current toolchains don't generate that.

Mark.

Re: [PATCH v5 8/8] unwind: arm64: Use sframe to unwind interrupt frames

[Jens Remus]

On 5/5/2026 12:29 PM, Mark Rutland wrote:
> On Mon, May 04, 2026 at 10:47:26AM +0200, Jens Remus wrote:
>> On 5/1/2026 6:46 PM, Mark Rutland wrote:

>>> Thanks for putting this together. I think this is looking pretty good.
>>> However, there are some things that aren't quite right and need some
>>> work, which I've commented on below.
>>
>>> (2) To make unwinding generally possible, we'll need to annotate some
>>>     assembly functions as unwindable. We'll need to do that for string
>>>     routines under lib/, and probably some crypto code, but we don't
>>>     need to do that for most code in head.S, entry.S, etc.
>>>
>>>     The vast majority of relevant assembly functions are leaf functions
>>>     (where the return address is never moved out of the LR), so we can
>>>     probably get away with a simple annotation for those that avoids the
>>>     need for open-coded CFI directives everywhere.
>>
>> Wrapping them in .cfi_startproc ... .cfi_endproc should do.  For instance
>> by extending SYM_FUNC_START() and SYM_FUNC_END() or introducing flavors
>> that do.  Or where you thinking of something else?
> 
> I was expecting we'd do something like that, either with distinct
> versions, or some entirely separate annotation.
> 
> We can't override SYM_FUNC_START() or SYM_FUNC_END() since those are
> also used for non-leaf functions. The bulk of the work is going to be
> making sure we only annotate leaf functions specifically, which will
> require some human analysis.

Makes sense.

>>> On Tue, Apr 28, 2026 at 06:36:43PM +0000, Dylan Hatch wrote:
>>>> diff --git a/arch/arm64/include/asm/stacktrace/common.h b/arch/arm64/include/asm/stacktrace/common.h
>>
>>>> @@ -21,6 +21,8 @@ struct stack_info {
>>>>   *
>>>>   * @fp:          The fp value in the frame record (or the real fp)
>>>>   * @pc:          The lr value in the frame record (or the real lr)
>>>> + * @sp:          The sp value at the call site of the current function.
>>>> + * @unreliable:  Stacktrace is unreliable.
>>>>   *
>>>>   * @stack:       The stack currently being unwound.
>>>>   * @stacks:      An array of stacks which can be unwound.
>>>> @@ -29,7 +31,11 @@ struct stack_info {
>>>>  struct unwind_state {
>>>>  	unsigned long fp;
>>>>  	unsigned long pc;
>>>> +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME
>>>> +	unsigned long sp;
>>>> +#endif
>>>
>>> As this is only used by the kernel unwinder (and not the hyp unwinder),
>>> this should live in struct kunwind_state in stacktrace.c.
>>>
>>> That said, for unwinding across exception boundaries we should not need
>>> this, as the SP value will be in the pt_regs. If we only use SFrame for
>>> the exception boundary case, we can remove this entirely. I would
>>> strongly prefer that we do that.
> 
>>>> +	/* Get the Canonical Frame Address (CFA) */
>>>> +	switch (frame.cfa.rule) {
>>>> +	case UNWIND_CFA_RULE_SP_OFFSET:
>>>> +		cfa = state->common.sp;
>>
>> IIUC you suggest this to be changed as follows?
>>
>> 		cfa = regs->regs[31];
> 
> I was suggesting:
> 
> 		cfa = regs->sp;
> 
> Note: arm64's struct pt_regs has:
> 
> 	union {
> 		struct user_pt_regs user_regs;
> 		struct {
> 			u64 regs[31];
> 			u64 sp;
> 			u64 pc;
> 			u64 pstate;
> 		};
> 	};	
> 
> ... so regs->regs[31] would be an out-of-bounds array access.

Aww, my bad!  Of course!

> 
> [...]
> 
>>>> +	case UNWIND_CFA_RULE_REG_OFFSET:
>>>> +	case UNWIND_CFA_RULE_REG_OFFSET_DEREF:
>>>> +		/* regs only available in topmost/interrupt frame */
>>>> +		if (!regs || frame.cfa.regnum > 30)
>>>> +			return -EINVAL;
>>>> +		cfa = regs->regs[frame.cfa.regnum];
>>>> +		break;
>>>
>>> Do we ever expect to see UNWIND_CFA_RULE_REG_OFFSET or
>>> UNWIND_CFA_RULE_REG_OFFSET_DEREF in practice for kernel code?
>>
>> No.  Those can only occur with SFrame V3 flexible FDE, which are
>> currently not generated by GNU assembler for arm64/aarch64, and thus
>> could be omitted in the arm64-specific kernel sframe unwinder:
>>
>> https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=gas/config/tc-aarch64.h;hb=binutils-2_46#l342
> 
> Ok.
> 
> Do we know whether there are currently cases on aarch64 that cannot be
> encoded in SFrame (without flexible FDE), or whether SFrame without
> flexible FDE is sufficient for arm64 as-is? ... or do we have
> counter-examples today?

Not that I am aware of.  IIUC this is why Indu, the SFrame maintainer,
did not enable SFrame V3 flexible FDE for arm64/aarch64 in the GNU
assembler.

> Looking at:
> 
>   https://sourceware.org/binutils/docs/sframe-spec.html#Flexible-FDE-Type-Interpretation-1
> 
> For arm64 I'm not sure whether we'd encounter the DRAP or stack
> realignment cases within the kernel (perhaps with SVE?), nor whether the
> Register-based RA/FP Locations cases would apply if we assume that we
> continue to use frame records.

@Indu:  Can you provide more insight?

> [... ]
> 
>> I must admit that while reviewing I thought it would be future-proof to
>> have support for rules that can only be represented with SFrame V3
>> flexible FDE, even if they are currently not used on arm64.  Ideally
>> kunwind_next_sframe() could be made common, if another architecture
>> would implement kernel unwinding using sframe.
> 
> While I understand that principle, I think that for now it would be
> better to keep this arch-specific and minimal:
> 
> * We have arch-specific concerns (e.g. the FRAME_META_TYPE_FINAL
>   frames), and factoring that into generic code is going to be painful
>   to adapt (which we're likely to need to do in the near future), and to
>   maintain going forwards. Keeping that arch-specific for now will make
>   it easier/quicker to get to a stable state.
> 
> * Code which isn't used is liable to be wrong or made wrong by accident.
>   For example, with all the SP cases I mentioned in my initial reply.
> 
> We can certainly look at making that more generic in future, but for now
> I'd prefer to omit the code that cannot be used (and have some sort of
> build or boot/module-load time check that SFrame only has elements that
> we expect), and make sure that we thoroughly test the cases that do
> exist in practice.

Makes sense.

> Do we expect SFrame V3 flexible FDE to be generated by toolchains in the
> near future?

There is work in progress to implement SFrame V3 generation in LLVM.
Their implementation could be less restrictive regarding to which
SFrame V3 feature to enable on arm64/aarch64.  But even then for
SFrame V3 flexible FDE to be generated there must be DWARF CFI patterns
that can only be represented using those.  So if those do not exist
(see your previous question above), then they should not be generated.

@Indu:  What are your thoughts on this?

> [...]
> 
>>>> +	/* CFA alignment 8 bytes */
>>>> +	if (cfa & 0x7)
>>>> +		return -EINVAL;
>>>
>>> If the CFA is the SP upon entry to the function, then per AAPCS64 rules
>>> it should be aligned to 16 bytes. Otherwise, where has this 8 byte
>>> alignment requirement come from? Does SFrame mandate that?
>>
>> That originates from the common unwind user logic (see
>> kernel/unwind/user.c, unwind_user_next_common()), which currently
>> assumes 8-byte/4-byte SP alignment for all 64-bit/32-bit architectures.
>>
>> So checking for 16-byte alignment here would make sense.
> 
> Just to confirm, am I correct to understand that the SFrame definition
> of CFA is intended to be the same as the DWARF definition of CFA, and so

Correct.

> for arm64 the CFA is the SP when the function is called?

Correct.

> 
> That's the case for DWARF on arm64:
> 
>   https://github.com/ARM-software/abi-aa/releases/download/2025Q4/aadwarf64.pdf
>   https://github.com/ARM-software/abi-aa/blob/daa7a94ca55973736c0e434a67a6e4bbcd35d7fa/aadwarf64/aadwarf64.rst
> 
> | The CFA is the value of the stack pointer (sp) at the call site in the
> | previous frame.
> 
> I couldn't find an explciit statement to that effect in:
> 
>   https://sourceware.org/binutils/docs/sframe-spec.html
> 
> ... but I guess that is implied, given the other bits inherited from
> DWARF.

I assume so.

> 
> I see that the documented behaviour for CFA on AMD64 and s390x are
> consistent with their DWARF behaviour.

Yes.

> 
>>>> +
>>>> +	/* Get the Return Address (RA) */
>>>> +	switch (frame.ra.rule) {
>>>> +	case UNWIND_RULE_RETAIN:
>>>> +		/* regs only available in topmost/interrupt frame */
>>>> +		if (!regs)
>>>> +			return -EINVAL;
>>>> +		ra = regs->regs[30];
>>>> +		source = KUNWIND_SOURCE_REGS_LR;
>>>> +		break;
>>>> +	/* UNWIND_USER_RULE_CFA_OFFSET not implemented on purpose */
>>
>> Nit: s/UNWIND_USER_RULE_CFA_OFFSET/UNWIND_RULE_CFA_OFFSET/
>>
>>>
>>> It would be better for the comment to say *why* that's not implemented.
>>>
>>> I assume that's because UNWIND_USER_RULE_CFA_OFFSET would mean that the return
>>> address is a stack address, and that's obviously not legitimate.
>>
>> That and SFrame V3 currently cannot represent FP/RA as CFA + offset
>> (i.e. UNWIND_RULE_CFA_OFFSET; .cfi_val_offset FP/RA).
>>
>> The comment originates from the common unwind user logic (see
>> kernel/unwind/user.c).  I am open to improve that.  What about:
>>
>> 	/*
>> 	 * UNWIND_RULE_CFA_OFFSET not implemented on purpose, as a stack
>> 	 * address cannot be a legitimate return address value.  It is
>> 	 * also not used (e.g. not represented in sframe).
>> 	 */
> 
> I'd go with something simpler, e.g.
> 
> 	/*
> 	 * UNWIND_RULE_CFA_OFFSET doesn't make sense for RA.
> 	 * The return address cannot legitimately be a stack addres.
> 	 */

Thanks!  I have updated the comments accordingly in my latest unwind
user sframe patch series sent today:

[PATCH v14 14/19] unwind_user: Flexible FP/RA recovery rules
https://lore.kernel.org/all/20260505121718.3572346-15-jremus@linux.ibm.com/

[...]

>>>> +	default:
>>>> +		WARN_ON_ONCE(1);
>>>> +		return -EINVAL;
>>>> +	}
>>>> +
>>>> +	/* Get the Frame Pointer (FP) */
>>>> +	switch (frame.fp.rule) {
>>>> +	case UNWIND_RULE_RETAIN:
>>>> +		fp = state->common.fp;
>>>> +		break;
>>>> +	/* UNWIND_USER_RULE_CFA_OFFSET not implemented on purpose */
>>>
>>> As for RA, the comment should explain why that's not implemented.
>>
>> I am open to improve the comment in the the common unwind user logic.
>> What about:
>>
>> 	/*
>> 	 * UNWIND_RULE_CFA_OFFSET not implemented on purpose, as it is
>> 	 * not used (e.g. not represented in sframe).
>> 	 */
> 
> For me, this wording raises more questions, e.g.
> 
> * Does 'not used' mean that toolchains don't use that, or that the spec
>   doesn't permit that?

unwind user currently only supports frame pointer, with SFrame to be
hopefully added soon.  Out of these only SFrame requires ("uses") these
elaborated rules.

> 
> * Does 'not represented' mean that this is not represntable, or that
>   toolchains currently don't generate SFrame with the appropriate
>   elements.

In DWARF CFI it is representable using .cfi_val_offset <FP>, <offset>.
But SFrame V3 currently cannot represent this:

"Note that, using a value of 0 as padding data word, does mean that
currently, e.g., for RA [JR: FP likewise], the rule RA = CFA + 0 cannot
be encoded.  NB: RA = CFA + 0 is distinct from RA = *(CFA + 0)."

https://sourceware.org/binutils/docs/sframe-spec.html#Flexible-FDE-Type-Interpretation-1

> 
> IIUC you're saying that this *is* representable with flexible FDE, but
> current toolchains don't generate that.

Thanks for the feedback!  I changed it as follows to clarify:

	/*
	 * UNWIND_USER_RULE_CFA_OFFSET is currently not used for FP
	 * (e.g. SFrame cannot represent this rule).
	 */

Regards,
Jens
-- 
Jens Remus
Linux on Z Development (D3303)
jremus@de.ibm.com / jremus@linux.ibm.com

IBM Deutschland Research & Development GmbH; Vorsitzender des Aufsichtsrats: Wolfgang Wendt; Geschäftsführung: David Faller; Sitz der Gesellschaft: Ehningen; Registergericht: Amtsgericht Stuttgart, HRB 243294
IBM Data Privacy Statement: https://www.ibm.com/privacy/

Re: [PATCH v5 0/8] unwind, arm64: add sframe unwinder for kernel

[Dylan Hatch]

On Thu, Apr 30, 2026 at 3:11 AM Jens Remus <jremus@linux.ibm.com> wrote:
>
> On 4/28/2026 8:36 PM, Dylan Hatch wrote:
> > Implement a generic kernel sframe-based [1] unwinder. The main goal is
> > to improve reliable stacktrace on arm64 by unwinding across exception
> > boundaries.
>
> Please add support to initialize the optional sframe unwinder debug
> information.  Either in the appropriate patches in this series or as a
> separate patch.

Sounds good, I'll add this in as a separate patch in the next version.

>
> Note that for the module case I wonder whether it would be preferable
> to somehow indicate that it is a module name in the string, e.g.
> "(<module-name>)" or "<module-name> (module)"?

I don't have a strong preference, though I agree it makes sense to
indicate that the section is from a module. For now I'll add the
parentheses "(<module-name>)".

>
> diff --git a/kernel/unwind/sframe.c b/kernel/unwind/sframe.c
> --- a/kernel/unwind/sframe.c
> +++ b/kernel/unwind/sframe.c
> @@ -1028,6 +1028,8 @@ void __init init_sframe_table(void)
>         kernel_sfsec.text_start         = (unsigned long)_stext;
>         kernel_sfsec.text_end           = (unsigned long)_etext;
>
> +       dbg_init(&kernel_sfsec);
> +
>         if (WARN_ON(sframe_read_header(&kernel_sfsec)))
>                 return;
>         if (WARN_ON(sframe_validate_section(&kernel_sfsec)))
> @@ -1047,6 +1049,8 @@ void sframe_module_init(struct module *mod, void *sframe, size_t sframe_size,
>         sec->text_start   = (unsigned long)text;
>         sec->text_end     = (unsigned long)text + text_size;
>
> +       dbg_init(sec);
> +
>         if (WARN_ON(sframe_read_header(sec)))
>                 return;
>         if (WARN_ON(sframe_validate_section(sec)))
> diff --git a/kernel/unwind/sframe_debug.h b/kernel/unwind/sframe_debug.h
> --- a/kernel/unwind/sframe_debug.h
> +++ b/kernel/unwind/sframe_debug.h
> @@ -32,6 +32,18 @@ static inline void dbg_init(struct sframe_section *sec)
>         struct mm_struct *mm = current->mm;
>         struct vm_area_struct *vma;
>
> +       if (sec->sec_type == SFRAME_KERNEL) {
> +               if (sec == &kernel_sfsec) {
> +                       sec->filename = kstrdup("(vmlinux)", GFP_KERNEL);
> +               } else {
> +                       struct module *mod = container_of(sec, struct module,
> +                                                         arch.sframe_sec);
> +                       sec->filename = kstrdup(mod->name, GFP_KERNEL);
> +               }
> +
> +               return;
> +       }
> +
>         guard(mmap_read_lock)(mm);
>         vma = vma_lookup(mm, sec->sframe_start);
>         if (!vma)
>
> Regards,
> Jens
> --
> Jens Remus
> Linux on Z Development (D3303)
> jremus@de.ibm.com / jremus@linux.ibm.com
>
> IBM Deutschland Research & Development GmbH; Vorsitzender des Aufsichtsrats: Wolfgang Wendt; Geschäftsführung: David Faller; Sitz der Gesellschaft: Ehningen; Registergericht: Amtsgericht Stuttgart, HRB 243294
> IBM Data Privacy Statement: https://www.ibm.com/privacy/
>

Thanks for the suggestion,
Dylan

Re: [PATCH v5 8/8] unwind: arm64: Use sframe to unwind interrupt frames

[Dylan Hatch]

Hi Mark,

Thanks for all the feedback and help on this. I'm planning on getting
your comments addressed in the coming days, but I have some initial
clarifying questions.

On Fri, May 1, 2026 at 9:46 AM Mark Rutland <mark.rutland@arm.com> wrote:
>
> Hi Dylan,
>
> Thanks for putting this together. I think this is looking pretty good.
> However, there are some things that aren't quite right and need some
> work, which I've commented on below.
>
> More generally, there are a few things that aren't addressed by this
> series that we will also need to address. Importantly:
>
> (1) For correctness, we'll need to address a latent issue with unwinding
>     across an fgraph return trampoline, where the return address is
>     transiently unrecoverable.
>
>     Before this series, that doesn't matter for livepatching because the
>     livepatching code isn't called synchronously within the fgraph
>     handler, and unwinds which cross an exception boundary are marked as
>     unreliable.
>
>     After this series, that does matter as we can unwind across an
>     exception boundary, and might happen to interrupt that transient
>     window.
>
>     I think we can solve that with some restructuring of that code,
>     restoring the original address *before* removing that from the
>     fgraph return stack, and ensuring that the unwinder can find it.

If my understanding is correct, the issue arrises in return_to_handler
as the return address is recovered:

mov x0, sp
bl ftrace_return_to_handler // addr = ftrace_return_to_hander(fregs);
mov x30, x0 // restore the original return address

Because ftrace_return_to_handler pops the return address from the
return stack before it can be restored into the LR, it cannot be
recovered.

Based on this, I believe you are suggesting to restructure this code
path such that the return address is removed from the return stack
only after it has been restored to LR. But since kernel/trace/fgraph.c
is core kernel code, will this end up requiring either (1) a similar
restructuring of other arches supporting ftrace, or (2) an
arm64-specific implementation of this recovery logic?

It looks to me like there is essentially the same recovery pattern on
other arches; is there a reason this transient unrecoverability isn't
an issue for reliable unwind on other platforms?

>
>     I'm not immediately sure whether kretprobes has a similar issue.
>
> (2) To make unwinding generally possible, we'll need to annotate some
>     assembly functions as unwindable. We'll need to do that for string
>     routines under lib/, and probably some crypto code, but we don't
>     need to do that for most code in head.S, entry.S, etc.
>
>     The vast majority of relevant assembly functions are leaf functions
>     (where the return address is never moved out of the LR), so we can
>     probably get away with a simple annotation for those that avoids the
>     need for open-coded CFI directives everywhere.

Are you suggesting something like a SYM_LEAF_FUNC_(START|END), that
wraps CFI directives for leaf functions?

>
> I've pushed some reliable stacktrace tests to:
>
>   git://git.kernel.org/pub/scm/linux/kernel/git/mark/linux.git stacktrace/tests
>
> That finds the fgraph issue (regardless of this series). When merged
> with this series triggers a warning in kunwind_next_frame_record_meta(),
> where unwind_next_frame_sframe() calls that erroneously as a fallback.

Thanks for the pointer on these tests, they're super useful! I've been
able to reproduce the fgraph failure you mentioned.

Thanks,
Dylan

Re: [PATCH v5 8/8] unwind: arm64: Use sframe to unwind interrupt frames

[Jens Remus]

Hello Dylan and Mark!

On 5/12/2026 5:00 AM, Dylan Hatch wrote:
> On Fri, May 1, 2026 at 9:46 AM Mark Rutland <mark.rutland@arm.com>
> wrote:

>> More generally, there are a few things that aren't addressed by
>> this series that we will also need to address. Importantly:
>> 
>> (1) For correctness, we'll need to address a latent issue with
>> unwinding across an fgraph return trampoline, where the return
>> address is transiently unrecoverable.
>> 
>> Before this series, that doesn't matter for livepatching because
>> the livepatching code isn't called synchronously within the fgraph 
>> handler, and unwinds which cross an exception boundary are marked
>> as unreliable.
>> 
>> After this series, that does matter as we can unwind across an 
>> exception boundary, and might happen to interrupt that transient 
>> window.
>> 
>> I think we can solve that with some restructuring of that code, 
>> restoring the original address *before* removing that from the 
>> fgraph return stack, and ensuring that the unwinder can find it.
> 
> If my understanding is correct, the issue arrises in
> return_to_handler as the return address is recovered:
> 
> mov x0, sp bl ftrace_return_to_handler // addr =
> ftrace_return_to_hander(fregs); mov x30, x0 // restore the original
> return address
> 
> Because ftrace_return_to_handler pops the return address from the 
> return stack before it can be restored into the LR, it cannot be 
> recovered.

Based on reliable-stacktrace.rst section "4.4 Rewriting of return
addresses" I wonder whether the following might work:

- If an unwound RA points at return_to_handler the actual RA needs to
  be obtained using ftrace_graph_ret_addr().  This might already be
  taken into account if ftrace_graph_ret_addr() is used unconditionally.

- If an unwound RA points into return_to_handler() mark the stack trace
  as unreliable.  This could be accomplished by marking LR in
  return_to_handler() as undefined (i.e. .cfi_undefined 30) to use
  SFrame's outermost frame indication to stop and mark the stack trace
  as unreliable:

diff --git a/arch/arm64/kernel/entry-ftrace.S b/arch/arm64/kernel/entry-ftrace.S
@@ -329,8 +329,12 @@ SYM_FUNC_END(ftrace_stub_graph)
  * @fp is checked against the value passed by ftrace_graph_caller().
  */
 SYM_CODE_START(return_to_handler)
+       .cfi_startproc
+       /* Mark unwinding of LR as unreliable */
+       .cfi_undefined 30
        /* Make room for ftrace_regs */
        sub     sp, sp, #FREGS_SIZE
+       .cfi_adjust_cfa_offset -FREGS_SIZE

        /* Save return value regs */
        stp     x0, x1, [sp, #FREGS_X0]
@@ -344,6 +348,8 @@ SYM_CODE_START(return_to_handler)
        mov     x0, sp
        bl      ftrace_return_to_handler        // addr = ftrace_return_to_hander(fregs);
        mov     x30, x0                         // restore the original return address
+       /* Mark unwinding of LR as reliable */
+       .cfi_restore 30

        /* Restore return value regs */
        ldp     x0, x1, [sp, #FREGS_X0]
@@ -351,7 +357,9 @@ SYM_CODE_START(return_to_handler)
        ldp     x4, x5, [sp, #FREGS_X4]
        ldp     x6, x7, [sp, #FREGS_X6]
        add     sp, sp, #FREGS_SIZE
+       .cfi_adjust_cfa_offset FREGS_SIZE

        ret
+       .cfi_endproc
 SYM_CODE_END(return_to_handler)
 #endif /* CONFIG_FUNCTION_GRAPH_TRACER */

Notes regarding above:
- return_to_handler() saves the caller's FP in ftrace_regs but never
  restores it.  I suppose this is because ftrace_regs is an input to
  ftrace_return_to_handler().  The DWARF CFI above assumes SP and FP
  can be restored at all times as SP=CFA and FP=FP.
- One might be tempted to add a .cfi_register 30, 0 after the call to
  ftrace_return_to_handler.  This would be wrong, because if unwinding
  comes from ftrace_return_to_handler() the unwound RA will point there
  and the unwinding logic would erroneously assume x0 to contain the RA.
- The DWARF CFI could be simplified as follows to just convey that
  unwinding through return_to_handler is impossible at all times:

diff --git a/arch/arm64/kernel/entry-ftrace.S b/arch/arm64/kernel/entry-ftrace.S
@@ -329,6 +329,9 @@ SYM_FUNC_END(ftrace_stub_graph)
  * @fp is checked against the value passed by ftrace_graph_caller().
  */
 SYM_CODE_START(return_to_handler)
+       .cfi_startproc simple
+       /* Mark unwinding of LR as unreliable */
+       .cfi_undefined 30
        /* Make room for ftrace_regs */
        sub     sp, sp, #FREGS_SIZE

@@ -353,5 +356,6 @@ SYM_CODE_START(return_to_handler)
        add     sp, sp, #FREGS_SIZE

        ret
+       .cfi_endproc
 SYM_CODE_END(return_to_handler)
 #endif /* CONFIG_FUNCTION_GRAPH_TRACER */

> 
> Based on this, I believe you are suggesting to restructure this code 
> path such that the return address is removed from the return stack 
> only after it has been restored to LR. But since kernel/trace/
> fgraph.c is core kernel code, will this end up requiring either (1)
> a similar restructuring of other arches supporting ftrace, or (2) an 
> arm64-specific implementation of this recovery logic?
> 
> It looks to me like there is essentially the same recovery pattern
> on other arches; is there a reason this transient unrecoverability
> isn't an issue for reliable unwind on other platforms?
> 
>> 
>> I'm not immediately sure whether kretprobes has a similar issue.

Regards,
Jens
-- 
Jens Remus
Linux on Z Development (D3303)
jremus@de.ibm.com / jremus@linux.ibm.com

IBM Deutschland Research & Development GmbH; Vorsitzender des Aufsichtsrats: Wolfgang Wendt; Geschäftsführung: David Faller; Sitz der Gesellschaft: Ehningen; Registergericht: Amtsgericht Stuttgart, HRB 243294
IBM Data Privacy Statement: https://www.ibm.com/privacy/

Re: [PATCH v5 8/8] unwind: arm64: Use sframe to unwind interrupt frames

[Mark Rutland]

On Mon, May 11, 2026 at 08:00:21PM -0700, Dylan Hatch wrote:
> Hi Mark,
> 
> Thanks for all the feedback and help on this. I'm planning on getting
> your comments addressed in the coming days, but I have some initial
> clarifying questions.
> 
> On Fri, May 1, 2026 at 9:46 AM Mark Rutland <mark.rutland@arm.com> wrote:
> >
> > Hi Dylan,
> >
> > Thanks for putting this together. I think this is looking pretty good.
> > However, there are some things that aren't quite right and need some
> > work, which I've commented on below.
> >
> > More generally, there are a few things that aren't addressed by this
> > series that we will also need to address. Importantly:
> >
> > (1) For correctness, we'll need to address a latent issue with unwinding
> >     across an fgraph return trampoline, where the return address is
> >     transiently unrecoverable.
> >
> >     Before this series, that doesn't matter for livepatching because the
> >     livepatching code isn't called synchronously within the fgraph
> >     handler, and unwinds which cross an exception boundary are marked as
> >     unreliable.
> >
> >     After this series, that does matter as we can unwind across an
> >     exception boundary, and might happen to interrupt that transient
> >     window.
> >
> >     I think we can solve that with some restructuring of that code,
> >     restoring the original address *before* removing that from the
> >     fgraph return stack, and ensuring that the unwinder can find it.
> 
> If my understanding is correct, the issue arrises in return_to_handler
> as the return address is recovered:
> 
> mov x0, sp
> bl ftrace_return_to_handler // addr = ftrace_return_to_hander(fregs);
> mov x30, x0 // restore the original return address
> 
> Because ftrace_return_to_handler pops the return address from the
> return stack before it can be restored into the LR, it cannot be
> recovered.

Yes.

To be clear, please don't worry about solving that for the next version
of this series; let's get the SFrame bits into shape first. I just
wanted to highlight that there's some more general work that we'll need
to do.

I think we can *detect* this case (and mark the unwind as unreliable)
with some tiny changes to the arm64 code. I'm happy to put that
together.

> Based on this, I believe you are suggesting to restructure this code
> path such that the return address is removed from the return stack
> only after it has been restored to LR. But since kernel/trace/fgraph.c
> is core kernel code, will this end up requiring either (1) a similar
> restructuring of other arches supporting ftrace, or (2) an
> arm64-specific implementation of this recovery logic?

Yes, I am say that to *recover* the address we'd need to make changes to
core code.

In the mean time we can *detect* this case with some minimal changes to
arm64 code, and abort. As above, I'm happy to go put that together.

> It looks to me like there is essentially the same recovery pattern on
> other arches; is there a reason this transient unrecoverability isn't
> an issue for reliable unwind on other platforms?

Yep; on all architectures there's a transient period where the address
cannot be recovered. It's not a correctness issue so long as the
architecture detects this case and marks the unwind as unreliable.

IIUC x86 will mark the unwind as unreliable in this case.

I don't know whether other architectures detect this reliably. That's a
question for loonarch, parisc, powerpc, and s390 folk.

> >     I'm not immediately sure whether kretprobes has a similar issue.
> >
> > (2) To make unwinding generally possible, we'll need to annotate some
> >     assembly functions as unwindable. We'll need to do that for string
> >     routines under lib/, and probably some crypto code, but we don't
> >     need to do that for most code in head.S, entry.S, etc.
> >
> >     The vast majority of relevant assembly functions are leaf functions
> >     (where the return address is never moved out of the LR), so we can
> >     probably get away with a simple annotation for those that avoids the
> >     need for open-coded CFI directives everywhere.
> 
> Are you suggesting something like a SYM_LEAF_FUNC_(START|END), that
> wraps CFI directives for leaf functions?

Yep; that's exactly the sort of thing I was thinking of.

That or have a seaprate annotation we can add, e.g.

SYM_FUNC_START(foo)
SYM_FUN_END(foo)
SYM_FUNC_IS_LEAF_AND_DOES_NOT_TOUCH_LR(foo)

> > I've pushed some reliable stacktrace tests to:
> >
> >   git://git.kernel.org/pub/scm/linux/kernel/git/mark/linux.git stacktrace/tests
> >
> > That finds the fgraph issue (regardless of this series). When merged
> > with this series triggers a warning in kunwind_next_frame_record_meta(),
> > where unwind_next_frame_sframe() calls that erroneously as a fallback.
> 
> Thanks for the pointer on these tests, they're super useful! I've been
> able to reproduce the fgraph failure you mentioned.

Great!

Mark.

Re: [PATCH v5 8/8] unwind: arm64: Use sframe to unwind interrupt frames

[Mark Rutland]

On Tue, May 12, 2026 at 10:55:28AM +0200, Jens Remus wrote:
> On 5/12/2026 5:00 AM, Dylan Hatch wrote:
> > On Fri, May 1, 2026 at 9:46 AM Mark Rutland <mark.rutland@arm.com>
> > wrote:
> 
> >> (1) For correctness, we'll need to address a latent issue with
> >> unwinding across an fgraph return trampoline, where the return
> >> address is transiently unrecoverable.

> >> I think we can solve that with some restructuring of that code, 
> >> restoring the original address *before* removing that from the 
> >> fgraph return stack, and ensuring that the unwinder can find it.
> > 
> > If my understanding is correct, the issue arrises in
> > return_to_handler as the return address is recovered:
> > 
> > mov x0, sp bl ftrace_return_to_handler // addr =
> > ftrace_return_to_hander(fregs); mov x30, x0 // restore the original
> > return address
> > 
> > Because ftrace_return_to_handler pops the return address from the 
> > return stack before it can be restored into the LR, it cannot be 
> > recovered.
> 
> Based on reliable-stacktrace.rst section "4.4 Rewriting of return
> addresses" I wonder whether the following might work:
> 
> - If an unwound RA points at return_to_handler the actual RA needs to
>   be obtained using ftrace_graph_ret_addr().  This might already be
>   taken into account if ftrace_graph_ret_addr() is used unconditionally.
> 
> - If an unwound RA points into return_to_handler() mark the stack trace
>   as unreliable.  This could be accomplished by marking LR in
>   return_to_handler() as undefined (i.e. .cfi_undefined 30) to use
>   SFrame's outermost frame indication to stop and mark the stack trace
>   as unreliable:

We don't currently have any CFI annotations for return_to_handler(), so
if we interrupt that, any unwind will naturally be marked as unreliable.

The problem is that we can try an unwind from an interrupted *callee* of
return_to_handler(). In that case, we'll unwind through
return_to_handler() using the frame pointer, without consulting SFrame.
In that case, the PC will be part-way through return_to_handler(), but
we only call ftrace_graph_ret_addr() when the PC is the start of
return_to_handler, and so we don't even try to recover the return
address.

We can handle that better by checking whether the PC is *within*
return_to_handler(), and aborting when the original return address
cannot be recoverted. I'm happy to go put that together, nad longer term
I would like to do the better reovery I described above such that we can
*always* recover the return address.

Mark.