Re: [PATCH 0/8] unwind, arm64: add sframe unwinder for kernel

Indu Bhagat
2025-02-25T19:38:55Z

On Mon, Feb 10, 2025 at 12:30 AM Weinan Liu <wnliu@google.com> wrote:
>> I already have a WIP patch to add sframe support to the kernel module.
>> However, it is not yet working. I had trouble unwinding frames for the
>> kernel module using the current algorithm.
>>
>> Indu has likely identified the issue and will be addressing it from the
>> toolchain side.
>>
>> https://sourceware.org/bugzilla/show_bug.cgi?id=32666
>
> I have a working in progress patch that adds sframe support for kernel
> module.
> https://github.com/heuza/linux/tree/sframe_unwinder.rfc
>
> According to the sframe table values I got during runtime testing, looks
> like the offsets are not correct .
>

I hope to sanitize the fix for 32666 and post upstream soon (I had to
address other related issues). Unless fixed, relocating .sframe
sections using the .rela.sframe is expected to generate incorrect output.

> When unwind symbols init_module(0xffff80007b155048) from the kernel
> module(livepatch-sample.ko), the start_address of the FDE entries in the
> sframe table of the kernel modules appear incorrect.

init_module will apply the relocations on the .sframe section, isnt it ?

> For instance, the first FDE's start_addr is reported as -20564. Adding
> this offset to the module's sframe section address (0xffff80007b15a040)
> yields 0xffff80007b154fec, which is not within the livepatch-sample.ko
> memory region(It should be larger than 0xffff80007b155000).
>

Hmm..something seems off here. Having tested a potential fix for 32666
locally, I do not expect the first FDE to show this symptom.

> Here are the sframe table values of the livepatch-samples.ko that I print
> by qemu + gdb.
>
> ```
> $ /usr/bin/aarch64-linux-gnu-objdump -L --sframe=.sframe ./samples/livepatch/livepatch-sample.ko
> ./samples/livepatch/livepatch-sample.ko: file format elf64-littleaarch64
>
> Contents of the SFrame section .sframe:
> Header :
>
> Version: SFRAME_VERSION_2
> Flags: SFRAME_F_FDE_SORTED
> Num FDEs: 3
> Num FREs: 11
>
> Function Index :
>
> func idx [0]: pc = 0x0, size = 12 bytes
> STARTPC CFA FP RA
> 0000000000000000 sp+0 u u
>
> func idx [1]: pc = 0x0, size = 44 bytes
> STARTPC CFA FP RA
> 0000000000000000 sp+0 u u
> 000000000000000c sp+0 u u[s]
> 0000000000000010 sp+16 c-16 c-8[s]
> 0000000000000024 sp+0 u u[s]
> 0000000000000028 sp+0 u u
>
> func idx [2]: pc = 0x0, size = 56 bytes
> STARTPC CFA FP RA
> 0000000000000000 sp+0 u u
> 000000000000000c sp+0 u u[s]
> 0000000000000010 sp+16 c-16 c-8[s]
> 0000000000000030 sp+0 u u[s]
> 0000000000000034 sp+0 u u
>
>
>
> (gdb) bt
> #0 find_fde (tbl=0xffff80007b157708, pc=18446603338286190664) at kernel/sframe_lookup.c:75
> #1 0xffff80008031e260 in sframe_find_pc (pc=18446603338286190664, entry=0xffff800086f83800) at kernel/sframe_lookup.c:175
> #2 0xffff800080035a48 in unwind_next_frame_sframe (state=0xffff800086f83828) at arch/arm64/kernel/stacktrace.c:270
> #3 kunwind_next (state=0xffff800086f83828) at arch/arm64/kernel/stacktrace.c:332
> ...
>
> (gdb) lx-symbols
> loading vmlinux
> scanning for modules in /home/wnliu/kernel
> loading @0xffff80007b155000: /home/wnliu/kernel/samples/livepatch/livepatch-sample.ko
> loading @0xffff80007b14d000: /home/wnliu/kernel/fs/fat/vfat.ko
> loading @0xffff80007b130000: /home/wnliu/kernel/fs/fat/fat.ko
>
> (gdb) p/x *tbl->sfhdr_p
> $5 = {preamble = {magic = 0xdee2, version = 0x2, flags = 0x1}, abi_arch = 0x2, cfa_fixed_fp_offset = 0x0, cfa_fixed_ra_offset = 0x0, auxhdr_len = 0x0, num_fdes = 0x3, num_fres = 0xb, fre_len = 0x25, fdes_off = 0x0, fres_off = 0x3c}
>
> (gdb) p/x tbl->sfhdr_p
> $6 = 0xffff80007b15a040
>
> (gdb) p *tbl->fde_p
> $7 = {start_addr = -20564, size = 12, fres_off = 0, fres_num = 1, info = 0 '\000', rep_size = 0 '\000', padding = 0}
>
> (gdb) p *(tbl->fde_p + 1)
> $11 = {start_addr = -20552, size = 44, fres_off = 3, fres_num = 5, info = 0 '\000', rep_size = 0 '\000', padding = 0}
>
> (gdb) p *(tbl->fde_p + 2)
> $12 = {start_addr = -20508, size = 56, fres_off = 20, fres_num = 5, info = 0 '\000', rep_size = 0 '\000', padding = 0}
>
> /* -20564 + 0xffff80007b15a040 = 0xffff80007b154fec */
> (gdb) info symbol 0xffff80007b154fec
> No symbol matches 0xffff80007b154fec
> ```