Re: [PATCH 0/8] unwind, arm64: add sframe unwinder for kernel

[Indu Bhagat <indu.bhagat@oracle.com>]

On Mon, Feb 10, 2025 at 12:30 AM Weinan Liu <wnliu@google.com> wrote:
>> I already have a WIP patch to add sframe support to the kernel module.
>> However, it is not yet working. I had trouble unwinding frames for the
>> kernel module using the current algorithm.
>>
>> Indu has likely identified the issue and will be addressing it from the
>> toolchain side.
>>
>> https://sourceware.org/bugzilla/show_bug.cgi?id=32666
> 
> I have a working in progress patch that adds sframe support for kernel
> module.
> https://github.com/heuza/linux/tree/sframe_unwinder.rfc
> 
> According to the sframe table values I got during runtime testing, looks
> like the offsets are not correct .
> 

I hope to sanitize the fix for 32666 and post upstream soon (I had to 
address other related issues).  Unless fixed, relocating .sframe 
sections using the .rela.sframe is expected to generate incorrect output.

> When unwind symbols init_module(0xffff80007b155048) from the kernel
> module(livepatch-sample.ko), the start_address of the FDE entries in the
> sframe table of the kernel modules appear incorrect.

init_module will apply the relocations on the .sframe section, isnt it ?

> For instance, the first FDE's start_addr is reported as -20564. Adding
> this offset to the module's sframe section address (0xffff80007b15a040)
> yields 0xffff80007b154fec, which is not within the livepatch-sample.ko
> memory region(It should be larger than 0xffff80007b155000).
> 

Hmm..something seems off here.  Having tested a potential fix for 32666 
locally, I do not expect the first FDE to show this symptom.

> Here are the sframe table values of the livepatch-samples.ko that I print
> by qemu + gdb.
> 
> ```
> $ /usr/bin/aarch64-linux-gnu-objdump -L --sframe=.sframe ./samples/livepatch/livepatch-sample.ko
> ./samples/livepatch/livepatch-sample.ko:     file format elf64-littleaarch64
> 
> Contents of the SFrame section .sframe:
>   Header :
> 
>     Version: SFRAME_VERSION_2
>     Flags: SFRAME_F_FDE_SORTED
>     Num FDEs: 3
>     Num FREs: 11
> 
>   Function Index :
> 
>     func idx [0]: pc = 0x0, size = 12 bytes
>     STARTPC         CFA       FP        RA
>     0000000000000000  sp+0      u         u
> 
>     func idx [1]: pc = 0x0, size = 44 bytes
>     STARTPC         CFA       FP        RA
>     0000000000000000  sp+0      u         u
>     000000000000000c  sp+0      u         u[s]
>     0000000000000010  sp+16     c-16      c-8[s]
>     0000000000000024  sp+0      u         u[s]
>     0000000000000028  sp+0      u         u
> 
>     func idx [2]: pc = 0x0, size = 56 bytes
>     STARTPC         CFA       FP        RA
>     0000000000000000  sp+0      u         u
>     000000000000000c  sp+0      u         u[s]
>     0000000000000010  sp+16     c-16      c-8[s]
>     0000000000000030  sp+0      u         u[s]
>     0000000000000034  sp+0      u         u
> 
> 
> 
> (gdb) bt
> #0  find_fde (tbl=0xffff80007b157708, pc=18446603338286190664) at kernel/sframe_lookup.c:75
> #1  0xffff80008031e260 in sframe_find_pc (pc=18446603338286190664, entry=0xffff800086f83800) at kernel/sframe_lookup.c:175
> #2  0xffff800080035a48 in unwind_next_frame_sframe (state=0xffff800086f83828) at arch/arm64/kernel/stacktrace.c:270
> #3  kunwind_next (state=0xffff800086f83828) at arch/arm64/kernel/stacktrace.c:332
> ...
> 
> (gdb) lx-symbols
> loading vmlinux
> scanning for modules in /home/wnliu/kernel
> loading @0xffff80007b155000: /home/wnliu/kernel/samples/livepatch/livepatch-sample.ko
> loading @0xffff80007b14d000: /home/wnliu/kernel/fs/fat/vfat.ko
> loading @0xffff80007b130000: /home/wnliu/kernel/fs/fat/fat.ko
> 
> (gdb) p/x *tbl->sfhdr_p
> $5 = {preamble = {magic = 0xdee2, version = 0x2, flags = 0x1}, abi_arch = 0x2, cfa_fixed_fp_offset = 0x0, cfa_fixed_ra_offset = 0x0, auxhdr_len = 0x0, num_fdes = 0x3, num_fres = 0xb, fre_len = 0x25, fdes_off = 0x0, fres_off = 0x3c}
> 
> (gdb) p/x tbl->sfhdr_p
> $6 = 0xffff80007b15a040
> 
> (gdb) p *tbl->fde_p
> $7 = {start_addr = -20564, size = 12, fres_off = 0, fres_num = 1, info = 0 '\000', rep_size = 0 '\000', padding = 0}
> 
> (gdb) p *(tbl->fde_p + 1)
> $11 = {start_addr = -20552, size = 44, fres_off = 3, fres_num = 5, info = 0 '\000', rep_size = 0 '\000', padding = 0}
> 
> (gdb) p *(tbl->fde_p + 2)
> $12 = {start_addr = -20508, size = 56, fres_off = 20, fres_num = 5, info = 0 '\000', rep_size = 0 '\000', padding = 0}
> 
> /* -20564 + 0xffff80007b15a040 = 0xffff80007b154fec */
> (gdb) info symbol 0xffff80007b154fec
> No symbol matches 0xffff80007b154fec
> ```