Re: Call graph dwarf unwinding fails with lld

[Fangrui Song <maskray@google.com>]

On 2022-05-25, Ian Rogers wrote:
>On Tue, May 24, 2022 at 1:50 PM Travis Downs <travis.downs@gmail.com> wrote:
>>
>> I have been tracking down an issue locally where `perf record
>> --call-graph=dwarf` stopped working in the sense that unwinding always
>> failed.
>>
>> Turns out it was related to `lld` - binaries linked with `lld` don't
>> seem to unwind correctly with `--call-graph=dwarf`. I've confirmed
>> that the `.eh_frame` sections are there, but I guess they are
>> different after using this linker.
>>
>> Has anyone run into this?
>
>I was unaware but thanks for filing the bug!
>
>> There's an open LLVM issue at https://github.com/llvm/llvm-project/issues/53156
>>
>> ... but no activity since it was filed, though it is not clear if the
>> problem is really with lld or the unwinders.
>>
>> In a related question: perf supports more than one DWARF unwinding
>> implementation, depending on what libraries are available at build
>> time, right? Is there a way to select which unwinder is used at
>> runtime?
>
>I don't think so. There are two build options NO_LIBUNWIND=1 and
>NO_LIBDW_DWARF_UNWIND=1. I played around with the example from the
>bug, confirmed no difference with the eh_frames and hand verified
>them, I also added -fno-omit-frame-pointer -O0 and -static. I've not
>got a good handle on the issue but my suspicion is that the
>eh_frame_hdr is broken - it is at least something that lld generates.
>If I change the eh_frame_hdr finding to some findable but unexpected
>offset like eh_frame here:
>https://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git/tree/tools/perf/util/unwind-libunwind-local.c?h=perf/core#n286
>and then test with a working linker the unwinding is broken similarly
>to with lld. I was looking for a useful tool (objdump, llvm-objdump,
>readelf, dwarfdump) to look at the eh_frame_hdr but I didn't see one.
>Perhaps someone has a suggestion or maybe pahole can do it. Hopefully
>this gives a lead for you to dig into for the bug.
>
>Thanks,
>Ian

I am very familiar with lld and have read some nongnu libunwind code, so I guess I
may be an appropriate one debugging the issue, but I am puzzled about how to build tools/perf:

/tmp/out/custom1 has needed llvm-project tools, built from origin/main
ninja -C /tmp/out/custom1 clang lld llvm-{nm,size,strings,objcopy,objdump,readelf,ar,strip}

```
% git checkout master  # torvalds/linux
% PATH=/tmp/out/custom1/bin:$PATH make O=/tmp/linux/x86_64 ARCH=x86_64 -j60 defconfig all
% PATH=/tmp/out/custom1/bin:$PATH make O=/tmp/linux/x86_64 ARCH=x86_64 -j60 -C tools/perf
...
In file included from bench/../../arch/x86/lib/memcpy_64.S:8,
                  from bench/mem-memcpy-x86-64-asm.S:14:
/tmp/linux/x86_64/arch/x86/include/generated/asm/export.h:1:10: fatal error: asm-generic/export.h: No such file or directory
     1 | #include <asm-generic/export.h>
       |          ^~~~~~~~~~~~~~~~~~~~~~
```

I am running a 5.16 kernel so I have tried `git checkout v5.16`, no luck as well.
I cannot find useful information about "asm-generic/export.h: no such file or directory" on www:(