Re: Not finding symbol information

[Milian Wolff <milian.wolff@kdab.com>]

[-- Attachment #1: Type: text/plain, Size: 3771 bytes --]

On Friday, February 2, 2018 3:06:32 PM CET Vuille, Martin (Martin) wrote:
> We are using 'perf record' to collect information on an embedded ARM board
> (the target) and then using 'perf report' on a PC (the host) to interpret
> the data.
> 
> The host has a copy of the rootfs of the target that also includes debug
> info. We pass the '--symfs' option to 'perf report' to point to the rootfs.
> 
> Whenever an event in the data corresponds to a DSO that exists on the host,
> 'perf report' complains that the debug info is not available, even though
> that info is in fact available in the rootfs.
> 
> What seems to be happening is that 'perf report' "sees" the host's copy of
> the DSO first, "remembers" this DSO's buildid (which is incorrect), then
> "sees" the debug info in the rootfs but rejects it because it doesn't match
> the buildid "remembered" earlier, and then concludes that there is no debug
> info available.

`perf record` (not `perf report`) is "remembering" build-ids in ~/.debug.

> Using /bin/bash as an example, selected output from 'perf report -v':
> 
> ...
> build id event received for /bin/bash:
> a35856957f6f165a1ca4ea98afbd1a986dfa785c ...
> symsrc__init: build id mismatch for
> sip96x1_prod/cp/lib/dvf99/sysroots/armv5te-dspg-linux-gnueabi///bin/bash.
> symsrc__init: build id mismatch for
> sip96x1_prod/cp/lib/dvf99/sysroots/armv5te-dspg-linux-gnueabi///bin/bash.
> symsrc__init: build id mismatch for
> sip96x1_prod/cp/lib/dvf99/sysroots/armv5te-dspg-linux-gnueabi///bin/.debug/
> bash. /bin/bash with build id 04eca96c5bf3e9a300952a29ef3218f00487d37b not
> found, continuing without symbols ...
> 
> $ file /bin/bash
> /bin/bash: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically
> linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32,
> BuildID[sha1]=04eca96c5bf3e9a300952a29ef3218f00487d37b, stripped
> 
> $ file
> sip96x1_prod/cp/lib/dvf99/sysroots/armv5te-dspg-linux-gnueabi///bin/bash
> sip96x1_prod/cp/lib/dvf99/sysroots/armv5te-dspg-linux-gnueabi///bin/bash:
> ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked,
> interpreter /lib/ld-linux.so.3, for GNU/Linux 3.2.0,
> BuildID[sha1]=a35856957f6f165a1ca4ea98afbd1a986dfa785c, stripped
> 
> $ file
> sip96x1_prod/cp/lib/dvf99/sysroots/armv5te-dspg-linux-gnueabi///bin/.debug/
> bash
> sip96x1_prod/cp/lib/dvf99/sysroots/armv5te-dspg-linux-gnueabi///bin/.debug/
> bash: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically
> linked, interpreter *empty*, for GNU/Linux 3.2.0,
> BuildID[sha1]=a35856957f6f165a1ca4ea98afbd1a986dfa785c, not stripped

This indeed looks like a bug in the build id matching algorithm. I suggest you 
debug it to figure out why it thinks the build id mismatches. I.e. expand the 
debug output and write the build id it has read and compare that to what file 
sees and what the perf file expects.

> We are using Linux 4.4-11 on the target and 4.4.0-109-generic (Ubuntu 16.04)
> on the host.
> 
> Is this a known issue? Has it been addressed in a more recent kernel?

This should not be part of the kernel, but rather part of the userland perf 
tools. You can easily build the latest versions of those yourself, without 
having to build a new kernel. So on your host, try this:

git clone git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git -b 
perf/core
cd tools/perf
make # make sure you have all the required dependencies, most notably 
elfutils, libiberty, ...
/perf report -i <path to perf.data>

Hope that helps

PS: You could also try to use our hotspot GUI:
https://github.com/KDAB/hotspot
-- 
Milian Wolff | milian.wolff@kdab.com | Senior Software Engineer
KDAB (Deutschland) GmbH, a KDAB Group company
Tel: +49-30-521325470
KDAB - The Qt, C++ and OpenGL Experts

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 3826 bytes --]