DW_AT_comp_dir and O= usage not working with objdump -dS, perf probe, etc

[Arnaldo Carvalho de Melo <acme@kernel.org>]

Hi,

	I noticed recently while testing some other patches that
disassembling with objdump -dS didn't work when building the kernel with
O= as it sets it to the build dir, not to where the sources are, for
instance:

Make sure perf uses objdump to disassembly:

root@number:~# rm -f ~/.perfconfig 
root@number:~# perf config annotate.disassemblers=objdump
root@number:~# perf record -a sleep 5 
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 2.012 MB perf.data (6307 samples) ]
root@number:~# pahole --running_kernel_vmlinux
/lib/modules/6.15.0-rc4+/build/vmlinux
root@number:~# readelf -wi /lib/modules/6.15.0-rc4+/build/vmlinux | grep -m1 DW_AT_comp_dir
    <17>   DW_AT_comp_dir    : (indirect line string, offset: 0): /home/acme/git/build/v6.15.0-rc4+
root@number:~# 

root@number:~# readelf -wi /lib/modules/6.15.0-rc4+/build/vmlinux | grep -m1 DW_AT_comp_dir
    <17>   DW_AT_comp_dir    : (indirect line string, offset: 0): /home/acme/git/build/v6.15.0-rc4+
root@number:~# perf report | grep -v ^# | head -10
    16.53%  swapper          [kernel.kallsyms]                  [k] poll_idle
    11.21%  swapper          [kernel.kallsyms]                  [k] acpi_os_read_port
     1.39%  swapper          [kernel.kallsyms]                  [k] io_idle
     0.92%  swapper          [kernel.kallsyms]                  [k] switch_mm_irqs_off
     0.87%  swapper          [kernel.kallsyms]                  [k] __update_load_avg_cfs_rq
     0.84%  swapper          [kernel.kallsyms]                  [k] read_tsc
     0.76%  swapper          [kernel.kallsyms]                  [k] psi_group_change
     0.76%  swapper          [kernel.kallsyms]                  [k] menu_select
     0.67%  swapper          [kernel.kallsyms]                  [k] native_sched_clock
     0.53%  Isolated Web Co  libxul.so                          [.] mozilla::EventListenerManager::AddEventListenerInternal(mozilla::dom::CallbackObjectHolder<mozilla::dom::EventListener, nsIDOMEventListener>, mozilla::EventMessage, nsAtom*, mozilla::EventListenerFlags const&, bool, bool, mozilla::dom::AbortSignal*)
root@number:~#
root@number:~# perf probe -L icmp_rcv
Failed to find source file path.
  Error: Failed to show lines.
root@number:~# perf probe -v -L icmp_rcv
Looking at the vmlinux_path (8 entries long)
Using /lib/modules/6.15.0-rc4+/build/vmlinux for symbols
Open Debuginfo file: /lib/modules/6.15.0-rc4+/build/vmlinux
fname: net/ipv4/icmp.c, lineno:1198
New line range: 1198 to 2147483647
path: net/ipv4/icmp.c
Search /home/acme/git/build/v6.15.0-rc4+/net/ipv4/icmp.c from debuginfod -> -2
Failed to find /home/acme/git/build/v6.15.0-rc4+/net/ipv4/icmp.c in debuginfod (d391f0e79126801bc8a8f907e763de7979941712)
Failed to find source file path.
  Error: Failed to show lines. Reason: No such file or directory (Code: -2)
root@number:~#

In 'perf probe' we have the old --source that allows us to override that
DW_AT_comp_dir pointing to the build dir (O= one) and then it works:

root@number:~# perf probe --source /home/acme/git/linux/ -L icmp_rcv | head -20
<icmp_rcv@/home/acme/git/linux//net/ipv4/icmp.c:0>
      0  int icmp_rcv(struct sk_buff *skb)
         {
      2  	enum skb_drop_reason reason = SKB_DROP_REASON_NOT_SPECIFIED;
         	struct rtable *rt = skb_rtable(skb);
         	struct net *net = dev_net_rcu(rt->dst.dev);
         	struct icmphdr *icmph;
         
         	if (!xfrm4_policy_check(NULL, XFRM_POLICY_IN, skb)) {
      8  		struct sec_path *sp = skb_sec_path(skb);
      9  		int nh;
         
         		if (!(sp && sp->xvec[sp->len - 1]->props.flags &
         				 XFRM_STATE_ICMP)) {
         			reason = SKB_DROP_REASON_XFRM_POLICY;
         			goto drop;
         		}
         
     17  		if (!pskb_may_pull(skb, sizeof(*icmph) + sizeof(struct iphdr)))
         			goto drop;
root@number:~#

But objdump has no override and thus doesn't work:

root@number:~# objdump --disassemble=icmp_rcv -S /lib/modules/6.15.0-rc4+/build/vmlinux | head -40

/lib/modules/6.15.0-rc4+/build/vmlinux:     file format elf64-x86-64


Disassembly of section .text:

ffffffff8231a6f0 <icmp_rcv>:
ffffffff8231a6f0:	f3 0f 1e fa          	endbr64
ffffffff8231a6f4:	e8 97 bf 05 ff       	call   ffffffff81376690 <__fentry__>
ffffffff8231a6f9:	41 56                	push   %r14
ffffffff8231a6fb:	31 f6                	xor    %esi,%esi
ffffffff8231a6fd:	41 54                	push   %r12
ffffffff8231a6ff:	55                   	push   %rbp
ffffffff8231a700:	53                   	push   %rbx
ffffffff8231a701:	48 89 fb             	mov    %rdi,%rbx
ffffffff8231a704:	48 83 ec 10          	sub    $0x10,%rsp
ffffffff8231a708:	48 8b 6f 58          	mov    0x58(%rdi),%rbp
ffffffff8231a70c:	48 83 e5 fe          	and    $0xfffffffffffffffe,%rbp
ffffffff8231a710:	48 8b 45 00          	mov    0x0(%rbp),%rax
ffffffff8231a714:	4c 8b a0 08 01 00 00 	mov    0x108(%rax),%r12
ffffffff8231a71b:	e8 d0 e4 ff ff       	call   ffffffff82318bf0 <__xfrm_policy_check2.constprop.0>
ffffffff8231a720:	85 c0                	test   %eax,%eax
ffffffff8231a722:	0f 85 c9 00 00 00    	jne    ffffffff8231a7f1 <icmp_rcv+0x101>
ffffffff8231a728:	f6 43 7f 02          	testb  $0x2,0x7f(%rbx)
ffffffff8231a72c:	0f 84 bc 01 00 00    	je     ffffffff8231a8ee <icmp_rcv+0x1fe>
ffffffff8231a732:	48 8b 83 e0 00 00 00 	mov    0xe0(%rbx),%rax
ffffffff8231a739:	0f b6 50 05          	movzbl 0x5(%rax),%edx
ffffffff8231a73d:	c1 e2 03             	shl    $0x3,%edx
ffffffff8231a740:	48 63 d2             	movslq %edx,%rdx
ffffffff8231a743:	48 01 c2             	add    %rax,%rdx
ffffffff8231a746:	0f 84 a2 01 00 00    	je     ffffffff8231a8ee <icmp_rcv+0x1fe>
ffffffff8231a74c:	8b 02                	mov    (%rdx),%eax
ffffffff8231a74e:	83 e8 01             	sub    $0x1,%eax
ffffffff8231a751:	48 98                	cltq
ffffffff8231a753:	48 83 f8 06          	cmp    $0x6,%rax
ffffffff8231a757:	0f 83 3b 03 00 00    	jae    ffffffff8231aa98 <icmp_rcv+0x3a8>
ffffffff8231a75d:	48 8b 44 c2 10       	mov    0x10(%rdx,%rax,8),%rax
ffffffff8231a762:	f6 80 01 01 00 00 10 	testb  $0x10,0x101(%rax)
ffffffff8231a769:	0f 84 7f 01 00 00    	je     ffffffff8231a8ee <icmp_rcv+0x1fe>
ffffffff8231a76f:	8b 53 70             	mov    0x70(%rbx),%edx
root@number:~#

I haven't checked, ran out of time today, but I think this may be
involved:

commit 97282e6d380db8a07120fe1b794ac969ee4a3b5c
Author: Thomas Weißschuh <linux@weissschuh.net>
Date:   Sat Mar 22 10:03:16 2025 +0100

    x86: drop unnecessary prefix map configuration

wdyt?

I'll continue tomorrow,

Cheers,

- Arnaldo