From: Arnaldo Carvalho de Melo <acme@kernel.org>
To: Lorenz Bauer <lmb@isovalent.com>
Cc: Andrii Nakryiko <andrii@kernel.org>,
Alan Maguire <alan.maguire@oracle.com>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
bpf@vger.kernel.org, linux-perf-users@vger.kernel.org
Subject: BTF loading failing on perf
Date: Fri, 6 Jun 2025 12:37:55 -0300 [thread overview]
Message-ID: <aEMLU2li1x2bAO4w@x1> (raw)
root@number:~# perf trace -e openat --max-events=1
libbpf: failed to read kernel BTF from '/sys/kernel/btf/vmlinux': -ENODEV
libbpf: failed to read kernel BTF from '/sys/kernel/btf/vmlinux': -ENODEV
0.000 ( 0.016 ms): ptyxis-agent/4375 openat(dfd: CWD, filename: "/proc/6593/cmdline", flags: RDONLY|CLOEXEC) = 13
root@number:~#
openat(AT_FDCWD, "/sys/kernel/btf/vmlinux", O_RDONLY) = 258
mmap(NULL, 6519699, PROT_READ, MAP_PRIVATE, 258, 0) = -1 ENODEV (No such device)
libbpf: failed to read kernel BTF from '/sys/kernel/btf/vmlinux': -ENODEV
root@number:~# ls -la /sys/kernel/btf/vmlinux
-r--r--r--. 1 root root 6519699 Jun 6 12:19 /sys/kernel/btf/vmlinux
root@number:~# uname -a
Linux number 6.14.9-300.fc42.x86_64 #1 SMP PREEMPT_DYNAMIC Thu May 29 14:27:53 UTC 2025 x86_64 GNU/Linux
root@number:~# head /etc/os-release
NAME="Fedora Linux"
VERSION="42 (Workstation Edition)"
RELEASE_TYPE=stable
ID=fedora
VERSION_ID=42
VERSION_CODENAME=""
PLATFORM_ID="platform:f42"
PRETTY_NAME="Fedora Linux 42 (Workstation Edition)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
root@number:~# rpm -q glibc-devel
package glibc-devel is not installed
root@number:~# rpm -q glibc
glibc-2.41-5.fc42.x86_64
root@number:~#
Reverting the patch below "cures" the problem.
⬢ [acme@toolbx perf-tools]$ git log --oneline -10 tools/lib/bpf/
370118ff875244d4 (HEAD -> perf-tools) Revert "libbpf: Use mmap to parse vmlinux BTF from sysfs"
90b83efa6701656e Merge tag 'bpf-next-6.16' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next
3c0421c93ce4ff0f libbpf: Use mmap to parse vmlinux BTF from sysfs
4e29128a9acec2a6 libbpf/btf: Fix string handling to support multi-split BTF
d0445d7dd3fd9b15 libbpf: Check bpf_map_skeleton link for NULL
fd5fd538a1f4b34c libbpf: Use proper errno value in nlattr
62e23f183839c3d7 libbpf: Improve BTF dedup handling of "identical" BTF types
41d4ce6df3f49453 bpf: Fix uninitialized values in BPF_{CORE,PROBE}_READ
358b1c0f56ebb699 libbpf: Use proper errno value in linker
8e64c387c942229c libbpf: Add identical pointer detection to btf_dedup_is_equiv()
⬢ [acme@toolbx perf-tools]$
root@number:~# perf -v
perf version 6.15.g370118ff8752
root@number:~# perf trace -e connect --max-events=5
0.000 ( 0.021 ms): DNS Res~ver #4/7932 connect(fd: 102, uservaddr: { .family: LOCAL, path: /run/systemd/r }, addrlen: 42) = 0
0.304 ( 0.013 ms): systemd-resolv/1420 connect(fd: 25, uservaddr: { .family: INET, port: 53, addr: 186.208.78.194 }, addrlen: 16) = 0
0.371 ( 0.002 ms): systemd-resolv/1420 connect(fd: 26, uservaddr: { .family: INET, port: 53, addr: 186.208.78.194 }, addrlen: 16) = 0
0.591 ( 0.009 ms): DNS Res~ver #2/7523 connect(fd: 104, uservaddr: { .family: LOCAL, path: /run/systemd/r }, addrlen: 42) = 0
0.731 ( 0.006 ms): systemd-resolv/1420 connect(fd: 28, uservaddr: { .family: INET, port: 53, addr: 186.208.78.194 }, addrlen: 16) = 0
root@number:~#
Ideas?
- Arnaldo
⬢ [acme@toolbx perf-tools]$ git bisect good
3c0421c93ce4ff0f5f2612666122c34fc941d569 is the first bad commit
commit 3c0421c93ce4ff0f5f2612666122c34fc941d569
Author: Lorenz Bauer <lmb@isovalent.com>
Date: Tue May 20 14:01:19 2025 +0100
libbpf: Use mmap to parse vmlinux BTF from sysfs
Teach libbpf to use mmap when parsing vmlinux BTF from /sys. We don't
apply this to fall-back paths on the regular file system because there
is no way to ensure that modifications underlying the MAP_PRIVATE
mapping are not visible to the process.
Signed-off-by: Lorenz Bauer <lmb@isovalent.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Tested-by: Alan Maguire <alan.maguire@oracle.com>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20250520-vmlinux-mmap-v5-3-e8c941acc414@isovalent.com
tools/lib/bpf/btf.c | 89 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------------
1 file changed, 71 insertions(+), 18 deletions(-)
⬢ [acme@toolbx perf-tools]$
next reply other threads:[~2025-06-06 15:37 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-06-06 15:37 Arnaldo Carvalho de Melo [this message]
2025-06-06 16:14 ` BTF loading failing on perf Leo Yan
2025-06-06 16:20 ` Andrii Nakryiko
2025-06-06 20:05 ` Arnaldo Carvalho de Melo
2025-06-06 20:22 ` Andrii Nakryiko
2025-06-10 20:27 ` Arnaldo Carvalho de Melo
2025-06-10 23:12 ` Andrii Nakryiko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=aEMLU2li1x2bAO4w@x1 \
--to=acme@kernel.org \
--cc=alan.maguire@oracle.com \
--cc=andrii@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=lmb@isovalent.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox