From: Arnaldo Carvalho de Melo <acme@kernel.org>
To: Lorenz Bauer <lmb@isovalent.com>
Cc: Andrii Nakryiko <andrii@kernel.org>,
Alan Maguire <alan.maguire@oracle.com>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
bpf@vger.kernel.org, linux-perf-users@vger.kernel.org
Subject: BTF loading failing on perf
Date: Fri, 6 Jun 2025 12:37:55 -0300 [thread overview]
Message-ID: <aEMLU2li1x2bAO4w@x1> (raw)
root@number:~# perf trace -e openat --max-events=1
libbpf: failed to read kernel BTF from '/sys/kernel/btf/vmlinux': -ENODEV
libbpf: failed to read kernel BTF from '/sys/kernel/btf/vmlinux': -ENODEV
0.000 ( 0.016 ms): ptyxis-agent/4375 openat(dfd: CWD, filename: "/proc/6593/cmdline", flags: RDONLY|CLOEXEC) = 13
root@number:~#
openat(AT_FDCWD, "/sys/kernel/btf/vmlinux", O_RDONLY) = 258
mmap(NULL, 6519699, PROT_READ, MAP_PRIVATE, 258, 0) = -1 ENODEV (No such device)
libbpf: failed to read kernel BTF from '/sys/kernel/btf/vmlinux': -ENODEV
root@number:~# ls -la /sys/kernel/btf/vmlinux
-r--r--r--. 1 root root 6519699 Jun 6 12:19 /sys/kernel/btf/vmlinux
root@number:~# uname -a
Linux number 6.14.9-300.fc42.x86_64 #1 SMP PREEMPT_DYNAMIC Thu May 29 14:27:53 UTC 2025 x86_64 GNU/Linux
root@number:~# head /etc/os-release
NAME="Fedora Linux"
VERSION="42 (Workstation Edition)"
RELEASE_TYPE=stable
ID=fedora
VERSION_ID=42
VERSION_CODENAME=""
PLATFORM_ID="platform:f42"
PRETTY_NAME="Fedora Linux 42 (Workstation Edition)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
root@number:~# rpm -q glibc-devel
package glibc-devel is not installed
root@number:~# rpm -q glibc
glibc-2.41-5.fc42.x86_64
root@number:~#
Reverting the patch below "cures" the problem.
⬢ [acme@toolbx perf-tools]$ git log --oneline -10 tools/lib/bpf/
370118ff875244d4 (HEAD -> perf-tools) Revert "libbpf: Use mmap to parse vmlinux BTF from sysfs"
90b83efa6701656e Merge tag 'bpf-next-6.16' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next
3c0421c93ce4ff0f libbpf: Use mmap to parse vmlinux BTF from sysfs
4e29128a9acec2a6 libbpf/btf: Fix string handling to support multi-split BTF
d0445d7dd3fd9b15 libbpf: Check bpf_map_skeleton link for NULL
fd5fd538a1f4b34c libbpf: Use proper errno value in nlattr
62e23f183839c3d7 libbpf: Improve BTF dedup handling of "identical" BTF types
41d4ce6df3f49453 bpf: Fix uninitialized values in BPF_{CORE,PROBE}_READ
358b1c0f56ebb699 libbpf: Use proper errno value in linker
8e64c387c942229c libbpf: Add identical pointer detection to btf_dedup_is_equiv()
⬢ [acme@toolbx perf-tools]$
root@number:~# perf -v
perf version 6.15.g370118ff8752
root@number:~# perf trace -e connect --max-events=5
0.000 ( 0.021 ms): DNS Res~ver #4/7932 connect(fd: 102, uservaddr: { .family: LOCAL, path: /run/systemd/r }, addrlen: 42) = 0
0.304 ( 0.013 ms): systemd-resolv/1420 connect(fd: 25, uservaddr: { .family: INET, port: 53, addr: 186.208.78.194 }, addrlen: 16) = 0
0.371 ( 0.002 ms): systemd-resolv/1420 connect(fd: 26, uservaddr: { .family: INET, port: 53, addr: 186.208.78.194 }, addrlen: 16) = 0
0.591 ( 0.009 ms): DNS Res~ver #2/7523 connect(fd: 104, uservaddr: { .family: LOCAL, path: /run/systemd/r }, addrlen: 42) = 0
0.731 ( 0.006 ms): systemd-resolv/1420 connect(fd: 28, uservaddr: { .family: INET, port: 53, addr: 186.208.78.194 }, addrlen: 16) = 0
root@number:~#
Ideas?
- Arnaldo
⬢ [acme@toolbx perf-tools]$ git bisect good
3c0421c93ce4ff0f5f2612666122c34fc941d569 is the first bad commit
commit 3c0421c93ce4ff0f5f2612666122c34fc941d569
Author: Lorenz Bauer <lmb@isovalent.com>
Date: Tue May 20 14:01:19 2025 +0100
libbpf: Use mmap to parse vmlinux BTF from sysfs
Teach libbpf to use mmap when parsing vmlinux BTF from /sys. We don't
apply this to fall-back paths on the regular file system because there
is no way to ensure that modifications underlying the MAP_PRIVATE
mapping are not visible to the process.
Signed-off-by: Lorenz Bauer <lmb@isovalent.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Tested-by: Alan Maguire <alan.maguire@oracle.com>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20250520-vmlinux-mmap-v5-3-e8c941acc414@isovalent.com
tools/lib/bpf/btf.c | 89 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------------
1 file changed, 71 insertions(+), 18 deletions(-)
⬢ [acme@toolbx perf-tools]$
next reply other threads:[~2025-06-06 15:37 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-06-06 15:37 Arnaldo Carvalho de Melo [this message]
2025-06-06 16:14 ` BTF loading failing on perf Leo Yan
2025-06-06 16:20 ` Andrii Nakryiko
2025-06-06 20:05 ` Arnaldo Carvalho de Melo
2025-06-06 20:22 ` Andrii Nakryiko
2025-06-10 20:27 ` Arnaldo Carvalho de Melo
2025-06-10 23:12 ` Andrii Nakryiko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=aEMLU2li1x2bAO4w@x1 \
--to=acme@kernel.org \
--cc=alan.maguire@oracle.com \
--cc=andrii@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=lmb@isovalent.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.