* [RFC 0/2] perf build: Improve header handling for BPF skeleton cross-builds
@ 2025-12-23 8:43 hupu
2025-12-23 8:43 ` [RFC 1/2] perf build: Allow passing extra Clang flags via EXTRA_BPF_FLAGS hupu
2025-12-23 8:43 ` [RFC 2/2] perf build: Prefer kernel source headers for BPF skeletons hupu
0 siblings, 2 replies; 7+ messages in thread
From: hupu @ 2025-12-23 8:43 UTC (permalink / raw)
To: peterz, mingo, acme, namhyung, mark.rutland, alexander.shishkin,
jolsa, irogers, adrian.hunter, james.clark, nathan,
nick.desaulniers+lkml, morbo, justinstitt, linux-perf-users,
linux-kernel, llvm, hupu.gm
I am currently cross-compiling perf for ARM64. During the build of the
eBPF skeleton (compiled by Clang), the build fails due to missing header
files. The error message is as follows:
/usr/include/linux/ioctl.h:5:10: fatal error: 'asm/ioctl.h' file not
found
5 | #include <asm/ioctl.h>
| ^~~~~~~~~~~~~
Leo suggested resolving this issue by installing the following packages
on the host:
$ sudo apt-get install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu
$ sudo apt-get install libc6-dev-aarch64-cross linux-libc-dev-aarch64-cross
$ sudo apt-get install libc6-dev-arm64-cross linux-libc-dev-arm64-cross
With these packages installed, the required header files can be found in
the following host paths (defined by the CLANG_SYS_INCLUDES variable),
and perf can indeed be built successfully:
*** Makefile.perf:1203: ***
CLANG_SYS_INCLUDES=-idirafter /usr/lib/llvm-18/lib/clang/18/include
-idirafter /usr/local/include
-idirafter /usr/bin/../lib/gcc-cross/aarch64-linux-gnu/14/../../../../aarch64-linux-gnu/include
-idirafter /usr/include/aarch64-linux-gnu
-idirafter /usr/include
However, I do not believe that relying on the host build environment is
the best approach, for several reasons:
a) These commands install UAPI header files on the host, especially
`linux-libc-dev-aarch64-cross` and `linux-libc-dev-arm64-cross`. These
headers originate from the kernel source tree’s `include/uapi/` and
`arch/arm64/include/uapi/` directories, and their versions are tied to
the *HOST* kernel version. If the target kernel version is different,
mismatches may cause compilation errors or even runtime failures.
b) Even if `perf` can be compiled and run successfully now, there is
no guarantee that the kernel source headers will always match the
host-installed UAPI headers as the upstream kernel evolves.
c) In scenarios where the host acts as a general build server and
needs to build multiple target kernel versions, it is not possible to
ensure that the host UAPI headers are compatible with all target
versions.
d) On the other hand, `CLANG_SYS_INCLUDES` does include host headers,
but it uses `-idirafter` instead of `-I`. This means the host headers
have lower priority. This change was introduced in commit a2af0f6b8ef7
("perf build: Add system include paths to BPF builds"); as noted in
the commit message, the preferred approach is to use kernel source
headers rather than potentially older ones from the system.
Therefore, although installing the corresponding packages on the host
can resolve the immediate build issue, I do not think this should be
considered the only solution, nor should it have the highest priority.
Instead, I think the header search order should be as follows.
- Prefer kernel source headers
[RFC 2/2] perf build: Prefer kernel source headers for BPF skeletons
With this approach, even as the kernel version evolves, the headers
used during the build will always match the target kernel version.
- Allow users to specify header search paths matching the target kernel
version (eg. via `EXTRA_BPF_FLAGS`)
[RFC 1/2] perf build: Allow passing extra Clang flags via EXTRA_BPF_FLAGS
With this approach, users can explicitly specify the appropriate
header search paths via EXTRA_BPF_FLAGS.
- Fall back to the host build environment only if necessary
Reliance on the host build environment should be the last resort.
In summary, while relying on the host build environment can address the
current build issue, I believe it is not the optimal solution.
hupu (2):
perf build: Allow passing extra Clang flags via EXTRA_BPF_FLAGS
perf build: Prefer kernel source headers for BPF skeletons
tools/perf/Makefile.perf | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
--
2.43.0
^ permalink raw reply [flat|nested] 7+ messages in thread* [RFC 1/2] perf build: Allow passing extra Clang flags via EXTRA_BPF_FLAGS
2025-12-23 8:43 [RFC 0/2] perf build: Improve header handling for BPF skeleton cross-builds hupu
@ 2025-12-23 8:43 ` hupu
2025-12-23 18:20 ` Namhyung Kim
2025-12-23 8:43 ` [RFC 2/2] perf build: Prefer kernel source headers for BPF skeletons hupu
1 sibling, 1 reply; 7+ messages in thread
From: hupu @ 2025-12-23 8:43 UTC (permalink / raw)
To: peterz, mingo, acme, namhyung, mark.rutland, alexander.shishkin,
jolsa, irogers, adrian.hunter, james.clark, nathan,
nick.desaulniers+lkml, morbo, justinstitt, linux-perf-users,
linux-kernel, llvm, hupu.gm
Add support for EXTRA_BPF_FLAGS in the eBPF skeleton build, allowing
users to pass additional Clang options such as --sysroot or custom
include paths when cross-compiling perf.
This is primarily intended for cross-build scenarios where the default
host include paths do not match the target kernel version.
Example usage:
make perf ARCH="arm64" EXTRA_BPF_FLAGS="--sysroot=..."
Change history:
v5:
- Update commit message
v4:
- Apply EXTRA_BPF_FLAGS directly to BPF skeleton build command
instead of modifying CLANG_OPTIONS
v3:
- Move the variable description comment to the top of Makefile.perf
- Update commit message
v2:
- Rename EXTRA_CLANG_FLAGS to EXTRA_BPF_FLAGS
- Update commit message
v1:
- Introduce EXTRA_CLANG_FLAGS to allow passing extra Clang options
Signed-off-by: hupu <hupu.gm@gmail.com>
---
tools/perf/Makefile.perf | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index b3f481a626af..8b8647fd4f41 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -35,6 +35,9 @@ include ../scripts/utilities.mak
#
# Define EXTRA_CFLAGS=-m64 or EXTRA_CFLAGS=-m32 as appropriate for cross-builds.
#
+# Define EXTRA_BPF_FLAGS="--sysroot=<path>" or other custom include paths for
+# cross-compiling BPF skeletons
+#
# Define EXCLUDE_EXTLIBS=-lmylib to exclude libmylib from the auto-generated
# EXTLIBS.
#
@@ -1250,7 +1253,7 @@ endif
$(SKEL_TMP_OUT)/%.bpf.o: $(OUTPUT)PERF-VERSION-FILE util/bpf_skel/perf_version.h | $(SKEL_TMP_OUT)
$(SKEL_TMP_OUT)/%.bpf.o: util/bpf_skel/%.bpf.c $(LIBBPF) $(SKEL_OUT)/vmlinux.h
$(QUIET_CLANG)$(CLANG) -g -O2 -fno-stack-protector --target=bpf \
- $(CLANG_OPTIONS) $(BPF_INCLUDE) $(TOOLS_UAPI_INCLUDE) \
+ $(CLANG_OPTIONS) $(EXTRA_BPF_FLAGS) $(BPF_INCLUDE) $(TOOLS_UAPI_INCLUDE) \
-include $(OUTPUT)PERF-VERSION-FILE -include util/bpf_skel/perf_version.h \
-c $(filter util/bpf_skel/%.bpf.c,$^) -o $@
--
2.43.0
^ permalink raw reply related [flat|nested] 7+ messages in thread* Re: [RFC 1/2] perf build: Allow passing extra Clang flags via EXTRA_BPF_FLAGS
2025-12-23 8:43 ` [RFC 1/2] perf build: Allow passing extra Clang flags via EXTRA_BPF_FLAGS hupu
@ 2025-12-23 18:20 ` Namhyung Kim
2026-02-10 5:54 ` hupu
0 siblings, 1 reply; 7+ messages in thread
From: Namhyung Kim @ 2025-12-23 18:20 UTC (permalink / raw)
To: hupu
Cc: peterz, mingo, acme, mark.rutland, alexander.shishkin, jolsa,
irogers, adrian.hunter, james.clark, nathan,
nick.desaulniers+lkml, morbo, justinstitt, linux-perf-users,
linux-kernel, llvm
Hello,
On Tue, Dec 23, 2025 at 04:43:34PM +0800, hupu wrote:
> Add support for EXTRA_BPF_FLAGS in the eBPF skeleton build, allowing
> users to pass additional Clang options such as --sysroot or custom
> include paths when cross-compiling perf.
>
> This is primarily intended for cross-build scenarios where the default
> host include paths do not match the target kernel version.
>
> Example usage:
> make perf ARCH="arm64" EXTRA_BPF_FLAGS="--sysroot=..."
>
> Change history:
> v5:
> - Update commit message
> v4:
> - Apply EXTRA_BPF_FLAGS directly to BPF skeleton build command
> instead of modifying CLANG_OPTIONS
> v3:
> - Move the variable description comment to the top of Makefile.perf
> - Update commit message
> v2:
> - Rename EXTRA_CLANG_FLAGS to EXTRA_BPF_FLAGS
> - Update commit message
> v1:
> - Introduce EXTRA_CLANG_FLAGS to allow passing extra Clang options
>
> Signed-off-by: hupu <hupu.gm@gmail.com>
Reviewed-by: Namhyung Kim <namhyung@kernel.org>
Thanks,
Namhyung
> ---
> tools/perf/Makefile.perf | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> index b3f481a626af..8b8647fd4f41 100644
> --- a/tools/perf/Makefile.perf
> +++ b/tools/perf/Makefile.perf
> @@ -35,6 +35,9 @@ include ../scripts/utilities.mak
> #
> # Define EXTRA_CFLAGS=-m64 or EXTRA_CFLAGS=-m32 as appropriate for cross-builds.
> #
> +# Define EXTRA_BPF_FLAGS="--sysroot=<path>" or other custom include paths for
> +# cross-compiling BPF skeletons
> +#
> # Define EXCLUDE_EXTLIBS=-lmylib to exclude libmylib from the auto-generated
> # EXTLIBS.
> #
> @@ -1250,7 +1253,7 @@ endif
> $(SKEL_TMP_OUT)/%.bpf.o: $(OUTPUT)PERF-VERSION-FILE util/bpf_skel/perf_version.h | $(SKEL_TMP_OUT)
> $(SKEL_TMP_OUT)/%.bpf.o: util/bpf_skel/%.bpf.c $(LIBBPF) $(SKEL_OUT)/vmlinux.h
> $(QUIET_CLANG)$(CLANG) -g -O2 -fno-stack-protector --target=bpf \
> - $(CLANG_OPTIONS) $(BPF_INCLUDE) $(TOOLS_UAPI_INCLUDE) \
> + $(CLANG_OPTIONS) $(EXTRA_BPF_FLAGS) $(BPF_INCLUDE) $(TOOLS_UAPI_INCLUDE) \
> -include $(OUTPUT)PERF-VERSION-FILE -include util/bpf_skel/perf_version.h \
> -c $(filter util/bpf_skel/%.bpf.c,$^) -o $@
>
> --
> 2.43.0
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [RFC 1/2] perf build: Allow passing extra Clang flags via EXTRA_BPF_FLAGS
2025-12-23 18:20 ` Namhyung Kim
@ 2026-02-10 5:54 ` hupu
2026-02-11 16:27 ` Arnaldo Carvalho de Melo
0 siblings, 1 reply; 7+ messages in thread
From: hupu @ 2026-02-10 5:54 UTC (permalink / raw)
To: Namhyung Kim
Cc: peterz, mingo, acme, mark.rutland, alexander.shishkin, jolsa,
irogers, adrian.hunter, james.clark, nathan,
nick.desaulniers+lkml, morbo, justinstitt, linux-perf-users,
linux-kernel, llvm
On Wed, Dec 24, 2025 at 2:20 AM Namhyung Kim <namhyung@kernel.org> wrote:
>
> Hello,
>
> On Tue, Dec 23, 2025 at 04:43:34PM +0800, hupu wrote:
> > Add support for EXTRA_BPF_FLAGS in the eBPF skeleton build, allowing
> > users to pass additional Clang options such as --sysroot or custom
> > include paths when cross-compiling perf.
> >
> > This is primarily intended for cross-build scenarios where the default
> > host include paths do not match the target kernel version.
> >
> > Example usage:
> > make perf ARCH="arm64" EXTRA_BPF_FLAGS="--sysroot=..."
> >
> > Change history:
> > v5:
> > - Update commit message
> > v4:
> > - Apply EXTRA_BPF_FLAGS directly to BPF skeleton build command
> > instead of modifying CLANG_OPTIONS
> > v3:
> > - Move the variable description comment to the top of Makefile.perf
> > - Update commit message
> > v2:
> > - Rename EXTRA_CLANG_FLAGS to EXTRA_BPF_FLAGS
> > - Update commit message
> > v1:
> > - Introduce EXTRA_CLANG_FLAGS to allow passing extra Clang options
> >
> > Signed-off-by: hupu <hupu.gm@gmail.com>
>
> Reviewed-by: Namhyung Kim <namhyung@kernel.org>
>
> Thanks,
> Namhyung
>
Hi Namhyung,
Sorry to bother you.
I noticed that the patch adding support for EXTRA_BPF_FLAGS, which was
previously reviewed by you, does not seem to be included in the latest
released version. I just wanted to follow up and check on the current
status of this patch, in case it was missed or delayed for some
reason.
Thanks a lot for your time and review.
Thanks
hupu
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [RFC 1/2] perf build: Allow passing extra Clang flags via EXTRA_BPF_FLAGS
2026-02-10 5:54 ` hupu
@ 2026-02-11 16:27 ` Arnaldo Carvalho de Melo
0 siblings, 0 replies; 7+ messages in thread
From: Arnaldo Carvalho de Melo @ 2026-02-11 16:27 UTC (permalink / raw)
To: hupu
Cc: Namhyung Kim, peterz, mingo, mark.rutland, alexander.shishkin,
jolsa, irogers, adrian.hunter, james.clark, nathan,
nick.desaulniers+lkml, morbo, justinstitt, linux-perf-users,
linux-kernel, llvm
On Tue, Feb 10, 2026 at 01:54:49PM +0800, hupu wrote:
> On Wed, Dec 24, 2025 at 2:20 AM Namhyung Kim <namhyung@kernel.org> wrote:
> >
> > Hello,
> >
> > On Tue, Dec 23, 2025 at 04:43:34PM +0800, hupu wrote:
> > > Add support for EXTRA_BPF_FLAGS in the eBPF skeleton build, allowing
> > > users to pass additional Clang options such as --sysroot or custom
> > > include paths when cross-compiling perf.
> > >
> > > This is primarily intended for cross-build scenarios where the default
> > > host include paths do not match the target kernel version.
> > >
> > > Example usage:
> > > make perf ARCH="arm64" EXTRA_BPF_FLAGS="--sysroot=..."
> > >
> > > Change history:
> > > v5:
> > > - Update commit message
> > > v4:
> > > - Apply EXTRA_BPF_FLAGS directly to BPF skeleton build command
> > > instead of modifying CLANG_OPTIONS
> > > v3:
> > > - Move the variable description comment to the top of Makefile.perf
> > > - Update commit message
> > > v2:
> > > - Rename EXTRA_CLANG_FLAGS to EXTRA_BPF_FLAGS
> > > - Update commit message
> > > v1:
> > > - Introduce EXTRA_CLANG_FLAGS to allow passing extra Clang options
> > >
> > > Signed-off-by: hupu <hupu.gm@gmail.com>
> >
> > Reviewed-by: Namhyung Kim <namhyung@kernel.org>
> >
>
> Hi Namhyung,
>
> Sorry to bother you.
>
> I noticed that the patch adding support for EXTRA_BPF_FLAGS, which was
> previously reviewed by you, does not seem to be included in the latest
> released version. I just wanted to follow up and check on the current
> status of this patch, in case it was missed or delayed for some
> reason.
>
> Thanks a lot for your time and review.
Applied this one,
- Arnaldo
^ permalink raw reply [flat|nested] 7+ messages in thread
* [RFC 2/2] perf build: Prefer kernel source headers for BPF skeletons
2025-12-23 8:43 [RFC 0/2] perf build: Improve header handling for BPF skeleton cross-builds hupu
2025-12-23 8:43 ` [RFC 1/2] perf build: Allow passing extra Clang flags via EXTRA_BPF_FLAGS hupu
@ 2025-12-23 8:43 ` hupu
2026-01-12 1:46 ` hupu
1 sibling, 1 reply; 7+ messages in thread
From: hupu @ 2025-12-23 8:43 UTC (permalink / raw)
To: peterz, mingo, acme, namhyung, mark.rutland, alexander.shishkin,
jolsa, irogers, adrian.hunter, james.clark, nathan,
nick.desaulniers+lkml, morbo, justinstitt, linux-perf-users,
linux-kernel, llvm, hupu.gm
When building eBPF skeletons with Clang, prefer header files provided by
the kernel source tree over headers installed on the host system.
While host-installed UAPI headers can be sufficient today, they are
tightly coupled to the host environment and may diverge from the target
kernel version over time as the kernel evolves. Relying on kernel source
headers helps ensure that the headers used during the build remain
consistent with the target kernel version and reduces dependence on the
host build environment.
This change adds the kernel source UAPI and generated header paths to
the BPF include list and places them before the system include paths,
while still keeping host headers (CLANG_SYS_INCLUDES) as a fallback.
Signed-off-by: hupu <hupu.gm@gmail.com>
---
tools/perf/Makefile.perf | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 8b8647fd4f41..57c73ab98703 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -1203,7 +1203,13 @@ endif
CLANG_OPTIONS = -Wall
CLANG_SYS_INCLUDES = $(call get_sys_includes,$(CLANG),$(CLANG_TARGET_ARCH))
-BPF_INCLUDE := -I$(SKEL_TMP_OUT)/.. -I$(LIBBPF_INCLUDE) $(CLANG_SYS_INCLUDES)
+
+KHDR_INCLUDES := -I$(abspath $(OUTPUT)../../usr/include) \
+-I$(abspath $(OUTPUT)../../arch/$(SRCARCH)/include/generated/uapi) \
+-I$(abspath $(OUTPUT)../../arch/$(SRCARCH)/include/uapi)
+
+BPF_INCLUDE := -I$(SKEL_TMP_OUT)/.. -I$(LIBBPF_INCLUDE) \
+ $(KHDR_INCLUDES) $(CLANG_SYS_INCLUDES)
TOOLS_UAPI_INCLUDE := -I$(srctree)/tools/include/uapi
ifneq ($(WERROR),0)
--
2.43.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [RFC 2/2] perf build: Prefer kernel source headers for BPF skeletons
2025-12-23 8:43 ` [RFC 2/2] perf build: Prefer kernel source headers for BPF skeletons hupu
@ 2026-01-12 1:46 ` hupu
0 siblings, 0 replies; 7+ messages in thread
From: hupu @ 2026-01-12 1:46 UTC (permalink / raw)
To: peterz, mingo, acme, namhyung, mark.rutland, alexander.shishkin,
jolsa, irogers, adrian.hunter, james.clark, nathan,
nick.desaulniers+lkml, morbo, justinstitt, linux-perf-users,
linux-kernel, llvm, hupu.gm
Hi Maintainers,
Sorry for the noise.
This patch was sent nearly three weeks ago, but I have not received
any feedback yet.
I am resending this email as a gentle reminder in case it was missed.
Thank you very much for your time and review.
Best regards,
hupu
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2026-02-11 16:27 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-12-23 8:43 [RFC 0/2] perf build: Improve header handling for BPF skeleton cross-builds hupu
2025-12-23 8:43 ` [RFC 1/2] perf build: Allow passing extra Clang flags via EXTRA_BPF_FLAGS hupu
2025-12-23 18:20 ` Namhyung Kim
2026-02-10 5:54 ` hupu
2026-02-11 16:27 ` Arnaldo Carvalho de Melo
2025-12-23 8:43 ` [RFC 2/2] perf build: Prefer kernel source headers for BPF skeletons hupu
2026-01-12 1:46 ` hupu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox