[PATCH v1] perf build: Add system include paths to BPF builds

[PATCH v1] perf build: Add system include paths to BPF builds

[Ian Rogers]

There are insufficient headers in tools/include to satisfy building
BPF programs and their header dependencies. Add the system include
paths from the non-BPF clang compile so that these headers can be
found.

This code was taken from:
tools/testing/selftests/bpf/Makefile

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/Makefile.perf | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 61c33d100b2b..37befdfa8ac8 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -1057,7 +1057,25 @@ $(SKEL_TMP_OUT) $(LIBAPI_OUTPUT) $(LIBBPF_OUTPUT) $(LIBPERF_OUTPUT) $(LIBSUBCMD_
 
 ifndef NO_BPF_SKEL
 BPFTOOL := $(SKEL_TMP_OUT)/bootstrap/bpftool
-BPF_INCLUDE := -I$(SKEL_TMP_OUT)/.. -I$(LIBBPF_INCLUDE)
+
+# Get Clang's default includes on this system, as opposed to those seen by
+# '-target bpf'. This fixes "missing" files on some architectures/distros,
+# such as asm/byteorder.h, asm/socket.h, asm/sockios.h, sys/cdefs.h etc.
+#
+# Use '-idirafter': Don't interfere with include mechanics except where the
+# build would have failed anyways.
+define get_sys_includes
+$(shell $(1) $(2) -v -E - </dev/null 2>&1 \
+	| sed -n '/<...> search starts here:/,/End of search list./{ s| \(/.*\)|-idirafter \1|p }') \
+$(shell $(1) $(2) -dM -E - </dev/null | grep '__riscv_xlen ' | awk '{printf("-D__riscv_xlen=%d -D__BITS_PER_LONG=%d", $$3, $$3)}')
+endef
+
+ifneq ($(CROSS_COMPILE),)
+CLANG_TARGET_ARCH = --target=$(notdir $(CROSS_COMPILE:%-=%))
+endif
+
+CLANG_SYS_INCLUDES = $(call get_sys_includes,$(CLANG),$(CLANG_TARGET_ARCH))
+BPF_INCLUDE := -I$(SKEL_TMP_OUT)/.. -I$(LIBBPF_INCLUDE) $(CLANG_SYS_INCLUDES)
 
 $(BPFTOOL): | $(SKEL_TMP_OUT)
 	$(Q)CFLAGS= $(MAKE) -C ../bpf/bpftool \
-- 
2.40.1.521.gf1e218fcd8-goog

Re: [PATCH v1] perf build: Add system include paths to BPF builds

[Namhyung Kim]

Hi Ian,

On Fri, May 5, 2023 at 7:15 PM Ian Rogers <irogers@google.com> wrote:
>
> There are insufficient headers in tools/include to satisfy building
> BPF programs and their header dependencies. Add the system include
> paths from the non-BPF clang compile so that these headers can be
> found.
>
> This code was taken from:
> tools/testing/selftests/bpf/Makefile
>
> Signed-off-by: Ian Rogers <irogers@google.com>

Thanks, this fixes the BPF build failure on my machine.

Tested-by: Namhyung Kim <namhyung@kernel.org>

Thanks,
Namhyung


> ---
>  tools/perf/Makefile.perf | 20 +++++++++++++++++++-
>  1 file changed, 19 insertions(+), 1 deletion(-)
>
> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> index 61c33d100b2b..37befdfa8ac8 100644
> --- a/tools/perf/Makefile.perf
> +++ b/tools/perf/Makefile.perf
> @@ -1057,7 +1057,25 @@ $(SKEL_TMP_OUT) $(LIBAPI_OUTPUT) $(LIBBPF_OUTPUT) $(LIBPERF_OUTPUT) $(LIBSUBCMD_
>
>  ifndef NO_BPF_SKEL
>  BPFTOOL := $(SKEL_TMP_OUT)/bootstrap/bpftool
> -BPF_INCLUDE := -I$(SKEL_TMP_OUT)/.. -I$(LIBBPF_INCLUDE)
> +
> +# Get Clang's default includes on this system, as opposed to those seen by
> +# '-target bpf'. This fixes "missing" files on some architectures/distros,
> +# such as asm/byteorder.h, asm/socket.h, asm/sockios.h, sys/cdefs.h etc.
> +#
> +# Use '-idirafter': Don't interfere with include mechanics except where the
> +# build would have failed anyways.
> +define get_sys_includes
> +$(shell $(1) $(2) -v -E - </dev/null 2>&1 \
> +       | sed -n '/<...> search starts here:/,/End of search list./{ s| \(/.*\)|-idirafter \1|p }') \
> +$(shell $(1) $(2) -dM -E - </dev/null | grep '__riscv_xlen ' | awk '{printf("-D__riscv_xlen=%d -D__BITS_PER_LONG=%d", $$3, $$3)}')
> +endef
> +
> +ifneq ($(CROSS_COMPILE),)
> +CLANG_TARGET_ARCH = --target=$(notdir $(CROSS_COMPILE:%-=%))
> +endif
> +
> +CLANG_SYS_INCLUDES = $(call get_sys_includes,$(CLANG),$(CLANG_TARGET_ARCH))
> +BPF_INCLUDE := -I$(SKEL_TMP_OUT)/.. -I$(LIBBPF_INCLUDE) $(CLANG_SYS_INCLUDES)
>
>  $(BPFTOOL): | $(SKEL_TMP_OUT)
>         $(Q)CFLAGS= $(MAKE) -C ../bpf/bpftool \
> --
> 2.40.1.521.gf1e218fcd8-goog
>

Re: [PATCH v1] perf build: Add system include paths to BPF builds

[Arnaldo Carvalho de Melo]

Em Fri, May 05, 2023 at 07:14:50PM -0700, Ian Rogers escreveu:
> There are insufficient headers in tools/include to satisfy building
> BPF programs and their header dependencies. Add the system include
> paths from the non-BPF clang compile so that these headers can be
> found.
> 
> This code was taken from:
> tools/testing/selftests/bpf/Makefile
> 
> Signed-off-by: Ian Rogers <irogers@google.com>
> ---
>  tools/perf/Makefile.perf | 20 +++++++++++++++++++-
>  1 file changed, 19 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> index 61c33d100b2b..37befdfa8ac8 100644
> --- a/tools/perf/Makefile.perf
> +++ b/tools/perf/Makefile.perf
> @@ -1057,7 +1057,25 @@ $(SKEL_TMP_OUT) $(LIBAPI_OUTPUT) $(LIBBPF_OUTPUT) $(LIBPERF_OUTPUT) $(LIBSUBCMD_
>  
>  ifndef NO_BPF_SKEL

So this patch was done before the reverts, I adjusted it to what is
upstream and to another patch that makes the build use the headers from
the perf sources instead of the system's (linux/bpf.h and
linux/perf_event.h, from vmlinux.h), please take a look at the patch
below, I'm also trying to figure out that other problem you pointed with
linux/types.s :-\

What I have now in tmp.perf-tools:

⬢[acme@toolbox perf-tools]$ git log --oneline torvalds/master..
a2af0f6b8ef7ea40 (HEAD -> perf-tools) perf build: Add system include paths to BPF builds
5be6cecda0802f23 perf bpf skels: Make vmlinux.h use bpf.h and perf_event.h in source directory
7d161165d9072dcb perf parse-events: Do not break up AUX event group
a468085011ea8bba perf test test_intel_pt.sh: Test sample mode with event with PMU name
123361659fa405de perf evsel: Modify group pmu name for software events
34e82891d995ab89 tools arch x86: Sync the msr-index.h copy with the kernel sources
705049ca4f5b7b00 tools headers kvm: Sync uapi/{asm/linux} kvm.h headers with the kernel sources
8d6a41c8065e1120 tools include UAPI: Sync the sound/asound.h copy with the kernel sources
92b8e61e88351091 tools headers UAPI: Sync the linux/const.h with the kernel headers
e7ec3a249c38a9c9 tools headers UAPI: Sync the i915_drm.h with the kernel sources
e6232180e524e112 tools headers UAPI: Sync the drm/drm.h with the kernel sources
5d1ac59ff7445e51 tools headers UAPI: Sync the linux/in.h with the kernel sources
b0618f38e2ab8ce3 perf build: Gracefully fail the build if BUILD_BPF_SKEL=1 is specified and clang isn't available
5f0b89e632ed81b6 perf test java symbol: Remove needless debuginfod queries
327daf34554d20a6 perf parse-events: Don't reorder ungrouped events by PMU
ccc66c6092802d68 perf metric: JSON flag to not group events if gathering a metric group
1b114824106ca468 perf stat: Introduce skippable evsels
2a939c8695035b11 perf metric: Change divide by zero and !support events behavior
⬢[acme@toolbox perf-tools]$

Please help me test this,

Regards,

- Arnaldo

Re: [PATCH v1] perf build: Add system include paths to BPF builds

[Ian Rogers]

On Wed, May 10, 2023 at 10:35 AM Arnaldo Carvalho de Melo
<acme@kernel.org> wrote:
>
> Em Fri, May 05, 2023 at 07:14:50PM -0700, Ian Rogers escreveu:
> > There are insufficient headers in tools/include to satisfy building
> > BPF programs and their header dependencies. Add the system include
> > paths from the non-BPF clang compile so that these headers can be
> > found.
> >
> > This code was taken from:
> > tools/testing/selftests/bpf/Makefile
> >
> > Signed-off-by: Ian Rogers <irogers@google.com>
> > ---
> >  tools/perf/Makefile.perf | 20 +++++++++++++++++++-
> >  1 file changed, 19 insertions(+), 1 deletion(-)
> >
> > diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> > index 61c33d100b2b..37befdfa8ac8 100644
> > --- a/tools/perf/Makefile.perf
> > +++ b/tools/perf/Makefile.perf
> > @@ -1057,7 +1057,25 @@ $(SKEL_TMP_OUT) $(LIBAPI_OUTPUT) $(LIBBPF_OUTPUT) $(LIBPERF_OUTPUT) $(LIBSUBCMD_
> >
> >  ifndef NO_BPF_SKEL
>
> So this patch was done before the reverts, I adjusted it to what is
> upstream and to another patch that makes the build use the headers from
> the perf sources instead of the system's (linux/bpf.h and
> linux/perf_event.h, from vmlinux.h), please take a look at the patch
> below, I'm also trying to figure out that other problem you pointed with
> linux/types.s :-\
>
> What I have now in tmp.perf-tools:
>
> ⬢[acme@toolbox perf-tools]$ git log --oneline torvalds/master..
> a2af0f6b8ef7ea40 (HEAD -> perf-tools) perf build: Add system include paths to BPF builds
> 5be6cecda0802f23 perf bpf skels: Make vmlinux.h use bpf.h and perf_event.h in source directory
> 7d161165d9072dcb perf parse-events: Do not break up AUX event group
> a468085011ea8bba perf test test_intel_pt.sh: Test sample mode with event with PMU name
> 123361659fa405de perf evsel: Modify group pmu name for software events
> 34e82891d995ab89 tools arch x86: Sync the msr-index.h copy with the kernel sources
> 705049ca4f5b7b00 tools headers kvm: Sync uapi/{asm/linux} kvm.h headers with the kernel sources
> 8d6a41c8065e1120 tools include UAPI: Sync the sound/asound.h copy with the kernel sources
> 92b8e61e88351091 tools headers UAPI: Sync the linux/const.h with the kernel headers
> e7ec3a249c38a9c9 tools headers UAPI: Sync the i915_drm.h with the kernel sources
> e6232180e524e112 tools headers UAPI: Sync the drm/drm.h with the kernel sources
> 5d1ac59ff7445e51 tools headers UAPI: Sync the linux/in.h with the kernel sources
> b0618f38e2ab8ce3 perf build: Gracefully fail the build if BUILD_BPF_SKEL=1 is specified and clang isn't available
> 5f0b89e632ed81b6 perf test java symbol: Remove needless debuginfod queries
> 327daf34554d20a6 perf parse-events: Don't reorder ungrouped events by PMU
> ccc66c6092802d68 perf metric: JSON flag to not group events if gathering a metric group
> 1b114824106ca468 perf stat: Introduce skippable evsels
> 2a939c8695035b11 perf metric: Change divide by zero and !support events behavior
> ⬢[acme@toolbox perf-tools]$
>
> Please help me test this,

build-test and compiling with/without BPF skeletons looked okay in
perf test on my Debian derived distro.

Thanks,
Ian

> Regards,
>
> - Arnaldo
>
>