* [PATCH bpf-next v3 1/3] selftests/bpf: Allow building with extra flags
2024-11-01 8:27 [PATCH bpf-next v3 0/3] selftests/bpf: Improve building with extra Viktor Malik
@ 2024-11-01 8:27 ` Viktor Malik
2024-11-01 19:34 ` Andrii Nakryiko
2024-11-04 14:03 ` Toke Høiland-Jørgensen
2024-11-01 8:27 ` [PATCH bpf-next v3 2/3] bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile Viktor Malik
` (4 subsequent siblings)
5 siblings, 2 replies; 13+ messages in thread
From: Viktor Malik @ 2024-11-01 8:27 UTC (permalink / raw)
To: bpf
Cc: Andrii Nakryiko, Eduard Zingerman, Mykola Lysenko,
Alexei Starovoitov, Daniel Borkmann, Martin KaFai Lau, Song Liu,
Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev,
Hao Luo, Jiri Olsa, Shuah Khan, Nathan Chancellor,
Nick Desaulniers, Bill Wendling, Justin Stitt, Viktor Malik
In order to specify extra compilation or linking flags to BPF selftests,
it is possible to set EXTRA_CFLAGS and EXTRA_LDFLAGS from the command
line. The problem is that they are not propagated to sub-make calls
(runqslower, bpftool, libbpf) and in the better case are not applied, in
the worse case cause the entire build fail.
Propagate EXTRA_CFLAGS and EXTRA_LDFLAGS to the sub-makes.
This, for instance, allows to build selftests as PIE with
$ make EXTRA_CFLAGS='-fPIE' EXTRA_LDFLAGS='-pie'
Without this change, the command would fail because libbpf.a would not
be built with -fPIE and other PIE binaries would not link against it.
The only problem is that we have to explicitly provide empty
EXTRA_CFLAGS='' and EXTRA_LDFLAGS='' to the builds of kernel modules as
we don't want to build modules with flags used for userspace (the above
example would fail as kernel doesn't support PIE).
Signed-off-by: Viktor Malik <vmalik@redhat.com>
---
tools/testing/selftests/bpf/Makefile | 34 +++++++++++++++++++---------
1 file changed, 23 insertions(+), 11 deletions(-)
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index a226d0647c4e..3b43d7db8d2c 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -295,25 +295,33 @@ $(OUTPUT)/sign-file: ../../../../scripts/sign-file.c
$(OUTPUT)/bpf_testmod.ko: $(VMLINUX_BTF) $(RESOLVE_BTFIDS) $(wildcard bpf_testmod/Makefile bpf_testmod/*.[ch])
$(call msg,MOD,,$@)
$(Q)$(RM) bpf_testmod/bpf_testmod.ko # force re-compilation
- $(Q)$(MAKE) $(submake_extras) RESOLVE_BTFIDS=$(RESOLVE_BTFIDS) -C bpf_testmod
+ $(Q)$(MAKE) $(submake_extras) -C bpf_testmod \
+ RESOLVE_BTFIDS=$(RESOLVE_BTFIDS) \
+ EXTRA_CFLAGS='' EXTRA_LDFLAGS=''
$(Q)cp bpf_testmod/bpf_testmod.ko $@
$(OUTPUT)/bpf_test_no_cfi.ko: $(VMLINUX_BTF) $(RESOLVE_BTFIDS) $(wildcard bpf_test_no_cfi/Makefile bpf_test_no_cfi/*.[ch])
$(call msg,MOD,,$@)
$(Q)$(RM) bpf_test_no_cfi/bpf_test_no_cfi.ko # force re-compilation
- $(Q)$(MAKE) $(submake_extras) RESOLVE_BTFIDS=$(RESOLVE_BTFIDS) -C bpf_test_no_cfi
+ $(Q)$(MAKE) $(submake_extras) -C bpf_test_no_cfi \
+ RESOLVE_BTFIDS=$(RESOLVE_BTFIDS) \
+ EXTRA_CFLAGS='' EXTRA_LDFLAGS=''
$(Q)cp bpf_test_no_cfi/bpf_test_no_cfi.ko $@
$(OUTPUT)/bpf_test_modorder_x.ko: $(VMLINUX_BTF) $(RESOLVE_BTFIDS) $(wildcard bpf_test_modorder_x/Makefile bpf_test_modorder_x/*.[ch])
$(call msg,MOD,,$@)
$(Q)$(RM) bpf_test_modorder_x/bpf_test_modorder_x.ko # force re-compilation
- $(Q)$(MAKE) $(submake_extras) RESOLVE_BTFIDS=$(RESOLVE_BTFIDS) -C bpf_test_modorder_x
+ $(Q)$(MAKE) $(submake_extras) -C bpf_test_modorder_x \
+ RESOLVE_BTFIDS=$(RESOLVE_BTFIDS) \
+ EXTRA_CFLAGS='' EXTRA_LDFLAGS=''
$(Q)cp bpf_test_modorder_x/bpf_test_modorder_x.ko $@
$(OUTPUT)/bpf_test_modorder_y.ko: $(VMLINUX_BTF) $(RESOLVE_BTFIDS) $(wildcard bpf_test_modorder_y/Makefile bpf_test_modorder_y/*.[ch])
$(call msg,MOD,,$@)
$(Q)$(RM) bpf_test_modorder_y/bpf_test_modorder_y.ko # force re-compilation
- $(Q)$(MAKE) $(submake_extras) RESOLVE_BTFIDS=$(RESOLVE_BTFIDS) -C bpf_test_modorder_y
+ $(Q)$(MAKE) $(submake_extras) -C bpf_test_modorder_y \
+ RESOLVE_BTFIDS=$(RESOLVE_BTFIDS) \
+ EXTRA_CFLAGS='' EXTRA_LDFLAGS=''
$(Q)cp bpf_test_modorder_y/bpf_test_modorder_y.ko $@
@@ -333,8 +341,8 @@ $(OUTPUT)/runqslower: $(BPFOBJ) | $(DEFAULT_BPFTOOL) $(RUNQSLOWER_OUTPUT)
BPFTOOL_OUTPUT=$(HOST_BUILD_DIR)/bpftool/ \
BPFOBJ_OUTPUT=$(BUILD_DIR)/libbpf/ \
BPFOBJ=$(BPFOBJ) BPF_INCLUDE=$(INCLUDE_DIR) \
- EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS)' \
- EXTRA_LDFLAGS='$(SAN_LDFLAGS)' && \
+ EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS) $(EXTRA_CFLAGS)' \
+ EXTRA_LDFLAGS='$(SAN_LDFLAGS) $(EXTRA_LDFLAGS)' && \
cp $(RUNQSLOWER_OUTPUT)runqslower $@
TEST_GEN_PROGS_EXTENDED += $(TRUNNER_BPFTOOL)
@@ -367,7 +375,8 @@ $(DEFAULT_BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile) \
$(HOST_BPFOBJ) | $(HOST_BUILD_DIR)/bpftool
$(Q)$(MAKE) $(submake_extras) -C $(BPFTOOLDIR) \
ARCH= CROSS_COMPILE= CC="$(HOSTCC)" LD="$(HOSTLD)" \
- EXTRA_CFLAGS='-g $(OPT_FLAGS)' \
+ EXTRA_CFLAGS='-g $(OPT_FLAGS) $(EXTRA_CFLAGS)' \
+ EXTRA_LDFLAGS='$(EXTRA_LDFLAGS)' \
OUTPUT=$(HOST_BUILD_DIR)/bpftool/ \
LIBBPF_OUTPUT=$(HOST_BUILD_DIR)/libbpf/ \
LIBBPF_DESTDIR=$(HOST_SCRATCH_DIR)/ \
@@ -378,7 +387,8 @@ $(CROSS_BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile) \
$(BPFOBJ) | $(BUILD_DIR)/bpftool
$(Q)$(MAKE) $(submake_extras) -C $(BPFTOOLDIR) \
ARCH=$(ARCH) CROSS_COMPILE=$(CROSS_COMPILE) \
- EXTRA_CFLAGS='-g $(OPT_FLAGS)' \
+ EXTRA_CFLAGS='-g $(OPT_FLAGS) $(EXTRA_CFLAGS)' \
+ EXTRA_LDFLAGS='$(EXTRA_LDFLAGS)' \
OUTPUT=$(BUILD_DIR)/bpftool/ \
LIBBPF_OUTPUT=$(BUILD_DIR)/libbpf/ \
LIBBPF_DESTDIR=$(SCRATCH_DIR)/ \
@@ -401,8 +411,8 @@ $(BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile) \
$(APIDIR)/linux/bpf.h \
| $(BUILD_DIR)/libbpf
$(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) OUTPUT=$(BUILD_DIR)/libbpf/ \
- EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS)' \
- EXTRA_LDFLAGS='$(SAN_LDFLAGS)' \
+ EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS) $(EXTRA_CFLAGS)' \
+ EXTRA_LDFLAGS='$(SAN_LDFLAGS) $(EXTRA_LDFLAGS)' \
DESTDIR=$(SCRATCH_DIR) prefix= all install_headers
ifneq ($(BPFOBJ),$(HOST_BPFOBJ))
@@ -410,7 +420,9 @@ $(HOST_BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile) \
$(APIDIR)/linux/bpf.h \
| $(HOST_BUILD_DIR)/libbpf
$(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) \
- EXTRA_CFLAGS='-g $(OPT_FLAGS)' ARCH= CROSS_COMPILE= \
+ ARCH= CROSS_COMPILE= \
+ EXTRA_CFLAGS='-g $(OPT_FLAGS) $(EXTRA_CFLAGS)' \
+ EXTRA_LDFLAGS='$(EXTRA_LDFLAGS)' \
OUTPUT=$(HOST_BUILD_DIR)/libbpf/ \
CC="$(HOSTCC)" LD="$(HOSTLD)" \
DESTDIR=$(HOST_SCRATCH_DIR)/ prefix= all install_headers
--
2.47.0
^ permalink raw reply related [flat|nested] 13+ messages in thread* Re: [PATCH bpf-next v3 1/3] selftests/bpf: Allow building with extra flags
2024-11-01 8:27 ` [PATCH bpf-next v3 1/3] selftests/bpf: Allow building with extra flags Viktor Malik
@ 2024-11-01 19:34 ` Andrii Nakryiko
2024-11-04 13:43 ` Toke Høiland-Jørgensen
2024-11-04 14:03 ` Toke Høiland-Jørgensen
1 sibling, 1 reply; 13+ messages in thread
From: Andrii Nakryiko @ 2024-11-01 19:34 UTC (permalink / raw)
To: Viktor Malik, Toke Høiland-Jørgensen
Cc: bpf, Andrii Nakryiko, Eduard Zingerman, Mykola Lysenko,
Alexei Starovoitov, Daniel Borkmann, Martin KaFai Lau, Song Liu,
Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev,
Hao Luo, Jiri Olsa, Shuah Khan, Nathan Chancellor,
Nick Desaulniers, Bill Wendling, Justin Stitt
On Fri, Nov 1, 2024 at 1:29 AM Viktor Malik <vmalik@redhat.com> wrote:
>
> In order to specify extra compilation or linking flags to BPF selftests,
> it is possible to set EXTRA_CFLAGS and EXTRA_LDFLAGS from the command
> line. The problem is that they are not propagated to sub-make calls
> (runqslower, bpftool, libbpf) and in the better case are not applied, in
> the worse case cause the entire build fail.
>
> Propagate EXTRA_CFLAGS and EXTRA_LDFLAGS to the sub-makes.
>
> This, for instance, allows to build selftests as PIE with
>
> $ make EXTRA_CFLAGS='-fPIE' EXTRA_LDFLAGS='-pie'
>
> Without this change, the command would fail because libbpf.a would not
> be built with -fPIE and other PIE binaries would not link against it.
>
> The only problem is that we have to explicitly provide empty
> EXTRA_CFLAGS='' and EXTRA_LDFLAGS='' to the builds of kernel modules as
> we don't want to build modules with flags used for userspace (the above
> example would fail as kernel doesn't support PIE).
>
> Signed-off-by: Viktor Malik <vmalik@redhat.com>
> ---
> tools/testing/selftests/bpf/Makefile | 34 +++++++++++++++++++---------
> 1 file changed, 23 insertions(+), 11 deletions(-)
>
Ok, so this will conflict with Toke's [0]. Who should go first? :)
And given you guys touch these more obscure parts of BPF selftests
Makefile, I'd really appreciate it if you can help reviewing them for
each other :)
[0] https://patchwork.kernel.org/project/netdevbpf/patch/20241031-bpf-selftests-mod-compile-v1-1-1a63af2385f1@redhat.com/
[...]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH bpf-next v3 1/3] selftests/bpf: Allow building with extra flags
2024-11-01 19:34 ` Andrii Nakryiko
@ 2024-11-04 13:43 ` Toke Høiland-Jørgensen
2024-11-06 20:37 ` Andrii Nakryiko
0 siblings, 1 reply; 13+ messages in thread
From: Toke Høiland-Jørgensen @ 2024-11-04 13:43 UTC (permalink / raw)
To: Andrii Nakryiko, Viktor Malik
Cc: bpf, Andrii Nakryiko, Eduard Zingerman, Mykola Lysenko,
Alexei Starovoitov, Daniel Borkmann, Martin KaFai Lau, Song Liu,
Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev,
Hao Luo, Jiri Olsa, Shuah Khan, Nathan Chancellor,
Nick Desaulniers, Bill Wendling, Justin Stitt
Andrii Nakryiko <andrii.nakryiko@gmail.com> writes:
> On Fri, Nov 1, 2024 at 1:29 AM Viktor Malik <vmalik@redhat.com> wrote:
>>
>> In order to specify extra compilation or linking flags to BPF selftests,
>> it is possible to set EXTRA_CFLAGS and EXTRA_LDFLAGS from the command
>> line. The problem is that they are not propagated to sub-make calls
>> (runqslower, bpftool, libbpf) and in the better case are not applied, in
>> the worse case cause the entire build fail.
>>
>> Propagate EXTRA_CFLAGS and EXTRA_LDFLAGS to the sub-makes.
>>
>> This, for instance, allows to build selftests as PIE with
>>
>> $ make EXTRA_CFLAGS='-fPIE' EXTRA_LDFLAGS='-pie'
>>
>> Without this change, the command would fail because libbpf.a would not
>> be built with -fPIE and other PIE binaries would not link against it.
>>
>> The only problem is that we have to explicitly provide empty
>> EXTRA_CFLAGS='' and EXTRA_LDFLAGS='' to the builds of kernel modules as
>> we don't want to build modules with flags used for userspace (the above
>> example would fail as kernel doesn't support PIE).
>>
>> Signed-off-by: Viktor Malik <vmalik@redhat.com>
>> ---
>> tools/testing/selftests/bpf/Makefile | 34 +++++++++++++++++++---------
>> 1 file changed, 23 insertions(+), 11 deletions(-)
>>
>
> Ok, so this will conflict with Toke's [0]. Who should go first? :)
I'm OK with rebasing on top of Viktor's patch :)
>
> And given you guys touch these more obscure parts of BPF selftests
> Makefile, I'd really appreciate it if you can help reviewing them for
> each other :)
Sure, can do!
-Toke
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH bpf-next v3 1/3] selftests/bpf: Allow building with extra flags
2024-11-04 13:43 ` Toke Høiland-Jørgensen
@ 2024-11-06 20:37 ` Andrii Nakryiko
0 siblings, 0 replies; 13+ messages in thread
From: Andrii Nakryiko @ 2024-11-06 20:37 UTC (permalink / raw)
To: Toke Høiland-Jørgensen
Cc: Viktor Malik, bpf, Andrii Nakryiko, Eduard Zingerman,
Mykola Lysenko, Alexei Starovoitov, Daniel Borkmann,
Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, Shuah Khan,
Nathan Chancellor, Nick Desaulniers, Bill Wendling, Justin Stitt
On Mon, Nov 4, 2024 at 5:43 AM Toke Høiland-Jørgensen <toke@redhat.com> wrote:
>
> Andrii Nakryiko <andrii.nakryiko@gmail.com> writes:
>
> > On Fri, Nov 1, 2024 at 1:29 AM Viktor Malik <vmalik@redhat.com> wrote:
> >>
> >> In order to specify extra compilation or linking flags to BPF selftests,
> >> it is possible to set EXTRA_CFLAGS and EXTRA_LDFLAGS from the command
> >> line. The problem is that they are not propagated to sub-make calls
> >> (runqslower, bpftool, libbpf) and in the better case are not applied, in
> >> the worse case cause the entire build fail.
> >>
> >> Propagate EXTRA_CFLAGS and EXTRA_LDFLAGS to the sub-makes.
> >>
> >> This, for instance, allows to build selftests as PIE with
> >>
> >> $ make EXTRA_CFLAGS='-fPIE' EXTRA_LDFLAGS='-pie'
> >>
> >> Without this change, the command would fail because libbpf.a would not
> >> be built with -fPIE and other PIE binaries would not link against it.
> >>
> >> The only problem is that we have to explicitly provide empty
> >> EXTRA_CFLAGS='' and EXTRA_LDFLAGS='' to the builds of kernel modules as
> >> we don't want to build modules with flags used for userspace (the above
> >> example would fail as kernel doesn't support PIE).
> >>
> >> Signed-off-by: Viktor Malik <vmalik@redhat.com>
> >> ---
> >> tools/testing/selftests/bpf/Makefile | 34 +++++++++++++++++++---------
> >> 1 file changed, 23 insertions(+), 11 deletions(-)
> >>
> >
> > Ok, so this will conflict with Toke's [0]. Who should go first? :)
>
> I'm OK with rebasing on top of Viktor's patch :)
Ok, then. Thanks for the review! I've applied this patch to
bpf-next/master as well. Please rebase and resend your change.
> >
> > And given you guys touch these more obscure parts of BPF selftests
> > Makefile, I'd really appreciate it if you can help reviewing them for
> > each other :)
>
> Sure, can do!
>
> -Toke
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH bpf-next v3 1/3] selftests/bpf: Allow building with extra flags
2024-11-01 8:27 ` [PATCH bpf-next v3 1/3] selftests/bpf: Allow building with extra flags Viktor Malik
2024-11-01 19:34 ` Andrii Nakryiko
@ 2024-11-04 14:03 ` Toke Høiland-Jørgensen
1 sibling, 0 replies; 13+ messages in thread
From: Toke Høiland-Jørgensen @ 2024-11-04 14:03 UTC (permalink / raw)
To: Viktor Malik, bpf
Cc: Andrii Nakryiko, Eduard Zingerman, Mykola Lysenko,
Alexei Starovoitov, Daniel Borkmann, Martin KaFai Lau, Song Liu,
Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev,
Hao Luo, Jiri Olsa, Shuah Khan, Nathan Chancellor,
Nick Desaulniers, Bill Wendling, Justin Stitt, Viktor Malik
Viktor Malik <vmalik@redhat.com> writes:
> In order to specify extra compilation or linking flags to BPF selftests,
> it is possible to set EXTRA_CFLAGS and EXTRA_LDFLAGS from the command
> line. The problem is that they are not propagated to sub-make calls
> (runqslower, bpftool, libbpf) and in the better case are not applied, in
> the worse case cause the entire build fail.
>
> Propagate EXTRA_CFLAGS and EXTRA_LDFLAGS to the sub-makes.
>
> This, for instance, allows to build selftests as PIE with
>
> $ make EXTRA_CFLAGS='-fPIE' EXTRA_LDFLAGS='-pie'
>
> Without this change, the command would fail because libbpf.a would not
> be built with -fPIE and other PIE binaries would not link against it.
>
> The only problem is that we have to explicitly provide empty
> EXTRA_CFLAGS='' and EXTRA_LDFLAGS='' to the builds of kernel modules as
> we don't want to build modules with flags used for userspace (the above
> example would fail as kernel doesn't support PIE).
>
> Signed-off-by: Viktor Malik <vmalik@redhat.com>
That last bit is a bit ugly, but I couldn't think of a better way of
doing it, so with that:
Reviewed-by: Toke Høiland-Jørgensen <toke@redhat.com>
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH bpf-next v3 2/3] bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile
2024-11-01 8:27 [PATCH bpf-next v3 0/3] selftests/bpf: Improve building with extra Viktor Malik
2024-11-01 8:27 ` [PATCH bpf-next v3 1/3] selftests/bpf: Allow building with extra flags Viktor Malik
@ 2024-11-01 8:27 ` Viktor Malik
2024-11-04 14:05 ` Toke Høiland-Jørgensen
2024-11-01 8:27 ` [PATCH bpf-next v3 3/3] selftests/bpf: Disable warnings on unused flags for Clang builds Viktor Malik
` (3 subsequent siblings)
5 siblings, 1 reply; 13+ messages in thread
From: Viktor Malik @ 2024-11-01 8:27 UTC (permalink / raw)
To: bpf
Cc: Andrii Nakryiko, Eduard Zingerman, Mykola Lysenko,
Alexei Starovoitov, Daniel Borkmann, Martin KaFai Lau, Song Liu,
Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev,
Hao Luo, Jiri Olsa, Shuah Khan, Nathan Chancellor,
Nick Desaulniers, Bill Wendling, Justin Stitt, Viktor Malik,
Quentin Monnet
When building selftests with CFLAGS set via env variable, the value of
CFLAGS is propagated into bpftool Makefile (called from selftests
Makefile). This makes the compilation fail as _GNU_SOURCE is defined two
times - once from selftests Makefile (by including lib.mk) and once from
bpftool Makefile (by calling `llvm-config --cflags`):
$ CFLAGS="" make -C tools/testing/selftests/bpf
[...]
CC /bpf-next/tools/testing/selftests/bpf/tools/build/bpftool/btf.o
<command-line>: error: "_GNU_SOURCE" redefined [-Werror]
<command-line>: note: this is the location of the previous definition
cc1: all warnings being treated as errors
[...]
Filter out -D_GNU_SOURCE from the result of `llvm-config --cflags` in
bpftool Makefile to prevent this error.
Signed-off-by: Viktor Malik <vmalik@redhat.com>
Acked-by: Quentin Monnet <qmo@kernel.org>
Acked-by: Jiri Olsa <jolsa@kernel.org>
---
tools/bpf/bpftool/Makefile | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
index ba927379eb20..a4263dfb5e03 100644
--- a/tools/bpf/bpftool/Makefile
+++ b/tools/bpf/bpftool/Makefile
@@ -147,7 +147,11 @@ ifeq ($(feature-llvm),1)
# If LLVM is available, use it for JIT disassembly
CFLAGS += -DHAVE_LLVM_SUPPORT
LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets
- CFLAGS += $(shell $(LLVM_CONFIG) --cflags)
+ # llvm-config always adds -D_GNU_SOURCE, however, it may already be in CFLAGS
+ # (e.g. when bpftool build is called from selftests build as selftests
+ # Makefile includes lib.mk which sets -D_GNU_SOURCE) which would cause
+ # compilation error due to redefinition. Let's filter it out here.
+ CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))
LIBS += $(shell $(LLVM_CONFIG) --libs $(LLVM_CONFIG_LIB_COMPONENTS))
ifeq ($(shell $(LLVM_CONFIG) --shared-mode),static)
LIBS += $(shell $(LLVM_CONFIG) --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
--
2.47.0
^ permalink raw reply related [flat|nested] 13+ messages in thread* Re: [PATCH bpf-next v3 2/3] bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile
2024-11-01 8:27 ` [PATCH bpf-next v3 2/3] bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile Viktor Malik
@ 2024-11-04 14:05 ` Toke Høiland-Jørgensen
0 siblings, 0 replies; 13+ messages in thread
From: Toke Høiland-Jørgensen @ 2024-11-04 14:05 UTC (permalink / raw)
To: Viktor Malik, bpf
Cc: Andrii Nakryiko, Eduard Zingerman, Mykola Lysenko,
Alexei Starovoitov, Daniel Borkmann, Martin KaFai Lau, Song Liu,
Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev,
Hao Luo, Jiri Olsa, Shuah Khan, Nathan Chancellor,
Nick Desaulniers, Bill Wendling, Justin Stitt, Viktor Malik,
Quentin Monnet
Viktor Malik <vmalik@redhat.com> writes:
> When building selftests with CFLAGS set via env variable, the value of
> CFLAGS is propagated into bpftool Makefile (called from selftests
> Makefile). This makes the compilation fail as _GNU_SOURCE is defined two
> times - once from selftests Makefile (by including lib.mk) and once from
> bpftool Makefile (by calling `llvm-config --cflags`):
>
> $ CFLAGS="" make -C tools/testing/selftests/bpf
> [...]
> CC /bpf-next/tools/testing/selftests/bpf/tools/build/bpftool/btf.o
> <command-line>: error: "_GNU_SOURCE" redefined [-Werror]
> <command-line>: note: this is the location of the previous definition
> cc1: all warnings being treated as errors
> [...]
>
> Filter out -D_GNU_SOURCE from the result of `llvm-config --cflags` in
> bpftool Makefile to prevent this error.
>
> Signed-off-by: Viktor Malik <vmalik@redhat.com>
> Acked-by: Quentin Monnet <qmo@kernel.org>
> Acked-by: Jiri Olsa <jolsa@kernel.org>
Reviewed-by: Toke Høiland-Jørgensen <toke@redhat.com>
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH bpf-next v3 3/3] selftests/bpf: Disable warnings on unused flags for Clang builds
2024-11-01 8:27 [PATCH bpf-next v3 0/3] selftests/bpf: Improve building with extra Viktor Malik
2024-11-01 8:27 ` [PATCH bpf-next v3 1/3] selftests/bpf: Allow building with extra flags Viktor Malik
2024-11-01 8:27 ` [PATCH bpf-next v3 2/3] bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile Viktor Malik
@ 2024-11-01 8:27 ` Viktor Malik
2024-11-01 19:46 ` [PATCH bpf-next v3 0/3] selftests/bpf: Improve building with extra Andrii Nakryiko
` (2 subsequent siblings)
5 siblings, 0 replies; 13+ messages in thread
From: Viktor Malik @ 2024-11-01 8:27 UTC (permalink / raw)
To: bpf
Cc: Andrii Nakryiko, Eduard Zingerman, Mykola Lysenko,
Alexei Starovoitov, Daniel Borkmann, Martin KaFai Lau, Song Liu,
Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev,
Hao Luo, Jiri Olsa, Shuah Khan, Nathan Chancellor,
Nick Desaulniers, Bill Wendling, Justin Stitt, Viktor Malik
There exist compiler flags supported by GCC but not supported by Clang
(e.g. -specs=...). Currently, these cannot be passed to BPF selftests
builds, even when building with GCC, as some binaries (urandom_read and
liburandom_read.so) are always built with Clang and the unsupported
flags make the compilation fail (as -Werror is turned on).
Add -Wno-unused-command-line-argument to these rules to suppress such
errors.
This allows to do things like:
$ CFLAGS="-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1" \
make -C tools/testing/selftests/bpf
Without this patch, the compilation would fail with:
[...]
clang: error: argument unused during compilation: '-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1' [-Werror,-Wunused-command-line-argument]
make: *** [Makefile:273: /bpf-next/tools/testing/selftests/bpf/liburandom_read.so] Error 1
[...]
Signed-off-by: Viktor Malik <vmalik@redhat.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
---
tools/testing/selftests/bpf/Makefile | 2 ++
1 file changed, 2 insertions(+)
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 3b43d7db8d2c..edef5df08cb2 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -274,6 +274,7 @@ $(OUTPUT)/liburandom_read.so: urandom_read_lib1.c urandom_read_lib2.c liburandom
$(Q)$(CLANG) $(CLANG_TARGET_ARCH) \
$(filter-out -static,$(CFLAGS) $(LDFLAGS)) \
$(filter %.c,$^) $(filter-out -static,$(LDLIBS)) \
+ -Wno-unused-command-line-argument \
-fuse-ld=$(LLD) -Wl,-znoseparate-code -Wl,--build-id=sha1 \
-Wl,--version-script=liburandom_read.map \
-fPIC -shared -o $@
@@ -282,6 +283,7 @@ $(OUTPUT)/urandom_read: urandom_read.c urandom_read_aux.c $(OUTPUT)/liburandom_r
$(call msg,BINARY,,$@)
$(Q)$(CLANG) $(CLANG_TARGET_ARCH) \
$(filter-out -static,$(CFLAGS) $(LDFLAGS)) $(filter %.c,$^) \
+ -Wno-unused-command-line-argument \
-lurandom_read $(filter-out -static,$(LDLIBS)) -L$(OUTPUT) \
-fuse-ld=$(LLD) -Wl,-znoseparate-code -Wl,--build-id=sha1 \
-Wl,-rpath=. -o $@
--
2.47.0
^ permalink raw reply related [flat|nested] 13+ messages in thread* Re: [PATCH bpf-next v3 0/3] selftests/bpf: Improve building with extra
2024-11-01 8:27 [PATCH bpf-next v3 0/3] selftests/bpf: Improve building with extra Viktor Malik
` (2 preceding siblings ...)
2024-11-01 8:27 ` [PATCH bpf-next v3 3/3] selftests/bpf: Disable warnings on unused flags for Clang builds Viktor Malik
@ 2024-11-01 19:46 ` Andrii Nakryiko
2024-11-04 11:34 ` Viktor Malik
2024-11-01 19:50 ` patchwork-bot+netdevbpf
2024-11-06 20:40 ` patchwork-bot+netdevbpf
5 siblings, 1 reply; 13+ messages in thread
From: Andrii Nakryiko @ 2024-11-01 19:46 UTC (permalink / raw)
To: Viktor Malik
Cc: bpf, Andrii Nakryiko, Eduard Zingerman, Mykola Lysenko,
Alexei Starovoitov, Daniel Borkmann, Martin KaFai Lau, Song Liu,
Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev,
Hao Luo, Jiri Olsa, Shuah Khan, Nathan Chancellor,
Nick Desaulniers, Bill Wendling, Justin Stitt
On Fri, Nov 1, 2024 at 1:38 AM Viktor Malik <vmalik@redhat.com> wrote:
>
> When trying to build BPF selftests with additional compiler and linker
> flags, we're running into multiple problems. This series addresses all
> of them:
>
> - CFLAGS are not passed to sub-makes of bpftool and libbpf. This is a
> problem when compiling with PIE as libbpf.a ends up being non-PIE and
> cannot be linked with other binaries (patch #1).
>
> - bpftool Makefile runs `llvm-config --cflags` and appends the result to
> CFLAGS. The result typically contains `-D_GNU_SOURCE` which may be
> already set in CFLAGS. That causes a compilation error (patch #2).
>
> - Some GCC flags are not supported by Clang but there are binaries which
> are always built with Clang but reuse user-defined CFLAGS. When CFLAGS
> contain such flags, compilation fails (patch #3).
>
> Changelog:
> ----------
> v2 -> v3:
> - resolve conflicts between patch #1 and 4192bb294f80 ("selftests/bpf:
> Provide a generic [un]load_module helper")
> - add Quentin's and Jiri's acks for patches #2 and #3
>
> v1 -> v2:
> - cover forgotten case in patch#1 (noted by Eduard)
> - remove -D_GNU_SOURCE unconditionally in patch#2 (suggested by Andrii)
> - rewrite patch#3 to just add -Wno-unused-command-line-argument
> (suggested by Andrii)
>
> Viktor Malik (3):
> selftests/bpf: Allow building with extra flags
> bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile
> selftests/bpf: Disable warnings on unused flags for Clang builds
>
I've applied the last two patches, they seem to be independent from
the first, right?
> tools/bpf/bpftool/Makefile | 6 ++++-
> tools/testing/selftests/bpf/Makefile | 36 +++++++++++++++++++---------
> 2 files changed, 30 insertions(+), 12 deletions(-)
>
> --
> 2.47.0
>
>
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH bpf-next v3 0/3] selftests/bpf: Improve building with extra
2024-11-01 19:46 ` [PATCH bpf-next v3 0/3] selftests/bpf: Improve building with extra Andrii Nakryiko
@ 2024-11-04 11:34 ` Viktor Malik
0 siblings, 0 replies; 13+ messages in thread
From: Viktor Malik @ 2024-11-04 11:34 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: bpf, Andrii Nakryiko, Eduard Zingerman, Mykola Lysenko,
Alexei Starovoitov, Daniel Borkmann, Martin KaFai Lau, Song Liu,
Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev,
Hao Luo, Jiri Olsa, Shuah Khan, Nathan Chancellor,
Nick Desaulniers, Bill Wendling, Justin Stitt
On 11/1/24 20:46, Andrii Nakryiko wrote:
> On Fri, Nov 1, 2024 at 1:38 AM Viktor Malik <vmalik@redhat.com> wrote:
>>
>> When trying to build BPF selftests with additional compiler and linker
>> flags, we're running into multiple problems. This series addresses all
>> of them:
>>
>> - CFLAGS are not passed to sub-makes of bpftool and libbpf. This is a
>> problem when compiling with PIE as libbpf.a ends up being non-PIE and
>> cannot be linked with other binaries (patch #1).
>>
>> - bpftool Makefile runs `llvm-config --cflags` and appends the result to
>> CFLAGS. The result typically contains `-D_GNU_SOURCE` which may be
>> already set in CFLAGS. That causes a compilation error (patch #2).
>>
>> - Some GCC flags are not supported by Clang but there are binaries which
>> are always built with Clang but reuse user-defined CFLAGS. When CFLAGS
>> contain such flags, compilation fails (patch #3).
>>
>> Changelog:
>> ----------
>> v2 -> v3:
>> - resolve conflicts between patch #1 and 4192bb294f80 ("selftests/bpf:
>> Provide a generic [un]load_module helper")
>> - add Quentin's and Jiri's acks for patches #2 and #3
>>
>> v1 -> v2:
>> - cover forgotten case in patch#1 (noted by Eduard)
>> - remove -D_GNU_SOURCE unconditionally in patch#2 (suggested by Andrii)
>> - rewrite patch#3 to just add -Wno-unused-command-line-argument
>> (suggested by Andrii)
>>
>> Viktor Malik (3):
>> selftests/bpf: Allow building with extra flags
>> bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile
>> selftests/bpf: Disable warnings on unused flags for Clang builds
>>
>
> I've applied the last two patches, they seem to be independent from
> the first, right?
Yes, they are independent, thanks. I'll sync with Toke on the first one.
Viktor
>
>
>> tools/bpf/bpftool/Makefile | 6 ++++-
>> tools/testing/selftests/bpf/Makefile | 36 +++++++++++++++++++---------
>> 2 files changed, 30 insertions(+), 12 deletions(-)
>>
>> --
>> 2.47.0
>>
>>
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH bpf-next v3 0/3] selftests/bpf: Improve building with extra
2024-11-01 8:27 [PATCH bpf-next v3 0/3] selftests/bpf: Improve building with extra Viktor Malik
` (3 preceding siblings ...)
2024-11-01 19:46 ` [PATCH bpf-next v3 0/3] selftests/bpf: Improve building with extra Andrii Nakryiko
@ 2024-11-01 19:50 ` patchwork-bot+netdevbpf
2024-11-06 20:40 ` patchwork-bot+netdevbpf
5 siblings, 0 replies; 13+ messages in thread
From: patchwork-bot+netdevbpf @ 2024-11-01 19:50 UTC (permalink / raw)
To: Viktor Malik
Cc: bpf, andrii, eddyz87, mykolal, ast, daniel, martin.lau, song,
yonghong.song, john.fastabend, kpsingh, sdf, haoluo, jolsa, shuah,
nathan, ndesaulniers, morbo, justinstitt
Hello:
This series was applied to bpf/bpf-next.git (master)
by Andrii Nakryiko <andrii@kernel.org>:
On Fri, 1 Nov 2024 09:27:10 +0100 you wrote:
> When trying to build BPF selftests with additional compiler and linker
> flags, we're running into multiple problems. This series addresses all
> of them:
>
> - CFLAGS are not passed to sub-makes of bpftool and libbpf. This is a
> problem when compiling with PIE as libbpf.a ends up being non-PIE and
> cannot be linked with other binaries (patch #1).
>
> [...]
Here is the summary with links:
- [bpf-next,v3,1/3] selftests/bpf: Allow building with extra flags
(no matching commit)
- [bpf-next,v3,2/3] bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile
https://git.kernel.org/bpf/bpf-next/c/0513eeee86d6
- [bpf-next,v3,3/3] selftests/bpf: Disable warnings on unused flags for Clang builds
https://git.kernel.org/bpf/bpf-next/c/77017b9c4682
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH bpf-next v3 0/3] selftests/bpf: Improve building with extra
2024-11-01 8:27 [PATCH bpf-next v3 0/3] selftests/bpf: Improve building with extra Viktor Malik
` (4 preceding siblings ...)
2024-11-01 19:50 ` patchwork-bot+netdevbpf
@ 2024-11-06 20:40 ` patchwork-bot+netdevbpf
5 siblings, 0 replies; 13+ messages in thread
From: patchwork-bot+netdevbpf @ 2024-11-06 20:40 UTC (permalink / raw)
To: Viktor Malik
Cc: bpf, andrii, eddyz87, mykolal, ast, daniel, martin.lau, song,
yonghong.song, john.fastabend, kpsingh, sdf, haoluo, jolsa, shuah,
nathan, ndesaulniers, morbo, justinstitt
Hello:
This series was applied to bpf/bpf-next.git (master)
by Andrii Nakryiko <andrii@kernel.org>:
On Fri, 1 Nov 2024 09:27:10 +0100 you wrote:
> When trying to build BPF selftests with additional compiler and linker
> flags, we're running into multiple problems. This series addresses all
> of them:
>
> - CFLAGS are not passed to sub-makes of bpftool and libbpf. This is a
> problem when compiling with PIE as libbpf.a ends up being non-PIE and
> cannot be linked with other binaries (patch #1).
>
> [...]
Here is the summary with links:
- [bpf-next,v3,1/3] selftests/bpf: Allow building with extra flags
https://git.kernel.org/bpf/bpf-next/c/9a28559932d2
- [bpf-next,v3,2/3] bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile
(no matching commit)
- [bpf-next,v3,3/3] selftests/bpf: Disable warnings on unused flags for Clang builds
(no matching commit)
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply [flat|nested] 13+ messages in thread