[PATCH bpf-next v2 0/3] selftests/bpf: Improve building with extra

[PATCH bpf-next v2 0/3] selftests/bpf: Improve building with extra

[Viktor Malik]

When trying to build BPF selftests with additional compiler and linker
flags, we're running into multiple problems. This series addresses all
of them:

- CFLAGS are not passed to sub-makes of bpftool and libbpf. This is a
  problem when compiling with PIE as libbpf.a ends up being non-PIE and
  cannot be linked with other binaries (patch #1).

- bpftool Makefile runs `llvm-config --cflags` and appends the result to
  CFLAGS. The result typically contains `-D_GNU_SOURCE` which may be
  already set in CFLAGS. That causes a compilation error (patch #2).

- Some GCC flags are not supported by Clang but there are binaries which
  are always built with Clang but reuse user-defined CFLAGS. When CFLAGS
  contain such flags, compilation fails (patch #3).

Changelog:
----------
v1 -> v2:
- cover forgotten case in patch#1 (noted by Eduard)
- remove -D_GNU_SOURCE unconditionally in patch#2 (suggested by Andrii)
- rewrite patch#3 to just add -Wno-unused-command-line-argument
  (suggested by Andrii)

Viktor Malik (3):
  selftests/bpf: Allow building with extra flags
  bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile
  selftests/bpf: Disable warnings on unused flags for Clang builds

 tools/bpf/bpftool/Makefile           |  6 +++++-
 tools/testing/selftests/bpf/Makefile | 28 +++++++++++++++++++---------
 2 files changed, 24 insertions(+), 10 deletions(-)

-- 
2.47.0

[PATCH bpf-next v2 1/3] selftests/bpf: Allow building with extra flags

[Viktor Malik]

In order to specify extra compilation or linking flags to BPF selftests,
it is possible to set EXTRA_CFLAGS and EXTRA_LDFLAGS from the command
line. The problem is that they are not propagated to sub-make calls
(runqslower, bpftool, libbpf) and in the better case are not applied, in
the worse case cause the entire build fail.

Propagate EXTRA_CFLAGS and EXTRA_LDFLAGS to the sub-makes.

This, for instance, allows to build selftests as PIE with

    $ make EXTRA_CFLAGS='-fPIE' EXTRA_LDFLAGS='-pie'

Without this change, the command would fail because libbpf.a would not
be built with -fPIE and other PIE binaries would not link against it.

The only problem is that we have to explicitly provide empty
EXTRA_CFLAGS='' and EXTRA_LDFLAGS='' to the builds of kernel modules
(bpf_testmod and bpf_test_no_cfi) as we don't want to build modules with
flags used for userspace (the above example would fail as kernel doesn't
support PIE).

Signed-off-by: Viktor Malik <vmalik@redhat.com>
Tested-by: Eduard Zingerman <eddyz87@gmail.com>
---
 tools/testing/selftests/bpf/Makefile | 26 +++++++++++++++++---------
 1 file changed, 17 insertions(+), 9 deletions(-)

diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 28a76baa854d..1fc7c38e56b5 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -294,13 +294,17 @@ $(OUTPUT)/sign-file: ../../../../scripts/sign-file.c
 $(OUTPUT)/bpf_testmod.ko: $(VMLINUX_BTF) $(RESOLVE_BTFIDS) $(wildcard bpf_testmod/Makefile bpf_testmod/*.[ch])
 	$(call msg,MOD,,$@)
 	$(Q)$(RM) bpf_testmod/bpf_testmod.ko # force re-compilation
-	$(Q)$(MAKE) $(submake_extras) RESOLVE_BTFIDS=$(RESOLVE_BTFIDS) -C bpf_testmod
+	$(Q)$(MAKE) $(submake_extras) -C bpf_testmod \
+		RESOLVE_BTFIDS=$(RESOLVE_BTFIDS)     \
+		EXTRA_CFLAGS='' EXTRA_LDFLAGS=''
 	$(Q)cp bpf_testmod/bpf_testmod.ko $@
 
 $(OUTPUT)/bpf_test_no_cfi.ko: $(VMLINUX_BTF) $(RESOLVE_BTFIDS) $(wildcard bpf_test_no_cfi/Makefile bpf_test_no_cfi/*.[ch])
 	$(call msg,MOD,,$@)
 	$(Q)$(RM) bpf_test_no_cfi/bpf_test_no_cfi.ko # force re-compilation
-	$(Q)$(MAKE) $(submake_extras) RESOLVE_BTFIDS=$(RESOLVE_BTFIDS) -C bpf_test_no_cfi
+	$(Q)$(MAKE) $(submake_extras) -C bpf_test_no_cfi \
+		RESOLVE_BTFIDS=$(RESOLVE_BTFIDS)	 \
+		EXTRA_CFLAGS='' EXTRA_LDFLAGS=''
 	$(Q)cp bpf_test_no_cfi/bpf_test_no_cfi.ko $@
 
 DEFAULT_BPFTOOL := $(HOST_SCRATCH_DIR)/sbin/bpftool
@@ -319,8 +323,8 @@ $(OUTPUT)/runqslower: $(BPFOBJ) | $(DEFAULT_BPFTOOL) $(RUNQSLOWER_OUTPUT)
 		    BPFTOOL_OUTPUT=$(HOST_BUILD_DIR)/bpftool/		       \
 		    BPFOBJ_OUTPUT=$(BUILD_DIR)/libbpf/			       \
 		    BPFOBJ=$(BPFOBJ) BPF_INCLUDE=$(INCLUDE_DIR)		       \
-		    EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS)'	       \
-		    EXTRA_LDFLAGS='$(SAN_LDFLAGS)' &&			       \
+		    EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS) $(EXTRA_CFLAGS)' \
+		    EXTRA_LDFLAGS='$(SAN_LDFLAGS) $(EXTRA_LDFLAGS)' &&	       \
 		    cp $(RUNQSLOWER_OUTPUT)runqslower $@
 
 TEST_GEN_PROGS_EXTENDED += $(TRUNNER_BPFTOOL)
@@ -354,7 +358,8 @@ $(DEFAULT_BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile)    \
 		    $(HOST_BPFOBJ) | $(HOST_BUILD_DIR)/bpftool
 	$(Q)$(MAKE) $(submake_extras)  -C $(BPFTOOLDIR)			       \
 		    ARCH= CROSS_COMPILE= CC="$(HOSTCC)" LD="$(HOSTLD)" 	       \
-		    EXTRA_CFLAGS='-g $(OPT_FLAGS)'			       \
+		    EXTRA_CFLAGS='-g $(OPT_FLAGS) $(EXTRA_CFLAGS)'	       \
+		    EXTRA_LDFLAGS='$(EXTRA_LDFLAGS)'			       \
 		    OUTPUT=$(HOST_BUILD_DIR)/bpftool/			       \
 		    LIBBPF_OUTPUT=$(HOST_BUILD_DIR)/libbpf/		       \
 		    LIBBPF_DESTDIR=$(HOST_SCRATCH_DIR)/			       \
@@ -365,7 +370,8 @@ $(CROSS_BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile)	\
 		    $(BPFOBJ) | $(BUILD_DIR)/bpftool
 	$(Q)$(MAKE) $(submake_extras)  -C $(BPFTOOLDIR)				\
 		    ARCH=$(ARCH) CROSS_COMPILE=$(CROSS_COMPILE)			\
-		    EXTRA_CFLAGS='-g $(OPT_FLAGS)'				\
+		    EXTRA_CFLAGS='-g $(OPT_FLAGS) $(EXTRA_CFLAGS)'		\
+		    EXTRA_LDFLAGS='$(EXTRA_LDFLAGS)'				\
 		    OUTPUT=$(BUILD_DIR)/bpftool/				\
 		    LIBBPF_OUTPUT=$(BUILD_DIR)/libbpf/				\
 		    LIBBPF_DESTDIR=$(SCRATCH_DIR)/				\
@@ -388,8 +394,8 @@ $(BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile)		       \
 	   $(APIDIR)/linux/bpf.h					       \
 	   | $(BUILD_DIR)/libbpf
 	$(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) OUTPUT=$(BUILD_DIR)/libbpf/ \
-		    EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS)'	       \
-		    EXTRA_LDFLAGS='$(SAN_LDFLAGS)'			       \
+		    EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS) $(EXTRA_CFLAGS)' \
+		    EXTRA_LDFLAGS='$(SAN_LDFLAGS) $(EXTRA_LDFLAGS)'	       \
 		    DESTDIR=$(SCRATCH_DIR) prefix= all install_headers
 
 ifneq ($(BPFOBJ),$(HOST_BPFOBJ))
@@ -397,7 +403,9 @@ $(HOST_BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile)		       \
 		$(APIDIR)/linux/bpf.h					       \
 		| $(HOST_BUILD_DIR)/libbpf
 	$(Q)$(MAKE) $(submake_extras) -C $(BPFDIR)                             \
-		    EXTRA_CFLAGS='-g $(OPT_FLAGS)' ARCH= CROSS_COMPILE=	       \
+		    ARCH= CROSS_COMPILE=				       \
+		    EXTRA_CFLAGS='-g $(OPT_FLAGS) $(EXTRA_CFLAGS)'	       \
+		    EXTRA_LDFLAGS='$(EXTRA_LDFLAGS)'			       \
 		    OUTPUT=$(HOST_BUILD_DIR)/libbpf/			       \
 		    CC="$(HOSTCC)" LD="$(HOSTLD)"			       \
 		    DESTDIR=$(HOST_SCRATCH_DIR)/ prefix= all install_headers
-- 
2.47.0

[PATCH bpf-next v2 2/3] bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile

[Viktor Malik]

When building selftests with CFLAGS set via env variable, the value of
CFLAGS is propagated into bpftool Makefile (called from selftests
Makefile). This makes the compilation fail as _GNU_SOURCE is defined two
times - once from selftests Makefile (by including lib.mk) and once from
bpftool Makefile (by calling `llvm-config --cflags`):

    $ CFLAGS="" make -C tools/testing/selftests/bpf
    [...]
    CC      /bpf-next/tools/testing/selftests/bpf/tools/build/bpftool/btf.o
    <command-line>: error: "_GNU_SOURCE" redefined [-Werror]
    <command-line>: note: this is the location of the previous definition
    cc1: all warnings being treated as errors
    [...]

Filter out -D_GNU_SOURCE from the result of `llvm-config --cflags` in
bpftool Makefile to prevent this error.

Signed-off-by: Viktor Malik <vmalik@redhat.com>
---
 tools/bpf/bpftool/Makefile | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
index ba927379eb20..a4263dfb5e03 100644
--- a/tools/bpf/bpftool/Makefile
+++ b/tools/bpf/bpftool/Makefile
@@ -147,7 +147,11 @@ ifeq ($(feature-llvm),1)
   # If LLVM is available, use it for JIT disassembly
   CFLAGS  += -DHAVE_LLVM_SUPPORT
   LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets
-  CFLAGS  += $(shell $(LLVM_CONFIG) --cflags)
+  # llvm-config always adds -D_GNU_SOURCE, however, it may already be in CFLAGS
+  # (e.g. when bpftool build is called from selftests build as selftests
+  # Makefile includes lib.mk which sets -D_GNU_SOURCE) which would cause
+  # compilation error due to redefinition. Let's filter it out here.
+  CFLAGS  += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))
   LIBS    += $(shell $(LLVM_CONFIG) --libs $(LLVM_CONFIG_LIB_COMPONENTS))
   ifeq ($(shell $(LLVM_CONFIG) --shared-mode),static)
     LIBS += $(shell $(LLVM_CONFIG) --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
-- 
2.47.0

[PATCH bpf-next v2 3/3] selftests/bpf: Disable warnings on unused flags for Clang builds

[Viktor Malik]

There exist compiler flags supported by GCC but not supported by Clang
(e.g. -specs=...). Currently, these cannot be passed to BPF selftests
builds, even when building with GCC, as some binaries (urandom_read and
liburandom_read.so) are always built with Clang and the unsupported
flags make the compilation fail (as -Werror is turned on).

Add -Wno-unused-command-line-argument to these rules to suppress such
errors.

This allows to do things like:

    $ CFLAGS="-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1" \
      make -C tools/testing/selftests/bpf

Without this patch, the compilation would fail with:

    [...]
    clang: error: argument unused during compilation: '-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1' [-Werror,-Wunused-command-line-argument]
    make: *** [Makefile:273: /bpf-next/tools/testing/selftests/bpf/liburandom_read.so] Error 1
    [...]

Signed-off-by: Viktor Malik <vmalik@redhat.com>
---
 tools/testing/selftests/bpf/Makefile | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 1fc7c38e56b5..3da1a61968b7 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -273,6 +273,7 @@ $(OUTPUT)/liburandom_read.so: urandom_read_lib1.c urandom_read_lib2.c liburandom
 	$(Q)$(CLANG) $(CLANG_TARGET_ARCH) \
 		     $(filter-out -static,$(CFLAGS) $(LDFLAGS)) \
 		     $(filter %.c,$^) $(filter-out -static,$(LDLIBS)) \
+		     -Wno-unused-command-line-argument \
 		     -fuse-ld=$(LLD) -Wl,-znoseparate-code -Wl,--build-id=sha1 \
 		     -Wl,--version-script=liburandom_read.map \
 		     -fPIC -shared -o $@
@@ -281,6 +282,7 @@ $(OUTPUT)/urandom_read: urandom_read.c urandom_read_aux.c $(OUTPUT)/liburandom_r
 	$(call msg,BINARY,,$@)
 	$(Q)$(CLANG) $(CLANG_TARGET_ARCH) \
 		     $(filter-out -static,$(CFLAGS) $(LDFLAGS)) $(filter %.c,$^) \
+		     -Wno-unused-command-line-argument \
 		     -lurandom_read $(filter-out -static,$(LDLIBS)) -L$(OUTPUT) \
 		     -fuse-ld=$(LLD) -Wl,-znoseparate-code -Wl,--build-id=sha1 \
 		     -Wl,-rpath=. -o $@
-- 
2.47.0

Re: [PATCH bpf-next v2 2/3] bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile

[Quentin Monnet]

2024-10-18 08:49 UTC+0200 ~ Viktor Malik <vmalik@redhat.com>
> When building selftests with CFLAGS set via env variable, the value of
> CFLAGS is propagated into bpftool Makefile (called from selftests
> Makefile). This makes the compilation fail as _GNU_SOURCE is defined two
> times - once from selftests Makefile (by including lib.mk) and once from
> bpftool Makefile (by calling `llvm-config --cflags`):
> 
>     $ CFLAGS="" make -C tools/testing/selftests/bpf
>     [...]
>     CC      /bpf-next/tools/testing/selftests/bpf/tools/build/bpftool/btf.o
>     <command-line>: error: "_GNU_SOURCE" redefined [-Werror]
>     <command-line>: note: this is the location of the previous definition
>     cc1: all warnings being treated as errors
>     [...]
> 
> Filter out -D_GNU_SOURCE from the result of `llvm-config --cflags` in
> bpftool Makefile to prevent this error.
> 
> Signed-off-by: Viktor Malik <vmalik@redhat.com>

I am not aware of any extension in use in bpftool that would require the
use of -D_GNU_SOURCE, so I suppose it's fine to remove it
unconditionally, even when it's not in CFLAGS before we append the flags
from llvm-config. Thanks!

Acked-by: Quentin Monnet <qmo@kernel.org>

Re: [PATCH bpf-next v2 3/3] selftests/bpf: Disable warnings on unused flags for Clang builds

[Jiri Olsa]

On Fri, Oct 18, 2024 at 08:49:01AM +0200, Viktor Malik wrote:
> There exist compiler flags supported by GCC but not supported by Clang
> (e.g. -specs=...). Currently, these cannot be passed to BPF selftests
> builds, even when building with GCC, as some binaries (urandom_read and
> liburandom_read.so) are always built with Clang and the unsupported
> flags make the compilation fail (as -Werror is turned on).
> 
> Add -Wno-unused-command-line-argument to these rules to suppress such
> errors.
> 
> This allows to do things like:
> 
>     $ CFLAGS="-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1" \
>       make -C tools/testing/selftests/bpf

hi,
might be my fedora setup, but this example gives me compile error below
even with the patch applied:

  EXT-OBJ  [test_progs] testing_helpers.o
In file included from testing_helpers.c:10:
disasm.h:11:10: fatal error: linux/stringify.h: No such file or directory
   11 | #include <linux/stringify.h>
      |          ^~~~~~~~~~~~~~~~~~~

jirka

> 
> Without this patch, the compilation would fail with:
> 
>     [...]
>     clang: error: argument unused during compilation: '-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1' [-Werror,-Wunused-command-line-argument]
>     make: *** [Makefile:273: /bpf-next/tools/testing/selftests/bpf/liburandom_read.so] Error 1
>     [...]
> 
> Signed-off-by: Viktor Malik <vmalik@redhat.com>
> ---
>  tools/testing/selftests/bpf/Makefile | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index 1fc7c38e56b5..3da1a61968b7 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -273,6 +273,7 @@ $(OUTPUT)/liburandom_read.so: urandom_read_lib1.c urandom_read_lib2.c liburandom
>  	$(Q)$(CLANG) $(CLANG_TARGET_ARCH) \
>  		     $(filter-out -static,$(CFLAGS) $(LDFLAGS)) \
>  		     $(filter %.c,$^) $(filter-out -static,$(LDLIBS)) \
> +		     -Wno-unused-command-line-argument \
>  		     -fuse-ld=$(LLD) -Wl,-znoseparate-code -Wl,--build-id=sha1 \
>  		     -Wl,--version-script=liburandom_read.map \
>  		     -fPIC -shared -o $@
> @@ -281,6 +282,7 @@ $(OUTPUT)/urandom_read: urandom_read.c urandom_read_aux.c $(OUTPUT)/liburandom_r
>  	$(call msg,BINARY,,$@)
>  	$(Q)$(CLANG) $(CLANG_TARGET_ARCH) \
>  		     $(filter-out -static,$(CFLAGS) $(LDFLAGS)) $(filter %.c,$^) \
> +		     -Wno-unused-command-line-argument \
>  		     -lurandom_read $(filter-out -static,$(LDLIBS)) -L$(OUTPUT) \
>  		     -fuse-ld=$(LLD) -Wl,-znoseparate-code -Wl,--build-id=sha1 \
>  		     -Wl,-rpath=. -o $@
> -- 
> 2.47.0
> 

Re: [PATCH bpf-next v2 3/3] selftests/bpf: Disable warnings on unused flags for Clang builds

[Viktor Malik]

On 10/20/24 19:24, Jiri Olsa wrote:
> On Fri, Oct 18, 2024 at 08:49:01AM +0200, Viktor Malik wrote:
>> There exist compiler flags supported by GCC but not supported by Clang
>> (e.g. -specs=...). Currently, these cannot be passed to BPF selftests
>> builds, even when building with GCC, as some binaries (urandom_read and
>> liburandom_read.so) are always built with Clang and the unsupported
>> flags make the compilation fail (as -Werror is turned on).
>>
>> Add -Wno-unused-command-line-argument to these rules to suppress such
>> errors.
>>
>> This allows to do things like:
>>
>>     $ CFLAGS="-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1" \
>>       make -C tools/testing/selftests/bpf
> 
> hi,
> might be my fedora setup, but this example gives me compile error below
> even with the patch applied:
> 
>   EXT-OBJ  [test_progs] testing_helpers.o
> In file included from testing_helpers.c:10:
> disasm.h:11:10: fatal error: linux/stringify.h: No such file or directory
>    11 | #include <linux/stringify.h>
>       |          ^~~~~~~~~~~~~~~~~~~

Aren't you doing `make CFLAGS="..."` instead of `CFLAGS="..." make`? The
difference is that the former overrides CFLAGS defined in selftests
Makefile and therefore the include dirs are not correctly added.

> 
> jirka
> 
>>
>> Without this patch, the compilation would fail with:
>>
>>     [...]
>>     clang: error: argument unused during compilation: '-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1' [-Werror,-Wunused-command-line-argument]
>>     make: *** [Makefile:273: /bpf-next/tools/testing/selftests/bpf/liburandom_read.so] Error 1
>>     [...]
>>
>> Signed-off-by: Viktor Malik <vmalik@redhat.com>
>> ---
>>  tools/testing/selftests/bpf/Makefile | 2 ++
>>  1 file changed, 2 insertions(+)
>>
>> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
>> index 1fc7c38e56b5..3da1a61968b7 100644
>> --- a/tools/testing/selftests/bpf/Makefile
>> +++ b/tools/testing/selftests/bpf/Makefile
>> @@ -273,6 +273,7 @@ $(OUTPUT)/liburandom_read.so: urandom_read_lib1.c urandom_read_lib2.c liburandom
>>  	$(Q)$(CLANG) $(CLANG_TARGET_ARCH) \
>>  		     $(filter-out -static,$(CFLAGS) $(LDFLAGS)) \
>>  		     $(filter %.c,$^) $(filter-out -static,$(LDLIBS)) \
>> +		     -Wno-unused-command-line-argument \
>>  		     -fuse-ld=$(LLD) -Wl,-znoseparate-code -Wl,--build-id=sha1 \
>>  		     -Wl,--version-script=liburandom_read.map \
>>  		     -fPIC -shared -o $@
>> @@ -281,6 +282,7 @@ $(OUTPUT)/urandom_read: urandom_read.c urandom_read_aux.c $(OUTPUT)/liburandom_r
>>  	$(call msg,BINARY,,$@)
>>  	$(Q)$(CLANG) $(CLANG_TARGET_ARCH) \
>>  		     $(filter-out -static,$(CFLAGS) $(LDFLAGS)) $(filter %.c,$^) \
>> +		     -Wno-unused-command-line-argument \
>>  		     -lurandom_read $(filter-out -static,$(LDLIBS)) -L$(OUTPUT) \
>>  		     -fuse-ld=$(LLD) -Wl,-znoseparate-code -Wl,--build-id=sha1 \
>>  		     -Wl,-rpath=. -o $@
>> -- 
>> 2.47.0
>>
> 

Re: [PATCH bpf-next v2 3/3] selftests/bpf: Disable warnings on unused flags for Clang builds

[Jiri Olsa]

On Mon, Oct 21, 2024 at 08:20:39AM +0200, Viktor Malik wrote:
> On 10/20/24 19:24, Jiri Olsa wrote:
> > On Fri, Oct 18, 2024 at 08:49:01AM +0200, Viktor Malik wrote:
> >> There exist compiler flags supported by GCC but not supported by Clang
> >> (e.g. -specs=...). Currently, these cannot be passed to BPF selftests
> >> builds, even when building with GCC, as some binaries (urandom_read and
> >> liburandom_read.so) are always built with Clang and the unsupported
> >> flags make the compilation fail (as -Werror is turned on).
> >>
> >> Add -Wno-unused-command-line-argument to these rules to suppress such
> >> errors.
> >>
> >> This allows to do things like:
> >>
> >>     $ CFLAGS="-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1" \
> >>       make -C tools/testing/selftests/bpf
> > 
> > hi,
> > might be my fedora setup, but this example gives me compile error below
> > even with the patch applied:
> > 
> >   EXT-OBJ  [test_progs] testing_helpers.o
> > In file included from testing_helpers.c:10:
> > disasm.h:11:10: fatal error: linux/stringify.h: No such file or directory
> >    11 | #include <linux/stringify.h>
> >       |          ^~~~~~~~~~~~~~~~~~~
> 
> Aren't you doing `make CFLAGS="..."` instead of `CFLAGS="..." make`? The
> difference is that the former overrides CFLAGS defined in selftests
> Makefile and therefore the include dirs are not correctly added.

right, I was doing that, thanks

Acked-by: Jiri Olsa <jolsa@kernel.org>

jirka

> 
> > 
> > jirka
> > 
> >>
> >> Without this patch, the compilation would fail with:
> >>
> >>     [...]
> >>     clang: error: argument unused during compilation: '-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1' [-Werror,-Wunused-command-line-argument]
> >>     make: *** [Makefile:273: /bpf-next/tools/testing/selftests/bpf/liburandom_read.so] Error 1
> >>     [...]
> >>
> >> Signed-off-by: Viktor Malik <vmalik@redhat.com>
> >> ---
> >>  tools/testing/selftests/bpf/Makefile | 2 ++
> >>  1 file changed, 2 insertions(+)
> >>
> >> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> >> index 1fc7c38e56b5..3da1a61968b7 100644
> >> --- a/tools/testing/selftests/bpf/Makefile
> >> +++ b/tools/testing/selftests/bpf/Makefile
> >> @@ -273,6 +273,7 @@ $(OUTPUT)/liburandom_read.so: urandom_read_lib1.c urandom_read_lib2.c liburandom
> >>  	$(Q)$(CLANG) $(CLANG_TARGET_ARCH) \
> >>  		     $(filter-out -static,$(CFLAGS) $(LDFLAGS)) \
> >>  		     $(filter %.c,$^) $(filter-out -static,$(LDLIBS)) \
> >> +		     -Wno-unused-command-line-argument \
> >>  		     -fuse-ld=$(LLD) -Wl,-znoseparate-code -Wl,--build-id=sha1 \
> >>  		     -Wl,--version-script=liburandom_read.map \
> >>  		     -fPIC -shared -o $@
> >> @@ -281,6 +282,7 @@ $(OUTPUT)/urandom_read: urandom_read.c urandom_read_aux.c $(OUTPUT)/liburandom_r
> >>  	$(call msg,BINARY,,$@)
> >>  	$(Q)$(CLANG) $(CLANG_TARGET_ARCH) \
> >>  		     $(filter-out -static,$(CFLAGS) $(LDFLAGS)) $(filter %.c,$^) \
> >> +		     -Wno-unused-command-line-argument \
> >>  		     -lurandom_read $(filter-out -static,$(LDLIBS)) -L$(OUTPUT) \
> >>  		     -fuse-ld=$(LLD) -Wl,-znoseparate-code -Wl,--build-id=sha1 \
> >>  		     -Wl,-rpath=. -o $@
> >> -- 
> >> 2.47.0
> >>
> > 
> 

Re: [PATCH bpf-next v2 2/3] bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile

[Jiri Olsa]

On Fri, Oct 18, 2024 at 08:49:00AM +0200, Viktor Malik wrote:
> When building selftests with CFLAGS set via env variable, the value of
> CFLAGS is propagated into bpftool Makefile (called from selftests
> Makefile). This makes the compilation fail as _GNU_SOURCE is defined two
> times - once from selftests Makefile (by including lib.mk) and once from
> bpftool Makefile (by calling `llvm-config --cflags`):
> 
>     $ CFLAGS="" make -C tools/testing/selftests/bpf
>     [...]
>     CC      /bpf-next/tools/testing/selftests/bpf/tools/build/bpftool/btf.o
>     <command-line>: error: "_GNU_SOURCE" redefined [-Werror]
>     <command-line>: note: this is the location of the previous definition
>     cc1: all warnings being treated as errors
>     [...]
> 
> Filter out -D_GNU_SOURCE from the result of `llvm-config --cflags` in
> bpftool Makefile to prevent this error.
> 
> Signed-off-by: Viktor Malik <vmalik@redhat.com>

Acked-by: Jiri Olsa <jolsa@kernel.org>

jirka

> ---
>  tools/bpf/bpftool/Makefile | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
> index ba927379eb20..a4263dfb5e03 100644
> --- a/tools/bpf/bpftool/Makefile
> +++ b/tools/bpf/bpftool/Makefile
> @@ -147,7 +147,11 @@ ifeq ($(feature-llvm),1)
>    # If LLVM is available, use it for JIT disassembly
>    CFLAGS  += -DHAVE_LLVM_SUPPORT
>    LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets
> -  CFLAGS  += $(shell $(LLVM_CONFIG) --cflags)
> +  # llvm-config always adds -D_GNU_SOURCE, however, it may already be in CFLAGS
> +  # (e.g. when bpftool build is called from selftests build as selftests
> +  # Makefile includes lib.mk which sets -D_GNU_SOURCE) which would cause
> +  # compilation error due to redefinition. Let's filter it out here.
> +  CFLAGS  += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))
>    LIBS    += $(shell $(LLVM_CONFIG) --libs $(LLVM_CONFIG_LIB_COMPONENTS))
>    ifeq ($(shell $(LLVM_CONFIG) --shared-mode),static)
>      LIBS += $(shell $(LLVM_CONFIG) --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
> -- 
> 2.47.0
> 

Re: [PATCH bpf-next v2 1/3] selftests/bpf: Allow building with extra flags

[Jiri Olsa]

On Fri, Oct 18, 2024 at 08:48:59AM +0200, Viktor Malik wrote:
> In order to specify extra compilation or linking flags to BPF selftests,
> it is possible to set EXTRA_CFLAGS and EXTRA_LDFLAGS from the command
> line. The problem is that they are not propagated to sub-make calls
> (runqslower, bpftool, libbpf) and in the better case are not applied, in
> the worse case cause the entire build fail.
> 
> Propagate EXTRA_CFLAGS and EXTRA_LDFLAGS to the sub-makes.
> 
> This, for instance, allows to build selftests as PIE with
> 
>     $ make EXTRA_CFLAGS='-fPIE' EXTRA_LDFLAGS='-pie'
> 
> Without this change, the command would fail because libbpf.a would not
> be built with -fPIE and other PIE binaries would not link against it.
> 
> The only problem is that we have to explicitly provide empty
> EXTRA_CFLAGS='' and EXTRA_LDFLAGS='' to the builds of kernel modules
> (bpf_testmod and bpf_test_no_cfi) as we don't want to build modules with
> flags used for userspace (the above example would fail as kernel doesn't
> support PIE).
> 
> Signed-off-by: Viktor Malik <vmalik@redhat.com>
> Tested-by: Eduard Zingerman <eddyz87@gmail.com>

lgtm

Acked-by: Jiri Olsa <jolsa@kernel.org>

jirka

> ---
>  tools/testing/selftests/bpf/Makefile | 26 +++++++++++++++++---------
>  1 file changed, 17 insertions(+), 9 deletions(-)
> 
> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index 28a76baa854d..1fc7c38e56b5 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -294,13 +294,17 @@ $(OUTPUT)/sign-file: ../../../../scripts/sign-file.c
>  $(OUTPUT)/bpf_testmod.ko: $(VMLINUX_BTF) $(RESOLVE_BTFIDS) $(wildcard bpf_testmod/Makefile bpf_testmod/*.[ch])
>  	$(call msg,MOD,,$@)
>  	$(Q)$(RM) bpf_testmod/bpf_testmod.ko # force re-compilation
> -	$(Q)$(MAKE) $(submake_extras) RESOLVE_BTFIDS=$(RESOLVE_BTFIDS) -C bpf_testmod
> +	$(Q)$(MAKE) $(submake_extras) -C bpf_testmod \
> +		RESOLVE_BTFIDS=$(RESOLVE_BTFIDS)     \
> +		EXTRA_CFLAGS='' EXTRA_LDFLAGS=''
>  	$(Q)cp bpf_testmod/bpf_testmod.ko $@
>  
>  $(OUTPUT)/bpf_test_no_cfi.ko: $(VMLINUX_BTF) $(RESOLVE_BTFIDS) $(wildcard bpf_test_no_cfi/Makefile bpf_test_no_cfi/*.[ch])
>  	$(call msg,MOD,,$@)
>  	$(Q)$(RM) bpf_test_no_cfi/bpf_test_no_cfi.ko # force re-compilation
> -	$(Q)$(MAKE) $(submake_extras) RESOLVE_BTFIDS=$(RESOLVE_BTFIDS) -C bpf_test_no_cfi
> +	$(Q)$(MAKE) $(submake_extras) -C bpf_test_no_cfi \
> +		RESOLVE_BTFIDS=$(RESOLVE_BTFIDS)	 \
> +		EXTRA_CFLAGS='' EXTRA_LDFLAGS=''
>  	$(Q)cp bpf_test_no_cfi/bpf_test_no_cfi.ko $@
>  
>  DEFAULT_BPFTOOL := $(HOST_SCRATCH_DIR)/sbin/bpftool
> @@ -319,8 +323,8 @@ $(OUTPUT)/runqslower: $(BPFOBJ) | $(DEFAULT_BPFTOOL) $(RUNQSLOWER_OUTPUT)
>  		    BPFTOOL_OUTPUT=$(HOST_BUILD_DIR)/bpftool/		       \
>  		    BPFOBJ_OUTPUT=$(BUILD_DIR)/libbpf/			       \
>  		    BPFOBJ=$(BPFOBJ) BPF_INCLUDE=$(INCLUDE_DIR)		       \
> -		    EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS)'	       \
> -		    EXTRA_LDFLAGS='$(SAN_LDFLAGS)' &&			       \
> +		    EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS) $(EXTRA_CFLAGS)' \
> +		    EXTRA_LDFLAGS='$(SAN_LDFLAGS) $(EXTRA_LDFLAGS)' &&	       \
>  		    cp $(RUNQSLOWER_OUTPUT)runqslower $@
>  
>  TEST_GEN_PROGS_EXTENDED += $(TRUNNER_BPFTOOL)
> @@ -354,7 +358,8 @@ $(DEFAULT_BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile)    \
>  		    $(HOST_BPFOBJ) | $(HOST_BUILD_DIR)/bpftool
>  	$(Q)$(MAKE) $(submake_extras)  -C $(BPFTOOLDIR)			       \
>  		    ARCH= CROSS_COMPILE= CC="$(HOSTCC)" LD="$(HOSTLD)" 	       \
> -		    EXTRA_CFLAGS='-g $(OPT_FLAGS)'			       \
> +		    EXTRA_CFLAGS='-g $(OPT_FLAGS) $(EXTRA_CFLAGS)'	       \
> +		    EXTRA_LDFLAGS='$(EXTRA_LDFLAGS)'			       \
>  		    OUTPUT=$(HOST_BUILD_DIR)/bpftool/			       \
>  		    LIBBPF_OUTPUT=$(HOST_BUILD_DIR)/libbpf/		       \
>  		    LIBBPF_DESTDIR=$(HOST_SCRATCH_DIR)/			       \
> @@ -365,7 +370,8 @@ $(CROSS_BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile)	\
>  		    $(BPFOBJ) | $(BUILD_DIR)/bpftool
>  	$(Q)$(MAKE) $(submake_extras)  -C $(BPFTOOLDIR)				\
>  		    ARCH=$(ARCH) CROSS_COMPILE=$(CROSS_COMPILE)			\
> -		    EXTRA_CFLAGS='-g $(OPT_FLAGS)'				\
> +		    EXTRA_CFLAGS='-g $(OPT_FLAGS) $(EXTRA_CFLAGS)'		\
> +		    EXTRA_LDFLAGS='$(EXTRA_LDFLAGS)'				\
>  		    OUTPUT=$(BUILD_DIR)/bpftool/				\
>  		    LIBBPF_OUTPUT=$(BUILD_DIR)/libbpf/				\
>  		    LIBBPF_DESTDIR=$(SCRATCH_DIR)/				\
> @@ -388,8 +394,8 @@ $(BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile)		       \
>  	   $(APIDIR)/linux/bpf.h					       \
>  	   | $(BUILD_DIR)/libbpf
>  	$(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) OUTPUT=$(BUILD_DIR)/libbpf/ \
> -		    EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS)'	       \
> -		    EXTRA_LDFLAGS='$(SAN_LDFLAGS)'			       \
> +		    EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS) $(EXTRA_CFLAGS)' \
> +		    EXTRA_LDFLAGS='$(SAN_LDFLAGS) $(EXTRA_LDFLAGS)'	       \
>  		    DESTDIR=$(SCRATCH_DIR) prefix= all install_headers
>  
>  ifneq ($(BPFOBJ),$(HOST_BPFOBJ))
> @@ -397,7 +403,9 @@ $(HOST_BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile)		       \
>  		$(APIDIR)/linux/bpf.h					       \
>  		| $(HOST_BUILD_DIR)/libbpf
>  	$(Q)$(MAKE) $(submake_extras) -C $(BPFDIR)                             \
> -		    EXTRA_CFLAGS='-g $(OPT_FLAGS)' ARCH= CROSS_COMPILE=	       \
> +		    ARCH= CROSS_COMPILE=				       \
> +		    EXTRA_CFLAGS='-g $(OPT_FLAGS) $(EXTRA_CFLAGS)'	       \
> +		    EXTRA_LDFLAGS='$(EXTRA_LDFLAGS)'			       \
>  		    OUTPUT=$(HOST_BUILD_DIR)/libbpf/			       \
>  		    CC="$(HOSTCC)" LD="$(HOSTLD)"			       \
>  		    DESTDIR=$(HOST_SCRATCH_DIR)/ prefix= all install_headers
> -- 
> 2.47.0
> 

Re: [PATCH bpf-next v2 0/3] selftests/bpf: Improve building with extra

[patchwork-bot+netdevbpf]

Hello:

This series was applied to bpf/bpf-next.git (master)
by Andrii Nakryiko <andrii@kernel.org>:

On Fri, 18 Oct 2024 08:48:58 +0200 you wrote:
> When trying to build BPF selftests with additional compiler and linker
> flags, we're running into multiple problems. This series addresses all
> of them:
> 
> - CFLAGS are not passed to sub-makes of bpftool and libbpf. This is a
>   problem when compiling with PIE as libbpf.a ends up being non-PIE and
>   cannot be linked with other binaries (patch #1).
> 
> [...]

Here is the summary with links:
  - [bpf-next,v2,1/3] selftests/bpf: Allow building with extra flags
    https://git.kernel.org/bpf/bpf-next/c/7b164c648ee2
  - [bpf-next,v2,2/3] bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile
    https://git.kernel.org/bpf/bpf-next/c/a89cf33e4e30
  - [bpf-next,v2,3/3] selftests/bpf: Disable warnings on unused flags for Clang builds
    https://git.kernel.org/bpf/bpf-next/c/832c03d644ba

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html