[PATCH bpf-next 0/3] selftests/bpf: Improve building with extra

[PATCH bpf-next 0/3] selftests/bpf: Improve building with extra

[Viktor Malik]

When trying to build BPF selftests with additional compiler and linker
flags, we're running into multiple problems. This series addresses all
of them:

- CFLAGS are not passed to sub-makes of bpftool and libbpf. This is a
  problem when compiling with PIE as libbpf.a ends up being non-PIE and
  cannot be linked with other binaries (patch #1).

- bpftool Makefile runs `llvm-config --cflags` and appends the result to
  CFLAGS. The result typically contains `-D_GNU_SOURCE` which may be
  already set in CFLAGS. That causes a compilation error (patch #2).

- Some GCC flags are not supported by Clang but there are binaries which
  are always built with Clang but reuse user-defined CFLAGS. When CFLAGS
  contain such flags, compilation fails (patch #3).

Viktor Malik (3):
  selftests/bpf: Allow building with extra flags
  bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile
  selftests/bpf: Allow ignoring some flags for Clang builds

 tools/bpf/bpftool/Makefile           |  8 ++++++-
 tools/testing/selftests/bpf/Makefile | 31 +++++++++++++++++++---------
 2 files changed, 28 insertions(+), 11 deletions(-)

-- 
2.47.0

[PATCH bpf-next 1/3] selftests/bpf: Allow building with extra flags

[Viktor Malik]

In order to specify extra compilation or linking flags to BPF selftests,
it is possible to set EXTRA_CFLAGS and EXTRA_LDFLAGS from the command
line. The problem is that they are not propagated to sub-make calls
(runqslower, bpftool, libbpf) and in the better case are not applied, in
the worse case cause the entire build fail.

Propagate EXTRA_CFLAGS and EXTRA_LDFLAGS to the sub-makes.

This, for instance, allows to build selftests as PIE with

    $ make EXTRA_CFLAGS='-fPIE' EXTRA_LDFLAGS='-pie'

Without this change, the command would fail because libbpf.a would not
be built with -fPIE and other PIE binaries would not link against it.

The only problem is that we have to explicitly provide empty
EXTRA_CFLAGS='' and EXTRA_LDFLAGS='' to the builds of kernel modules
(bpf_testmod and bpf_test_no_cfi) as we don't want to build modules with
flags used for userspace (the above example would fail as kernel doesn't
support PIE).

Signed-off-by: Viktor Malik <vmalik@redhat.com>
---
 tools/testing/selftests/bpf/Makefile | 22 ++++++++++++++--------
 1 file changed, 14 insertions(+), 8 deletions(-)

diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 28a76baa854d..d81583b2aef9 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -294,13 +294,17 @@ $(OUTPUT)/sign-file: ../../../../scripts/sign-file.c
 $(OUTPUT)/bpf_testmod.ko: $(VMLINUX_BTF) $(RESOLVE_BTFIDS) $(wildcard bpf_testmod/Makefile bpf_testmod/*.[ch])
 	$(call msg,MOD,,$@)
 	$(Q)$(RM) bpf_testmod/bpf_testmod.ko # force re-compilation
-	$(Q)$(MAKE) $(submake_extras) RESOLVE_BTFIDS=$(RESOLVE_BTFIDS) -C bpf_testmod
+	$(Q)$(MAKE) $(submake_extras) -C bpf_testmod \
+		RESOLVE_BTFIDS=$(RESOLVE_BTFIDS)     \
+		EXTRA_CFLAGS='' EXTRA_LDFLAGS=''
 	$(Q)cp bpf_testmod/bpf_testmod.ko $@
 
 $(OUTPUT)/bpf_test_no_cfi.ko: $(VMLINUX_BTF) $(RESOLVE_BTFIDS) $(wildcard bpf_test_no_cfi/Makefile bpf_test_no_cfi/*.[ch])
 	$(call msg,MOD,,$@)
 	$(Q)$(RM) bpf_test_no_cfi/bpf_test_no_cfi.ko # force re-compilation
-	$(Q)$(MAKE) $(submake_extras) RESOLVE_BTFIDS=$(RESOLVE_BTFIDS) -C bpf_test_no_cfi
+	$(Q)$(MAKE) $(submake_extras) -C bpf_test_no_cfi \
+		RESOLVE_BTFIDS=$(RESOLVE_BTFIDS)	 \
+		EXTRA_CFLAGS='' EXTRA_LDFLAGS=''
 	$(Q)cp bpf_test_no_cfi/bpf_test_no_cfi.ko $@
 
 DEFAULT_BPFTOOL := $(HOST_SCRATCH_DIR)/sbin/bpftool
@@ -319,8 +323,8 @@ $(OUTPUT)/runqslower: $(BPFOBJ) | $(DEFAULT_BPFTOOL) $(RUNQSLOWER_OUTPUT)
 		    BPFTOOL_OUTPUT=$(HOST_BUILD_DIR)/bpftool/		       \
 		    BPFOBJ_OUTPUT=$(BUILD_DIR)/libbpf/			       \
 		    BPFOBJ=$(BPFOBJ) BPF_INCLUDE=$(INCLUDE_DIR)		       \
-		    EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS)'	       \
-		    EXTRA_LDFLAGS='$(SAN_LDFLAGS)' &&			       \
+		    EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS) $(EXTRA_CFLAGS)' \
+		    EXTRA_LDFLAGS='$(SAN_LDFLAGS) $(EXTRA_LDFLAGS)' &&	       \
 		    cp $(RUNQSLOWER_OUTPUT)runqslower $@
 
 TEST_GEN_PROGS_EXTENDED += $(TRUNNER_BPFTOOL)
@@ -354,7 +358,8 @@ $(DEFAULT_BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile)    \
 		    $(HOST_BPFOBJ) | $(HOST_BUILD_DIR)/bpftool
 	$(Q)$(MAKE) $(submake_extras)  -C $(BPFTOOLDIR)			       \
 		    ARCH= CROSS_COMPILE= CC="$(HOSTCC)" LD="$(HOSTLD)" 	       \
-		    EXTRA_CFLAGS='-g $(OPT_FLAGS)'			       \
+		    EXTRA_CFLAGS='-g $(OPT_FLAGS) $(EXTRA_CFLAGS)'	       \
+		    EXTRA_LDFLAGS='$(EXTRA_LDFLAGS)'			       \
 		    OUTPUT=$(HOST_BUILD_DIR)/bpftool/			       \
 		    LIBBPF_OUTPUT=$(HOST_BUILD_DIR)/libbpf/		       \
 		    LIBBPF_DESTDIR=$(HOST_SCRATCH_DIR)/			       \
@@ -365,7 +370,8 @@ $(CROSS_BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile)	\
 		    $(BPFOBJ) | $(BUILD_DIR)/bpftool
 	$(Q)$(MAKE) $(submake_extras)  -C $(BPFTOOLDIR)				\
 		    ARCH=$(ARCH) CROSS_COMPILE=$(CROSS_COMPILE)			\
-		    EXTRA_CFLAGS='-g $(OPT_FLAGS)'				\
+		    EXTRA_CFLAGS='-g $(OPT_FLAGS) $(EXTRA_CFLAGS)'		\
+		    EXTRA_LDFLAGS='$(EXTRA_LDFLAGS)'				\
 		    OUTPUT=$(BUILD_DIR)/bpftool/				\
 		    LIBBPF_OUTPUT=$(BUILD_DIR)/libbpf/				\
 		    LIBBPF_DESTDIR=$(SCRATCH_DIR)/				\
@@ -388,8 +394,8 @@ $(BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile)		       \
 	   $(APIDIR)/linux/bpf.h					       \
 	   | $(BUILD_DIR)/libbpf
 	$(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) OUTPUT=$(BUILD_DIR)/libbpf/ \
-		    EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS)'	       \
-		    EXTRA_LDFLAGS='$(SAN_LDFLAGS)'			       \
+		    EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS) $(EXTRA_CFLAGS)' \
+		    EXTRA_LDFLAGS='$(SAN_LDFLAGS) $(EXTRA_LDFLAGS)'	       \
 		    DESTDIR=$(SCRATCH_DIR) prefix= all install_headers
 
 ifneq ($(BPFOBJ),$(HOST_BPFOBJ))
-- 
2.47.0

[PATCH bpf-next 2/3] bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile

[Viktor Malik]

When building selftests with CFLAGS set via env variable, the value of
CFLAGS is propagated into bpftool Makefile (called from selftests
Makefile). This makes the compilation fail as _GNU_SOURCE is defined two
times - once from selftests Makefile (by including lib.mk) and once from
bpftool Makefile (by calling `llvm-config --cflags`):

    $ CFLAGS="" make -C tools/testing/selftests/bpf
    [...]
    CC      /bpf-next/tools/testing/selftests/bpf/tools/build/bpftool/btf.o
    <command-line>: error: "_GNU_SOURCE" redefined [-Werror]
    <command-line>: note: this is the location of the previous definition
    cc1: all warnings being treated as errors
    [...]

Let bpftool Makefile check if _GNU_SOURCE is already defined and if so,
do not let llvm-config add it again.

Signed-off-by: Viktor Malik <vmalik@redhat.com>
---
 tools/bpf/bpftool/Makefile | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
index ba927379eb20..2b5a713d71d8 100644
--- a/tools/bpf/bpftool/Makefile
+++ b/tools/bpf/bpftool/Makefile
@@ -147,7 +147,13 @@ ifeq ($(feature-llvm),1)
   # If LLVM is available, use it for JIT disassembly
   CFLAGS  += -DHAVE_LLVM_SUPPORT
   LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets
-  CFLAGS  += $(shell $(LLVM_CONFIG) --cflags)
+  # When bpftool build is called from another Makefile which already sets
+  # -D_GNU_SOURCE, do not let llvm-config add it again as it will cause conflict.
+  ifneq ($(filter -D_GNU_SOURCE=,$(CFLAGS)),)
+    CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))
+  else
+    CFLAGS += $(shell $(LLVM_CONFIG) --cflags)
+  endif
   LIBS    += $(shell $(LLVM_CONFIG) --libs $(LLVM_CONFIG_LIB_COMPONENTS))
   ifeq ($(shell $(LLVM_CONFIG) --shared-mode),static)
     LIBS += $(shell $(LLVM_CONFIG) --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
-- 
2.47.0

[PATCH bpf-next 3/3] selftests/bpf: Allow ignoring some flags for Clang builds

[Viktor Malik]

There exist compiler flags supported by GCC but not supported by Clang
(e.g. -specs=...). Currently, these cannot be passed to BPF selftests
builds, even when building with GCC, as some binaries (urandom_read and
liburandom_read.so) are always built with Clang and the unsupported
flags make the compilation fail (as -Werror is turned on).

Add new Makefile variable CLANG_FILTEROUT_FLAGS which can be used by
users to specify which flags (from the user-provided CFLAGS or LDFLAGS)
should be filtered out for Clang invocations.

This allows to do things like:

    $ CFLAGS="-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1" \
      CLANG_FILTEROUT_FLAGS="-specs=%" \
      make -C tools/testing/selftests/bpf

Without this patch, the compilation would fail with:

    [...]
    clang: error: argument unused during compilation: '-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1' [-Werror,-Wunused-command-line-argument]
    make: *** [Makefile:273: /bpf-next/tools/testing/selftests/bpf/liburandom_read.so] Error 1
    [...]

Signed-off-by: Viktor Malik <vmalik@redhat.com>
---
 tools/testing/selftests/bpf/Makefile | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index d81583b2aef9..89662fe0470a 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -31,6 +31,11 @@ SAN_LDFLAGS	?= $(SAN_CFLAGS)
 RELEASE		?=
 OPT_FLAGS	?= $(if $(RELEASE),-O2,-O0)
 
+# Some flags supported by GCC are not supported by Clang.
+# This variable allows users to specify such flags so that they can use custom
+# CFLAGS and binaries built with Clang do not fail to build.
+CLANG_FILTEROUT_FLAGS ?=
+
 LIBELF_CFLAGS	:= $(shell $(PKG_CONFIG) libelf --cflags 2>/dev/null)
 LIBELF_LIBS	:= $(shell $(PKG_CONFIG) libelf --libs 2>/dev/null || echo -lelf)
 
@@ -271,7 +276,7 @@ endif
 $(OUTPUT)/liburandom_read.so: urandom_read_lib1.c urandom_read_lib2.c liburandom_read.map
 	$(call msg,LIB,,$@)
 	$(Q)$(CLANG) $(CLANG_TARGET_ARCH) \
-		     $(filter-out -static,$(CFLAGS) $(LDFLAGS)) \
+		     $(filter-out -static $(CLANG_FILTEROUT_FLAGS),$(CFLAGS) $(LDFLAGS)) \
 		     $(filter %.c,$^) $(filter-out -static,$(LDLIBS)) \
 		     -fuse-ld=$(LLD) -Wl,-znoseparate-code -Wl,--build-id=sha1 \
 		     -Wl,--version-script=liburandom_read.map \
@@ -280,7 +285,7 @@ $(OUTPUT)/liburandom_read.so: urandom_read_lib1.c urandom_read_lib2.c liburandom
 $(OUTPUT)/urandom_read: urandom_read.c urandom_read_aux.c $(OUTPUT)/liburandom_read.so
 	$(call msg,BINARY,,$@)
 	$(Q)$(CLANG) $(CLANG_TARGET_ARCH) \
-		     $(filter-out -static,$(CFLAGS) $(LDFLAGS)) $(filter %.c,$^) \
+		     $(filter-out -static $(CLANG_FILTEROUT_FLAGS),$(CFLAGS) $(LDFLAGS)) $(filter %.c,$^) \
 		     -lurandom_read $(filter-out -static,$(LDLIBS)) -L$(OUTPUT) \
 		     -fuse-ld=$(LLD) -Wl,-znoseparate-code -Wl,--build-id=sha1 \
 		     -Wl,-rpath=. -o $@
-- 
2.47.0

Re: [PATCH bpf-next 2/3] bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile

[Quentin Monnet]

2024-10-15 08:54 UTC+0200 ~ Viktor Malik <vmalik@redhat.com>
> When building selftests with CFLAGS set via env variable, the value of
> CFLAGS is propagated into bpftool Makefile (called from selftests
> Makefile). This makes the compilation fail as _GNU_SOURCE is defined two
> times - once from selftests Makefile (by including lib.mk) and once from
> bpftool Makefile (by calling `llvm-config --cflags`):
> 
>      $ CFLAGS="" make -C tools/testing/selftests/bpf
>      [...]
>      CC      /bpf-next/tools/testing/selftests/bpf/tools/build/bpftool/btf.o
>      <command-line>: error: "_GNU_SOURCE" redefined [-Werror]
>      <command-line>: note: this is the location of the previous definition
>      cc1: all warnings being treated as errors
>      [...]
> 
> Let bpftool Makefile check if _GNU_SOURCE is already defined and if so,
> do not let llvm-config add it again.
> 
> Signed-off-by: Viktor Malik <vmalik@redhat.com>
> ---
>   tools/bpf/bpftool/Makefile | 8 +++++++-
>   1 file changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
> index ba927379eb20..2b5a713d71d8 100644
> --- a/tools/bpf/bpftool/Makefile
> +++ b/tools/bpf/bpftool/Makefile
> @@ -147,7 +147,13 @@ ifeq ($(feature-llvm),1)
>     # If LLVM is available, use it for JIT disassembly
>     CFLAGS  += -DHAVE_LLVM_SUPPORT
>     LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets
> -  CFLAGS  += $(shell $(LLVM_CONFIG) --cflags)
> +  # When bpftool build is called from another Makefile which already sets
> +  # -D_GNU_SOURCE, do not let llvm-config add it again as it will cause conflict.


Thanks! Can you please make your comment more explicit and mention the 
file tools/testing/selftests/lib.mk and/or the use case addressed 
(building bpftool from selftests), given that you match on the exact 
string "-D_GNU_SOURCE="? Your check won't skip adding the duplicate 
definition if someone passes "-D_GNU_SOURCE", without the "=", by 
calling the Makefile from another path; that's fine, but I don't want 
users to read the Makefile and expect it to remove the second definition 
in such a case.


> +  ifneq ($(filter -D_GNU_SOURCE=,$(CFLAGS)),)
> +    CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))
> +  else
> +    CFLAGS += $(shell $(LLVM_CONFIG) --cflags)
> +  endif

Looks good otherwise:

Reviewed-by: Quentin Monnet <qmo@kernel.org>

Re: [PATCH bpf-next 2/3] bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile

[Viktor Malik]

On 10/15/24 12:03, Quentin Monnet wrote:
> 2024-10-15 08:54 UTC+0200 ~ Viktor Malik <vmalik@redhat.com>
>> When building selftests with CFLAGS set via env variable, the value of
>> CFLAGS is propagated into bpftool Makefile (called from selftests
>> Makefile). This makes the compilation fail as _GNU_SOURCE is defined two
>> times - once from selftests Makefile (by including lib.mk) and once from
>> bpftool Makefile (by calling `llvm-config --cflags`):
>>
>>      $ CFLAGS="" make -C tools/testing/selftests/bpf
>>      [...]
>>      CC      /bpf-next/tools/testing/selftests/bpf/tools/build/bpftool/btf.o
>>      <command-line>: error: "_GNU_SOURCE" redefined [-Werror]
>>      <command-line>: note: this is the location of the previous definition
>>      cc1: all warnings being treated as errors
>>      [...]
>>
>> Let bpftool Makefile check if _GNU_SOURCE is already defined and if so,
>> do not let llvm-config add it again.
>>
>> Signed-off-by: Viktor Malik <vmalik@redhat.com>
>> ---
>>   tools/bpf/bpftool/Makefile | 8 +++++++-
>>   1 file changed, 7 insertions(+), 1 deletion(-)
>>
>> diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
>> index ba927379eb20..2b5a713d71d8 100644
>> --- a/tools/bpf/bpftool/Makefile
>> +++ b/tools/bpf/bpftool/Makefile
>> @@ -147,7 +147,13 @@ ifeq ($(feature-llvm),1)
>>     # If LLVM is available, use it for JIT disassembly
>>     CFLAGS  += -DHAVE_LLVM_SUPPORT
>>     LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets
>> -  CFLAGS  += $(shell $(LLVM_CONFIG) --cflags)
>> +  # When bpftool build is called from another Makefile which already sets
>> +  # -D_GNU_SOURCE, do not let llvm-config add it again as it will cause conflict.
> 
> 
> Thanks! Can you please make your comment more explicit and mention the 
> file tools/testing/selftests/lib.mk and/or the use case addressed 
> (building bpftool from selftests), given that you match on the exact 
> string "-D_GNU_SOURCE="? Your check won't skip adding the duplicate 
> definition if someone passes "-D_GNU_SOURCE", without the "=", by 
> calling the Makefile from another path; that's fine, but I don't want 
> users to read the Makefile and expect it to remove the second definition 
> in such a case.

Right, that's a good point, I'll expand the comment.

Thanks!

> 
> 
>> +  ifneq ($(filter -D_GNU_SOURCE=,$(CFLAGS)),)
>> +    CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))
>> +  else
>> +    CFLAGS += $(shell $(LLVM_CONFIG) --cflags)
>> +  endif
> 
> Looks good otherwise:
> 
> Reviewed-by: Quentin Monnet <qmo@kernel.org>
> 

Re: [PATCH bpf-next 2/3] bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile

[Andrii Nakryiko]

On Mon, Oct 14, 2024 at 11:55 PM Viktor Malik <vmalik@redhat.com> wrote:
>
> When building selftests with CFLAGS set via env variable, the value of
> CFLAGS is propagated into bpftool Makefile (called from selftests
> Makefile). This makes the compilation fail as _GNU_SOURCE is defined two
> times - once from selftests Makefile (by including lib.mk) and once from
> bpftool Makefile (by calling `llvm-config --cflags`):
>
>     $ CFLAGS="" make -C tools/testing/selftests/bpf
>     [...]
>     CC      /bpf-next/tools/testing/selftests/bpf/tools/build/bpftool/btf.o
>     <command-line>: error: "_GNU_SOURCE" redefined [-Werror]
>     <command-line>: note: this is the location of the previous definition
>     cc1: all warnings being treated as errors
>     [...]
>
> Let bpftool Makefile check if _GNU_SOURCE is already defined and if so,
> do not let llvm-config add it again.
>
> Signed-off-by: Viktor Malik <vmalik@redhat.com>
> ---
>  tools/bpf/bpftool/Makefile | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
> index ba927379eb20..2b5a713d71d8 100644
> --- a/tools/bpf/bpftool/Makefile
> +++ b/tools/bpf/bpftool/Makefile
> @@ -147,7 +147,13 @@ ifeq ($(feature-llvm),1)
>    # If LLVM is available, use it for JIT disassembly
>    CFLAGS  += -DHAVE_LLVM_SUPPORT
>    LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets
> -  CFLAGS  += $(shell $(LLVM_CONFIG) --cflags)
> +  # When bpftool build is called from another Makefile which already sets
> +  # -D_GNU_SOURCE, do not let llvm-config add it again as it will cause conflict.
> +  ifneq ($(filter -D_GNU_SOURCE=,$(CFLAGS)),)
> +    CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))

why not always do filter-out and avoid this ugly ifneq?

> +  else
> +    CFLAGS += $(shell $(LLVM_CONFIG) --cflags)
> +  endif
>    LIBS    += $(shell $(LLVM_CONFIG) --libs $(LLVM_CONFIG_LIB_COMPONENTS))
>    ifeq ($(shell $(LLVM_CONFIG) --shared-mode),static)
>      LIBS += $(shell $(LLVM_CONFIG) --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
> --
> 2.47.0
>

Re: [PATCH bpf-next 3/3] selftests/bpf: Allow ignoring some flags for Clang builds

[Andrii Nakryiko]

On Mon, Oct 14, 2024 at 11:55 PM Viktor Malik <vmalik@redhat.com> wrote:
>
> There exist compiler flags supported by GCC but not supported by Clang
> (e.g. -specs=...). Currently, these cannot be passed to BPF selftests
> builds, even when building with GCC, as some binaries (urandom_read and
> liburandom_read.so) are always built with Clang and the unsupported
> flags make the compilation fail (as -Werror is turned on).
>
> Add new Makefile variable CLANG_FILTEROUT_FLAGS which can be used by
> users to specify which flags (from the user-provided CFLAGS or LDFLAGS)
> should be filtered out for Clang invocations.
>
> This allows to do things like:
>
>     $ CFLAGS="-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1" \
>       CLANG_FILTEROUT_FLAGS="-specs=%" \
>       make -C tools/testing/selftests/bpf
>
> Without this patch, the compilation would fail with:
>
>     [...]
>     clang: error: argument unused during compilation: '-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1' [-Werror,-Wunused-command-line-argument]

maybe we should just not error out (i.e., enable
-Wno-unused-command-line-argument)?

>     make: *** [Makefile:273: /bpf-next/tools/testing/selftests/bpf/liburandom_read.so] Error 1
>     [...]
>
> Signed-off-by: Viktor Malik <vmalik@redhat.com>
> ---
>  tools/testing/selftests/bpf/Makefile | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index d81583b2aef9..89662fe0470a 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -31,6 +31,11 @@ SAN_LDFLAGS  ?= $(SAN_CFLAGS)
>  RELEASE                ?=
>  OPT_FLAGS      ?= $(if $(RELEASE),-O2,-O0)
>
> +# Some flags supported by GCC are not supported by Clang.
> +# This variable allows users to specify such flags so that they can use custom
> +# CFLAGS and binaries built with Clang do not fail to build.
> +CLANG_FILTEROUT_FLAGS ?=
> +
>  LIBELF_CFLAGS  := $(shell $(PKG_CONFIG) libelf --cflags 2>/dev/null)
>  LIBELF_LIBS    := $(shell $(PKG_CONFIG) libelf --libs 2>/dev/null || echo -lelf)
>
> @@ -271,7 +276,7 @@ endif
>  $(OUTPUT)/liburandom_read.so: urandom_read_lib1.c urandom_read_lib2.c liburandom_read.map
>         $(call msg,LIB,,$@)
>         $(Q)$(CLANG) $(CLANG_TARGET_ARCH) \
> -                    $(filter-out -static,$(CFLAGS) $(LDFLAGS)) \
> +                    $(filter-out -static $(CLANG_FILTEROUT_FLAGS),$(CFLAGS) $(LDFLAGS)) \
>                      $(filter %.c,$^) $(filter-out -static,$(LDLIBS)) \
>                      -fuse-ld=$(LLD) -Wl,-znoseparate-code -Wl,--build-id=sha1 \
>                      -Wl,--version-script=liburandom_read.map \
> @@ -280,7 +285,7 @@ $(OUTPUT)/liburandom_read.so: urandom_read_lib1.c urandom_read_lib2.c liburandom
>  $(OUTPUT)/urandom_read: urandom_read.c urandom_read_aux.c $(OUTPUT)/liburandom_read.so
>         $(call msg,BINARY,,$@)
>         $(Q)$(CLANG) $(CLANG_TARGET_ARCH) \
> -                    $(filter-out -static,$(CFLAGS) $(LDFLAGS)) $(filter %.c,$^) \
> +                    $(filter-out -static $(CLANG_FILTEROUT_FLAGS),$(CFLAGS) $(LDFLAGS)) $(filter %.c,$^) \
>                      -lurandom_read $(filter-out -static,$(LDLIBS)) -L$(OUTPUT) \
>                      -fuse-ld=$(LLD) -Wl,-znoseparate-code -Wl,--build-id=sha1 \
>                      -Wl,-rpath=. -o $@
> --
> 2.47.0
>

Re: [PATCH bpf-next 3/3] selftests/bpf: Allow ignoring some flags for Clang builds

[Eduard Zingerman]

On Wed, 2024-10-16 at 13:37 -0700, Andrii Nakryiko wrote:
> On Mon, Oct 14, 2024 at 11:55 PM Viktor Malik <vmalik@redhat.com> wrote:
> > 
> > There exist compiler flags supported by GCC but not supported by Clang
> > (e.g. -specs=...). Currently, these cannot be passed to BPF selftests
> > builds, even when building with GCC, as some binaries (urandom_read and
> > liburandom_read.so) are always built with Clang and the unsupported
> > flags make the compilation fail (as -Werror is turned on).
> > 
> > Add new Makefile variable CLANG_FILTEROUT_FLAGS which can be used by
> > users to specify which flags (from the user-provided CFLAGS or LDFLAGS)
> > should be filtered out for Clang invocations.
> > 
> > This allows to do things like:
> > 
> >     $ CFLAGS="-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1" \
> >       CLANG_FILTEROUT_FLAGS="-specs=%" \
> >       make -C tools/testing/selftests/bpf
> > 
> > Without this patch, the compilation would fail with:
> > 
> >     [...]
> >     clang: error: argument unused during compilation: '-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1' [-Werror,-Wunused-command-line-argument]
> 
> maybe we should just not error out (i.e., enable
> -Wno-unused-command-line-argument)?

I agree with Andrii, grepping for FILTEROUT in kernel source code does
not show anything similar to this. Are such filter-out variables some
kind of convention?

Another option might be to remove `-Werror` and add it on CI via EXTRA_CFLAGS.

[...]

Re: [PATCH bpf-next 1/3] selftests/bpf: Allow building with extra flags

[Eduard Zingerman]

On Tue, 2024-10-15 at 08:54 +0200, Viktor Malik wrote:

Tried this, seems to work as expected, but see below.

Tested-by: Eduard Zingerman <eddyz87@gmail.com>

[...]

> @@ -388,8 +394,8 @@ $(BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile)		       \
>  	   $(APIDIR)/linux/bpf.h					       \
>  	   | $(BUILD_DIR)/libbpf
>  	$(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) OUTPUT=$(BUILD_DIR)/libbpf/ \
> -		    EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS)'	       \
> -		    EXTRA_LDFLAGS='$(SAN_LDFLAGS)'			       \
> +		    EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS) $(EXTRA_CFLAGS)' \
> +		    EXTRA_LDFLAGS='$(SAN_LDFLAGS) $(EXTRA_LDFLAGS)'	       \
>  		    DESTDIR=$(SCRATCH_DIR) prefix= all install_headers
>  
>  ifneq ($(BPFOBJ),$(HOST_BPFOBJ))

Note, that there is also the following code just below this hunk:

ifneq ($(BPFOBJ),$(HOST_BPFOBJ))
$(HOST_BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile)		       \
		$(APIDIR)/linux/bpf.h					       \
		| $(HOST_BUILD_DIR)/libbpf
	$(Q)$(MAKE) $(submake_extras) -C $(BPFDIR)                             \
		    EXTRA_CFLAGS='-g $(OPT_FLAGS)' ARCH= CROSS_COMPILE=	       \
                    ^^^^^^^^^^^^
                    needs an update?
                    
		    OUTPUT=$(HOST_BUILD_DIR)/libbpf/			       \
		    CC="$(HOSTCC)" LD="$(HOSTLD)"			       \
		    DESTDIR=$(HOST_SCRATCH_DIR)/ prefix= all install_headers
endif

Re: [PATCH bpf-next 2/3] bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile

[Viktor Malik]

On 10/16/24 22:34, Andrii Nakryiko wrote:
> On Mon, Oct 14, 2024 at 11:55 PM Viktor Malik <vmalik@redhat.com> wrote:
>>
>> When building selftests with CFLAGS set via env variable, the value of
>> CFLAGS is propagated into bpftool Makefile (called from selftests
>> Makefile). This makes the compilation fail as _GNU_SOURCE is defined two
>> times - once from selftests Makefile (by including lib.mk) and once from
>> bpftool Makefile (by calling `llvm-config --cflags`):
>>
>>     $ CFLAGS="" make -C tools/testing/selftests/bpf
>>     [...]
>>     CC      /bpf-next/tools/testing/selftests/bpf/tools/build/bpftool/btf.o
>>     <command-line>: error: "_GNU_SOURCE" redefined [-Werror]
>>     <command-line>: note: this is the location of the previous definition
>>     cc1: all warnings being treated as errors
>>     [...]
>>
>> Let bpftool Makefile check if _GNU_SOURCE is already defined and if so,
>> do not let llvm-config add it again.
>>
>> Signed-off-by: Viktor Malik <vmalik@redhat.com>
>> ---
>>  tools/bpf/bpftool/Makefile | 8 +++++++-
>>  1 file changed, 7 insertions(+), 1 deletion(-)
>>
>> diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
>> index ba927379eb20..2b5a713d71d8 100644
>> --- a/tools/bpf/bpftool/Makefile
>> +++ b/tools/bpf/bpftool/Makefile
>> @@ -147,7 +147,13 @@ ifeq ($(feature-llvm),1)
>>    # If LLVM is available, use it for JIT disassembly
>>    CFLAGS  += -DHAVE_LLVM_SUPPORT
>>    LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets
>> -  CFLAGS  += $(shell $(LLVM_CONFIG) --cflags)
>> +  # When bpftool build is called from another Makefile which already sets
>> +  # -D_GNU_SOURCE, do not let llvm-config add it again as it will cause conflict.
>> +  ifneq ($(filter -D_GNU_SOURCE=,$(CFLAGS)),)
>> +    CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))
> 
> why not always do filter-out and avoid this ugly ifneq?

Because in that case, _GNU_SOURCE would not be defined for some builds
(e.g. plain bpftool build). I'm not entirely sure what the implications
are so I wanted to stay on the safe side. Anyways, I gave it a try and
bpftool builds without _GNU_SOURCE just fine so I think that we can drop
the ifneq.

> 
>> +  else
>> +    CFLAGS += $(shell $(LLVM_CONFIG) --cflags)
>> +  endif
>>    LIBS    += $(shell $(LLVM_CONFIG) --libs $(LLVM_CONFIG_LIB_COMPONENTS))
>>    ifeq ($(shell $(LLVM_CONFIG) --shared-mode),static)
>>      LIBS += $(shell $(LLVM_CONFIG) --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
>> --
>> 2.47.0
>>
> 

Re: [PATCH bpf-next 1/3] selftests/bpf: Allow building with extra flags

[Viktor Malik]

On 10/17/24 01:17, Eduard Zingerman wrote:
> On Tue, 2024-10-15 at 08:54 +0200, Viktor Malik wrote:
> 
> Tried this, seems to work as expected, but see below.
> 
> Tested-by: Eduard Zingerman <eddyz87@gmail.com>
> 
> [...]
> 
>> @@ -388,8 +394,8 @@ $(BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile)		       \
>>  	   $(APIDIR)/linux/bpf.h					       \
>>  	   | $(BUILD_DIR)/libbpf
>>  	$(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) OUTPUT=$(BUILD_DIR)/libbpf/ \
>> -		    EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS)'	       \
>> -		    EXTRA_LDFLAGS='$(SAN_LDFLAGS)'			       \
>> +		    EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS) $(EXTRA_CFLAGS)' \
>> +		    EXTRA_LDFLAGS='$(SAN_LDFLAGS) $(EXTRA_LDFLAGS)'	       \
>>  		    DESTDIR=$(SCRATCH_DIR) prefix= all install_headers
>>  
>>  ifneq ($(BPFOBJ),$(HOST_BPFOBJ))
> 
> Note, that there is also the following code just below this hunk:
> 
> ifneq ($(BPFOBJ),$(HOST_BPFOBJ))
> $(HOST_BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile)		       \
> 		$(APIDIR)/linux/bpf.h					       \
> 		| $(HOST_BUILD_DIR)/libbpf
> 	$(Q)$(MAKE) $(submake_extras) -C $(BPFDIR)                             \
> 		    EXTRA_CFLAGS='-g $(OPT_FLAGS)' ARCH= CROSS_COMPILE=	       \
>                     ^^^^^^^^^^^^
>                     needs an update?

Yes, good catch, I'll add it.

Thanks!

>                     
> 		    OUTPUT=$(HOST_BUILD_DIR)/libbpf/			       \
> 		    CC="$(HOSTCC)" LD="$(HOSTLD)"			       \
> 		    DESTDIR=$(HOST_SCRATCH_DIR)/ prefix= all install_headers
> endif
> 
> 

Re: [PATCH bpf-next 3/3] selftests/bpf: Allow ignoring some flags for Clang builds

[Viktor Malik]

On 10/17/24 00:18, Eduard Zingerman wrote:
> On Wed, 2024-10-16 at 13:37 -0700, Andrii Nakryiko wrote:
>> On Mon, Oct 14, 2024 at 11:55 PM Viktor Malik <vmalik@redhat.com> wrote:
>>>
>>> There exist compiler flags supported by GCC but not supported by Clang
>>> (e.g. -specs=...). Currently, these cannot be passed to BPF selftests
>>> builds, even when building with GCC, as some binaries (urandom_read and
>>> liburandom_read.so) are always built with Clang and the unsupported
>>> flags make the compilation fail (as -Werror is turned on).
>>>
>>> Add new Makefile variable CLANG_FILTEROUT_FLAGS which can be used by
>>> users to specify which flags (from the user-provided CFLAGS or LDFLAGS)
>>> should be filtered out for Clang invocations.
>>>
>>> This allows to do things like:
>>>
>>>     $ CFLAGS="-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1" \
>>>       CLANG_FILTEROUT_FLAGS="-specs=%" \
>>>       make -C tools/testing/selftests/bpf
>>>
>>> Without this patch, the compilation would fail with:
>>>
>>>     [...]
>>>     clang: error: argument unused during compilation: '-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1' [-Werror,-Wunused-command-line-argument]
>>
>> maybe we should just not error out (i.e., enable
>> -Wno-unused-command-line-argument)?
> 
> I agree with Andrii, grepping for FILTEROUT in kernel source code does
> not show anything similar to this. Are such filter-out variables some
> kind of convention?
> 
> Another option might be to remove `-Werror` and add it on CI via EXTRA_CFLAGS.

Enabling -Wno-unused-command-line-argument is the simplest way here,
let's do that.

Thank you both for suggestions!

> 
> [...]
>