[PATCH bpf-next] selftests/bpf: Tolerate libarena skeleton failures under BPF_STRICT_BUILD=0

[PATCH bpf-next] selftests/bpf: Tolerate libarena skeleton failures under BPF_STRICT_BUILD=0

[HyeongJun An]

The Makefile documents BPF_STRICT_BUILD=0 as tolerating any BPF object,
skeleton, test object or benchmark failure so that the remaining tests
still build.  Every skeleton rule honours that through
$(if $(PERMISSIVE),...), except the three that build the libarena
skeletons.  Those invoke a sub-make with no guard, so a libarena failure
is fatal even in permissive mode.

With a libarena source that fails to compile, BPF_STRICT_BUILD=0 stops at

  make: *** [libarena/libarena.skel.h] Error 2

while an ordinary program failing the same way prints SKIP-BPF and the
build carries on.

Guard the three sub-makes the way the other skeleton rules are guarded.
The build then prints SKIP-SKEL and continues, and test_progs,
test_progs-no_alu32, test_progs-cpuv4 and bench all link.  Nothing
downstream needs a change, because the missing skeleton drops out through
the existing permissive paths and libarena.test.o is reported as
SKIP-TEST.  The default BPF_STRICT_BUILD=1 still fails on the same input.

Fixes: a6850fa388f6 ("selftests/bpf: Add BPF_STRICT_BUILD toggle")
Signed-off-by: HyeongJun An <sammiee5311@gmail.com>
Assisted-by: Claude:claude-opus-5
---
 tools/testing/selftests/bpf/Makefile | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 7ea5ba1df29e..a9951774e5c8 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -835,17 +835,20 @@ LIBARENA_SKEL := libarena/libarena.skel.h
 LIBARENA_BENCH_SKEL := libarena/libarena_bench.skel.h
 
 $(LIBARENA_SKEL): $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ) $(LIBARENA_BPF_DEPS)
-	+$(MAKE) -C libarena libarena.skel.h $(LIBARENA_MAKE_ARGS)
+	+$(MAKE) -C libarena libarena.skel.h $(LIBARENA_MAKE_ARGS) $(if $(PERMISSIVE),|| \
+		printf '  %-12s %s\n' 'SKIP-SKEL' '$(notdir $@)' 1>&2)
 
 $(LIBARENA_BENCH_SKEL): $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ) $(LIBARENA_BENCH_BPF_DEPS) | $(LIBARENA_SKEL)
-	+$(MAKE) -C libarena benchmarks $(LIBARENA_MAKE_ARGS)
+	+$(MAKE) -C libarena benchmarks $(LIBARENA_MAKE_ARGS) $(if $(PERMISSIVE),|| \
+		printf '  %-12s %s\n' 'SKIP-SKEL' '$(notdir $@)' 1>&2)
 
 ifneq ($(CLANG_HAS_ARENA_ASAN),)
 LIBARENA_ASAN_SKEL := libarena/libarena_asan.skel.h
 CFLAGS += -DHAS_BPF_ARENA_ASAN
 
 $(LIBARENA_ASAN_SKEL): $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ) $(LIBARENA_BPF_DEPS)
-	+$(MAKE) -C libarena libarena_asan.skel.h $(LIBARENA_MAKE_ARGS)
+	+$(MAKE) -C libarena libarena_asan.skel.h $(LIBARENA_MAKE_ARGS) $(if $(PERMISSIVE),|| \
+		printf '  %-12s %s\n' 'SKIP-SKEL' '$(notdir $@)' 1>&2)
 endif
 
 # Define test_progs test runner.

base-commit: af0b84a9215d951d16f26b7ee34353b970cf5d4e
-- 
2.43.0

Re: [PATCH bpf-next] selftests/bpf: Tolerate libarena skeleton failures under BPF_STRICT_BUILD=0

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] The permissive fallback for libarena sub-makes does not remove the target file on failure, leaving a corrupted or empty skeleton header that permanently breaks incremental builds.
- [Medium] The `bench` executable fails to link and is completely skipped when the libarena benchmark skeleton generation fails, contradicting the commit message's claim that 'bench all link'.
--

commit 03a9a4a189e91e03941dfb620bf2f4bda1071094
Author: HyeongJun An <sammiee5311@gmail.com>

selftests/bpf: Tolerate libarena skeleton failures under BPF_STRICT_BUILD=0

This patch modifies the libarena skeleton build rules to respect
BPF_STRICT_BUILD=0, allowing the build to continue if the libarena
skeletons fail to generate. It adds permissive fallback logic to the
three libarena sub-makes.

> The build then prints SKIP-SKEL and continues, and test_progs,
> test_progs-no_alu32, test_progs-cpuv4 and bench all link.

[Severity: Medium]
Is this claim about bench linking successfully accurate when the benchmark
skeleton fails?

Looking at the bench target in the Makefile, it has a hardcoded
dependency on $(OUTPUT)/bench_libarena.o. The link command unconditionally
passes all objects to the compiler:

$(Q)$(CC) $(CFLAGS) $(LDFLAGS) $(filter %.a %.o,$^) $(LDLIBS) -o $@ ...

If the libarena benchmark skeleton fails under permissive mode,
bench_libarena.o will be missing. Because the link command doesn't
filter out skipped benchmark objects, won't this cause the entire bench
executable to fail to link and trigger SKIP-LINK instead?

> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index 7ea5ba1df29e3..a9951774e5c85 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -835,17 +835,20 @@ LIBARENA_SKEL := libarena/libarena.skel.h
>  LIBARENA_BENCH_SKEL := libarena/libarena_bench.skel.h
>  
>  $(LIBARENA_SKEL): $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ) $(LIBARENA_BPF_DEPS)
> -	+$(MAKE) -C libarena libarena.skel.h $(LIBARENA_MAKE_ARGS)
> +	+$(MAKE) -C libarena libarena.skel.h $(LIBARENA_MAKE_ARGS) $(if $(PERMISSIVE),|| \
> +		printf '  %-12s %s\n' 'SKIP-SKEL' '$(notdir $@)' 1>&2)

[Severity: High]
Does this missing cleanup leave behind a corrupted skeleton header?

In other permissive fallback rules, ($(RM) $@; printf ...) is used to
ensure the target file is removed on failure. Because shell redirection
in the sub-make might truncate the skeleton header before bpftool runs,
could an error leave a 0-byte or incomplete file behind?

If the file isn't explicitly removed here, won't subsequent incremental
builds attempt to use the corrupted skeleton header instead of rebuilding
it?

[ ... ]

>  $(LIBARENA_BENCH_SKEL): $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ) $(LIBARENA_BENCH_BPF_DEPS) | $(LIBARENA_SKEL)
> -	+$(MAKE) -C libarena benchmarks $(LIBARENA_MAKE_ARGS)
> +	+$(MAKE) -C libarena benchmarks $(LIBARENA_MAKE_ARGS) $(if $(PERMISSIVE),|| \
> +		printf '  %-12s %s\n' 'SKIP-SKEL' '$(notdir $@)' 1>&2)

[Severity: High]
Similarly, should this fallback also remove the target file with $(RM) $@ to
prevent breaking future incremental builds with a corrupted
libarena_bench.skel.h?

[ ... ]

>  ifneq ($(CLANG_HAS_ARENA_ASAN),)
>  LIBARENA_ASAN_SKEL := libarena/libarena_asan.skel.h
>  CFLAGS += -DHAS_BPF_ARENA_ASAN
>  
>  $(LIBARENA_ASAN_SKEL): $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ) $(LIBARENA_BPF_DEPS)
> -	+$(MAKE) -C libarena libarena_asan.skel.h $(LIBARENA_MAKE_ARGS)
> +	+$(MAKE) -C libarena libarena_asan.skel.h $(LIBARENA_MAKE_ARGS) $(if $(PERMISSIVE),|| \
> +		printf '  %-12s %s\n' 'SKIP-SKEL' '$(notdir $@)' 1>&2)

[Severity: High]
Does this fallback need the same target cleanup for libarena_asan.skel.h?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260911141621.1944900-1-sammiee5311@gmail.com?part=1

Re: [PATCH bpf-next] selftests/bpf: Tolerate libarena skeleton failures under BPF_STRICT_BUILD=0

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 1829 bytes --]

> The Makefile documents BPF_STRICT_BUILD=0 as tolerating any BPF object,
> skeleton, test object or benchmark failure so that the remaining tests
> still build.  Every skeleton rule honours that through
> $(if $(PERMISSIVE),...), except the three that build the libarena
> skeletons.  Those invoke a sub-make with no guard, so a libarena failure
> is fatal even in permissive mode.
>
> With a libarena source that fails to compile, BPF_STRICT_BUILD=0 stops at
>
>   make: *** [libarena/libarena.skel.h] Error 2
>
> while an ordinary program failing the same way prints SKIP-BPF and the
> build carries on.
>
> Guard the three sub-makes the way the other skeleton rules are guarded.
> The build then prints SKIP-SKEL and continues, and test_progs,
> test_progs-no_alu32, test_progs-cpuv4 and bench all link.  Nothing
> downstream needs a change, because the missing skeleton drops out through
> the existing permissive paths and libarena.test.o is reported as
> SKIP-TEST.  The default BPF_STRICT_BUILD=1 still fails on the same input.
>
> Fixes: a6850fa388f6 ("selftests/bpf: Add BPF_STRICT_BUILD toggle")
> Signed-off-by: HyeongJun An <sammiee5311@gmail.com>
> Assisted-by: Claude:claude-opus-5

The Fixes: tag points to a6850fa388f6 which introduced BPF_STRICT_BUILD,
but the inconsistency where other skeleton rules use PERMISSIVE while
libarena rules don't was actually introduced by commit 7e729028d7ff. That
commit added PERMISSIVE guards to most skeleton generation rules but
missed the three libarena skeleton rules. Should the tag be:

  Fixes: 7e729028d7ff ("adding ci files")


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34611055093

Re: [PATCH bpf-next] selftests/bpf: Tolerate libarena skeleton failures under BPF_STRICT_BUILD=0

[Ricardo B. Marlière (SUSE)]

On Fri Sep 11, 2026 at 11:16 AM -03, HyeongJun An wrote:
> The Makefile documents BPF_STRICT_BUILD=0 as tolerating any BPF object,
> skeleton, test object or benchmark failure so that the remaining tests
> still build.  Every skeleton rule honours that through
> $(if $(PERMISSIVE),...), except the three that build the libarena
> skeletons.  Those invoke a sub-make with no guard, so a libarena failure
> is fatal even in permissive mode.
>
> With a libarena source that fails to compile, BPF_STRICT_BUILD=0 stops at

Under which circumstances does this happen to you? I missed this because I
was using defconfig + DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT + DEBUG_INFO_BTF +
BPF_SYSCALL + BPF_JIT

>
>   make: *** [libarena/libarena.skel.h] Error 2
>
> while an ordinary program failing the same way prints SKIP-BPF and the
> build carries on.
>
> Guard the three sub-makes the way the other skeleton rules are guarded.
> The build then prints SKIP-SKEL and continues, and test_progs,
> test_progs-no_alu32, test_progs-cpuv4 and bench all link.  Nothing
> downstream needs a change, because the missing skeleton drops out through
> the existing permissive paths and libarena.test.o is reported as
> SKIP-TEST.  The default BPF_STRICT_BUILD=1 still fails on the same input.
>
> Fixes: a6850fa388f6 ("selftests/bpf: Add BPF_STRICT_BUILD toggle")

Acked-by: Ricardo B. Marlière (SUSE) <ricardo@marliere.net>

> Signed-off-by: HyeongJun An <sammiee5311@gmail.com>
> Assisted-by: Claude:claude-opus-5
> ---
>  tools/testing/selftests/bpf/Makefile | 9 ++++++---
>  1 file changed, 6 insertions(+), 3 deletions(-)
>
> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index 7ea5ba1df29e..a9951774e5c8 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -835,17 +835,20 @@ LIBARENA_SKEL := libarena/libarena.skel.h
>  LIBARENA_BENCH_SKEL := libarena/libarena_bench.skel.h
>  
>  $(LIBARENA_SKEL): $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ) $(LIBARENA_BPF_DEPS)
> -	+$(MAKE) -C libarena libarena.skel.h $(LIBARENA_MAKE_ARGS)
> +	+$(MAKE) -C libarena libarena.skel.h $(LIBARENA_MAKE_ARGS) $(if $(PERMISSIVE),|| \
> +		printf '  %-12s %s\n' 'SKIP-SKEL' '$(notdir $@)' 1>&2)
>  
>  $(LIBARENA_BENCH_SKEL): $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ) $(LIBARENA_BENCH_BPF_DEPS) | $(LIBARENA_SKEL)
> -	+$(MAKE) -C libarena benchmarks $(LIBARENA_MAKE_ARGS)
> +	+$(MAKE) -C libarena benchmarks $(LIBARENA_MAKE_ARGS) $(if $(PERMISSIVE),|| \
> +		printf '  %-12s %s\n' 'SKIP-SKEL' '$(notdir $@)' 1>&2)
>  
>  ifneq ($(CLANG_HAS_ARENA_ASAN),)
>  LIBARENA_ASAN_SKEL := libarena/libarena_asan.skel.h
>  CFLAGS += -DHAS_BPF_ARENA_ASAN
>  
>  $(LIBARENA_ASAN_SKEL): $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ) $(LIBARENA_BPF_DEPS)
> -	+$(MAKE) -C libarena libarena_asan.skel.h $(LIBARENA_MAKE_ARGS)
> +	+$(MAKE) -C libarena libarena_asan.skel.h $(LIBARENA_MAKE_ARGS) $(if $(PERMISSIVE),|| \
> +		printf '  %-12s %s\n' 'SKIP-SKEL' '$(notdir $@)' 1>&2)
>  endif
>  
>  # Define test_progs test runner.
>
> base-commit: af0b84a9215d951d16f26b7ee34353b970cf5d4e

Re: [PATCH bpf-next] selftests/bpf: Tolerate libarena skeleton failures under BPF_STRICT_BUILD=0

[Eduard Zingerman]

On Fri, 2026-09-11 at 23:16 +0900, HyeongJun An wrote:
> The Makefile documents BPF_STRICT_BUILD=0 as tolerating any BPF object,
> skeleton, test object or benchmark failure so that the remaining tests
> still build.  Every skeleton rule honours that through
> $(if $(PERMISSIVE),...), except the three that build the libarena
> skeletons.  Those invoke a sub-make with no guard, so a libarena failure
> is fatal even in permissive mode.
> 
> With a libarena source that fails to compile, BPF_STRICT_BUILD=0 stops at
> 
>   make: *** [libarena/libarena.skel.h] Error 2
> 
> while an ordinary program failing the same way prints SKIP-BPF and the
> build carries on.
> 
> Guard the three sub-makes the way the other skeleton rules are guarded.
> The build then prints SKIP-SKEL and continues, and test_progs,
> test_progs-no_alu32, test_progs-cpuv4 and bench all link.  Nothing
> downstream needs a change, because the missing skeleton drops out through
> the existing permissive paths and libarena.test.o is reported as
> SKIP-TEST.  The default BPF_STRICT_BUILD=1 still fails on the same input.
> 
> Fixes: a6850fa388f6 ("selftests/bpf: Add BPF_STRICT_BUILD toggle")
> Signed-off-by: HyeongJun An <sammiee5311@gmail.com>
> Assisted-by: Claude:claude-opus-5
> ---

Why is this change necessary?
The whole PERMISSIVE thing is quite ugly and I regret not being
involved in it's review.

...

Re: [PATCH bpf-next] selftests/bpf: Tolerate libarena skeleton failures under BPF_STRICT_BUILD=0

[HyeongJun An]

On Fri Sep 11, 2026 at 03:02:35PM -0300, Ricardo B. Marlière (SUSE) wrote:

> Under which circumstances does this happen to you? I missed this because I
> was using defconfig + DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT + DEBUG_INFO_BTF +
> BPF_SYSCALL + BPF_JIT

Building against the distro kernel's BTF rather than one from the tree.
On Ubuntu 6.17, /sys/kernel/btf/vmlinux carries no bpf_kfunc DECL_TAGs,
so the vmlinux.h generated from it has 5 __ksym declarations where a
tree-built kernel gives 269, and all six libarena BPF sources fail on
undeclared bpf_preempt_disable, bpf_preempt_enable and
bpf_stream_vprintk. It seems that is why you did not run into it with a
kernel you built yourself.

Thanks for the ack. I checked what the two bots raised and they are both
right, including the Fixes: tag, so the patch needs more work than this.

HyeongJun

Re: [PATCH bpf-next] selftests/bpf: Tolerate libarena skeleton failures under BPF_STRICT_BUILD=0

[HyeongJun An]

On Fri, 2026-09-11 at 13:24 -0700, Eduard Zingerman wrote:

> Why is this change necessary?
> The whole PERMISSIVE thing is quite ugly and I regret not being
> involved in it's review.

I hit it building selftests against the distro kernel's BTF rather than
one from the tree. On Ubuntu 6.17 that vmlinux.h has 5 __ksym
declarations against 269 from a tree-built kernel, and all six libarena
sources fail on undeclared bpf_preempt_disable, bpf_preempt_enable and
bpf_stream_vprintk. BPF_STRICT_BUILD=0 looked like the switch for that,
and the build still stopped at libarena while every other skeleton was
skipped.

The patch only makes the three libarena rules consistent with the other
skeleton rules. I don't have a strong opinion on PERMISSIVE itself, so if
it is going away or being reworked, please drop this rather than take it.

HyeongJun