Re: [PATCH bpf-next v5 03/11] selftests/bpf: Tolerate BPF and skeleton generation failures

[sashiko-bot@kernel.org]

Sashiko AI review found 2 potential issue(s):
- [High] Silent masking of skeleton generation failures when PERMISSIVE is disabled (strict mode).
- [High] Stale skeleton artifacts are left on disk on incremental builds when prerequisite BPF objects fail to compile.
--

commit 3cf9cc67632ce30dc796d085a5a01c842575cf10
Author: Ricardo B. Marlière <rbm@suse.com>

selftests/bpf: Tolerate BPF and skeleton generation failures

Some BPF programs cannot be built on distro kernels because required BTF
types or features are missing. Make BPF object and skeleton generation best
effort: emit SKIP-BPF or SKIP-SKEL, remove failed outputs so downstream rules
can detect absence, and continue with remaining tests.

> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index 5c4cdf1dafc8b..a305d13409e4b 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
[ ... ]
> @@ -598,47 +602,77 @@ $(TRUNNER_BPF_OBJS): $(TRUNNER_OUTPUT)/%.bpf.o:				\
>  					  $$($$<-$2-CFLAGS),$(TRUNNER_BINARY))
>  
>  $(TRUNNER_BPF_SKELS): %.skel.h: %.bpf.o $(BPFTOOL) | $(TRUNNER_OUTPUT)
> -	$$(call msg,GEN-SKEL,$(TRUNNER_BINARY),$$@)
> -	$(Q)$$(BPFTOOL) gen object $$(<:.o=.linked1.o) $$<
> -	$(Q)$$(BPFTOOL) gen object $$(<:.o=.linked2.o) $$(<:.o=.linked1.o)
> -	$(Q)$$(BPFTOOL) gen object $$(<:.o=.linked3.o) $$(<:.o=.linked2.o)
> -	$(Q)diff $$(<:.o=.linked2.o) $$(<:.o=.linked3.o)
> -	$(Q)$$(BPFTOOL) gen skeleton $$(<:.o=.linked3.o) name $$(notdir $$(<:.bpf.o=)) > $$@
> -	$(Q)$$(BPFTOOL) gen subskeleton $$(<:.o=.linked3.o) name $$(notdir $$(<:.bpf.o=)) > $$(@:.skel.h=.subskel.h)
> -	$(Q)rm -f $$(<:.o=.linked1.o) $$(<:.o=.linked2.o) $$(<:.o=.linked3.o)
> +	$(Q)$(if $(PERMISSIVE),if [ ! -f $$< ]; then			\
> +		printf '  %-12s %s\n' 'SKIP-SKEL' '$$(notdir $$@)' 1>&2; \
> +		exit 0;							\
> +	fi;)								\

If $$< is missing during an incremental build because a previous permissive run
failed to compile it, does exiting 0 here leave a stale .skel.h on disk?

Since $@ is not removed before exiting, it appears downstream targets might
incorrectly compile against the stale skeleton instead of failing or skipping.

> +	printf '  %-12s %s\n' 'GEN-SKEL' '[$(TRUNNER_BINARY)] $$(notdir $$@)' 1>&2; \
> +	$$(BPFTOOL) gen object $$(<:.o=.linked1.o) $$< &&		\
> +	$$(BPFTOOL) gen object $$(<:.o=.linked2.o) $$(<:.o=.linked1.o) && \
> +	$$(BPFTOOL) gen object $$(<:.o=.linked3.o) $$(<:.o=.linked2.o) && \
> +	diff $$(<:.o=.linked2.o) $$(<:.o=.linked3.o) &&		\
> +	$$(BPFTOOL) gen skeleton $$(<:.o=.linked3.o) name $$(notdir $$(<:.bpf.o=)) > $$@ && \
> +	$$(BPFTOOL) gen subskeleton $$(<:.o=.linked3.o) name $$(notdir $$(<:.bpf.o=)) > $$(@:.skel.h=.subskel.h) $(if $(PERMISSIVE),|| { \
> +		$$(RM) $$@ $$(@:.skel.h=.subskel.h); \
> +		printf '  %-12s %s\n' 'SKIP-SKEL' '$$(notdir $$@)' 1>&2; \
> +	}); \
> +	rm -f $$(<:.o=.linked1.o) $$(<:.o=.linked2.o) $$(<:.o=.linked3.o)

When PERMISSIVE is disabled, does the generated shell command chain hide
generation failures?

Because the $(if) block evaluates to nothing in strict mode, the generated
command ends with a semicolon:

> > $$(@:.skel.h=.subskel.h) ; \
> rm -f ...

If bpftool or diff fails earlier in the && chain, the shell will skip to the
semicolon and execute rm -f. Since rm -f always succeeds, it will return exit
code 0, which seems like it would cause make to silently ignore actual
compilation regressions in strict mode.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260415-selftests-bpf_misconfig-v5-0-03d0a52a898a@suse.com?part=3