Re: [PATCH 5/6] selftests: Preserve subtarget failures in all/install

[Mark Brown <broonie@kernel.org>]

[-- Attachment #1: Type: text/plain, Size: 2321 bytes --]

On Fri, Mar 20, 2026 at 03:29:20PM -0300, Ricardo B. Marlière wrote:
> Track failures explicitly in the top-level selftests all/install loops.
> 
> The current code multiplies `ret` by each sub-make exit status. For
> example, with `TARGETS=net`, the implicit `net/lib` dependency runs after
> `net`, so a failed `net` build can be followed by a successful `net/lib`
> build and reset the final result to success.
> 
> Set `ret` to 1 on any non-zero sub-make exit code and keep it sticky, so
> the top-level make returns failure when any selected selftest target
> fails.

This patch, which is now in mainline as 7e47389142b8, is breaking a
bunch of CI systems - at least KernelCI, our Arm internal CI and my
personal stuff.  It causes the equivalent of FORCE_TARGETS behaviour in
the top level Makefile, the prior behaviour where the exit status of the
top level Makefile ignores failures from individual directories is
desirable since by default we try to build almost all the selftests but
between quality issues and build time dependencies it's very common for
at least one of them to fail.  With this commit unless the user has
configured a more restricted set of selftests it would be surprising if
we manage to get a successful build and install.

As well as being a poor default due to the very high likelyhood of build
failures this also has the undesirable effect of causing a build failure
in one selftest to cause the whole install target to fail, meaning that
the build failure is escallated to a complete lost of coverge for all
selftests in common CI usage.

This wasn't showing up in my -next build tests since I set FORCE_TARGETS
and explicitly choose a restricted set of kselftests which actually
build with my system and configuration.  It was less obvious than it
should have been with the other systems since they did not expect there
to be a complete failure to generate a kselftest tarball and variously
masked the error or reported it in a manner that looked like an
infrastructure issue.

It would be really nice to get to the point where we can reasonably do
this but we're simply not there at the current time.  At the moment if
people want to see build failures reported at the top level that really
needs to be opt in, we have FORCE_TARGETS for that.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]