Re: [PATCH] selftests/nolibc: add 7 tests for memcmp()

[Willy Tarreau <w@1wt.eu>]

Hi Paul,

On Mon, Oct 24, 2022 at 08:53:57AM -0700, Paul E. McKenney wrote:
> > Will keep thinking about it and hopefully propose a patch to make the
> > tests easier to use before we're too far in the 6.1 release.
> 
> Another possibility is to have a separate developers' and maintainers'
> option.  Linus and I do "make whatever" for some value of "whatever"
> that builds from scratch, doing whatever cleaning that might be required.
> Developers use targets that are faster but have the possibility of false
> positives and false negatives.
> 
> But maybe you have something better in mind.
> 
> > Thanks for keeping the conversation flowing, that helps me!
> 
> Looking forward to seeing what you come up with!

I could finally figure what was taking time in the installation process.
Interestingly, it's "make headers", which is not redone without a "make
clean" at the kernel level. The "make headers_install" only takes a few
hundred milliseconds, so issuing a systematic "make clean" in the nolibc
test dir only takes ~800ms here to perform a full rebuild, which is totally
acceptable to me.

Thus what I've done is to mark the sysroot target as .phony and start it
with a removal of the current include dir so that we systematically rebuild
it. Now there's no such risk of running a test against an earlier version
anymore, and there are no "make clean" to worry about anymore either.
That looks much better to me!

And I could confirm that just issuing:

  $ time make -j8 -C tools/testing/selftests/nolibc run

after reverting Rasmus' fix led me to this pretty quickly:

  ...
  Kernel: arch/x86/boot/bzImage is ready  (#3)
  make[1]: Leaving directory '/k'
  15 memcmp_20_e0 = 64                    [FAIL]
  16 memcmp_e0_20 = -64                   [FAIL]
  See all results in /k/tools/testing/selftests/nolibc/run.out
  make: Leaving directory '/k/tools/testing/selftests/nolibc'

  real    0m14.538s
  user    0m27.828s
  sys     0m4.576s

No more false positives nor false negatives anymore. I'm sending you
the patch separately.

Thanks for the discussion, the solution is way better now!
Willy