[PATCH 0/3] selftests/nolibc: enable qemu-system tests with LLVM builds

[PATCH 0/3] selftests/nolibc: enable qemu-system tests with LLVM builds

[Thomas Weißschuh]

Currently the test setup does not support running nolibc-test built with
LLVM in qemu-system. Enable this.

FYI, sparc32 on LLVM seems to be broken at the moment. To me this looks
like a LLVM regression, emitting invalid object code.

Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
---
Thomas Weißschuh (3):
      selftests/nolibc: deduplicate invocations of toplevel Makefile
      selftests/nolibc: don't pass CC to toplevel Makefile
      selftests/nolibc: always compile the kernel with GCC

 tools/testing/selftests/nolibc/Makefile.nolibc | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
---
base-commit: b9e50363178a40c76bebaf2f00faa2b0b6baf8d1
change-id: 20250719-nolibc-llvm-system-311762b62829

Best regards,
-- 
Thomas Weißschuh <linux@weissschuh.net>

[PATCH 1/3] selftests/nolibc: deduplicate invocations of toplevel Makefile

[Thomas Weißschuh]

Various targets of the testsuite call back into the toplevel kernel
Makefile. These calls use various parameters and are quite long.

Introduce a common variable to make future changes smaller and the lines
shorter.

Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
---
 tools/testing/selftests/nolibc/Makefile.nolibc | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/tools/testing/selftests/nolibc/Makefile.nolibc b/tools/testing/selftests/nolibc/Makefile.nolibc
index 0fb759ba992ee6b1693b88f1b2e77463afa9f38b..8e7a832ba3f6da3ca0d669b941acd1cf79813a8d 100644
--- a/tools/testing/selftests/nolibc/Makefile.nolibc
+++ b/tools/testing/selftests/nolibc/Makefile.nolibc
@@ -262,6 +262,9 @@ REPORT  ?= awk '/\[OK\][\r]*$$/{p++} /\[FAIL\][\r]*$$/{if (!f) printf("\n"); f++
 		if (f || !p || !done) printf("failure\n"); else if (s) printf("warning\n"); else printf("success\n");; \
 		printf("\nSee all results in %s\n", ARGV[1]); }'
 
+# Execute the toplevel kernel Makefile
+KBUILD_MAKE = $(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE)
+
 help:
 	@echo "Supported targets under selftests/nolibc:"
 	@echo "  all               call the \"run\" target below"
@@ -340,17 +343,17 @@ initramfs: nolibc-test
 	$(Q)cp nolibc-test initramfs/init
 
 defconfig:
-	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) $(DEFCONFIG)
+	$(Q)$(KBUILD_MAKE) $(DEFCONFIG)
 	$(Q)if [ -n "$(EXTRACONFIG)" ]; then \
 		$(srctree)/scripts/config --file $(objtree)/.config $(EXTRACONFIG); \
-		$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) olddefconfig < /dev/null; \
+		$(KBUILD_MAKE) olddefconfig < /dev/null; \
 	fi
 
 kernel:
-	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) $(IMAGE_NAME) < /dev/null
+	$(Q)$(KBUILD_MAKE) $(IMAGE_NAME) < /dev/null
 
 kernel-standalone: initramfs
-	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) $(IMAGE_NAME) CONFIG_INITRAMFS_SOURCE=$(CURDIR)/initramfs < /dev/null
+	$(Q)$(KBUILD_MAKE) $(IMAGE_NAME) CONFIG_INITRAMFS_SOURCE=$(CURDIR)/initramfs < /dev/null
 
 # run the tests after building the kernel
 run: kernel initramfs.cpio

-- 
2.50.1

[PATCH 2/3] selftests/nolibc: don't pass CC to toplevel Makefile

[Thomas Weißschuh]

The toplevel Makefile is capable of calculating CC from CROSS_COMPILE
and/or ARCH.

Stop passing the unnecessary variable.

Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
---
 tools/testing/selftests/nolibc/Makefile.nolibc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/nolibc/Makefile.nolibc b/tools/testing/selftests/nolibc/Makefile.nolibc
index 8e7a832ba3f6da3ca0d669b941acd1cf79813a8d..6e6dddaa2e3ed56886cd76c423297648d13027c7 100644
--- a/tools/testing/selftests/nolibc/Makefile.nolibc
+++ b/tools/testing/selftests/nolibc/Makefile.nolibc
@@ -263,7 +263,7 @@ REPORT  ?= awk '/\[OK\][\r]*$$/{p++} /\[FAIL\][\r]*$$/{if (!f) printf("\n"); f++
 		printf("\nSee all results in %s\n", ARGV[1]); }'
 
 # Execute the toplevel kernel Makefile
-KBUILD_MAKE = $(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE)
+KBUILD_MAKE = $(MAKE) -C $(srctree) ARCH=$(ARCH) CROSS_COMPILE=$(CROSS_COMPILE)
 
 help:
 	@echo "Supported targets under selftests/nolibc:"

-- 
2.50.1

Re: [PATCH 2/3] selftests/nolibc: don't pass CC to toplevel Makefile

[Willy Tarreau]

Hi Thomas,

On Sat, Jul 19, 2025 at 05:38:28PM +0200, Thomas Weißschuh wrote:
> The toplevel Makefile is capable of calculating CC from CROSS_COMPILE
> and/or ARCH.
> 
> Stop passing the unnecessary variable.
(...) 
>  # Execute the toplevel kernel Makefile
> -KBUILD_MAKE = $(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE)
> +KBUILD_MAKE = $(MAKE) -C $(srctree) ARCH=$(ARCH) CROSS_COMPILE=$(CROSS_COMPILE)

Here the goal was not to help the toplevel Makefile figure CC, but rather
to permit the user to override it, and it's also listed in "make help",
and even used in cc-option.

I understnad that you're trying to avoid forcing CC to clang when
building, but in this case, what will CROSS_COMPILE contain ?  My
guess is that you intend to make CROSS_COMPILE point to the gcc-based
toolchain, and have CC point to clang for userland only. Is this the
case ? I think I'd be fine with this, but then we need to make it
explicit in the help message and fix the current one, possibly just
with this:

-	@echo "  nolibc-test       build the executable (uses \$$CC and \$$CROSS_COMPILE)"
+	@echo "  nolibc-test       build the executable (uses \$$CC)"
	@echo "  libc-test         build an executable using the compiler's default libc instead"
	@echo "  run-user          runs the executable under QEMU (uses \$$XARCH, \$$TEST)"
	@echo "  initramfs.cpio    prepare the initramfs archive with nolibc-test"
	@echo "  initramfs         prepare the initramfs tree with nolibc-test"
 	@echo "  defconfig         create a fresh new default config (uses \$$XARCH)"
-	@echo "  kernel            (re)build the kernel (uses \$$XARCH)"
+	@echo "  kernel            (re)build the kernel (uses \$$XARCH, \$$CROSS_COMPILE)"

Thanks,
Willy

Re: [PATCH 2/3] selftests/nolibc: don't pass CC to toplevel Makefile

[Thomas Weißschuh]

Hi Willy,

On 2025-07-21 04:56:27+0200, Willy Tarreau wrote:
> On Sat, Jul 19, 2025 at 05:38:28PM +0200, Thomas Weißschuh wrote:
> > The toplevel Makefile is capable of calculating CC from CROSS_COMPILE
> > and/or ARCH.
> > 
> > Stop passing the unnecessary variable.
> (...) 
> >  # Execute the toplevel kernel Makefile
> > -KBUILD_MAKE = $(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE)
> > +KBUILD_MAKE = $(MAKE) -C $(srctree) ARCH=$(ARCH) CROSS_COMPILE=$(CROSS_COMPILE)
> 
> Here the goal was not to help the toplevel Makefile figure CC, but rather
> to permit the user to override it, and it's also listed in "make help",
> and even used in cc-option.
> 
> I understnad that you're trying to avoid forcing CC to clang when
> building, but in this case, what will CROSS_COMPILE contain ?  My
> guess is that you intend to make CROSS_COMPILE point to the gcc-based
> toolchain, and have CC point to clang for userland only. Is this the
> case ?

Correct.

> I think I'd be fine with this, but then we need to make it
> explicit in the help message and fix the current one, possibly just
> with this:
> 
> -	@echo "  nolibc-test       build the executable (uses \$$CC and \$$CROSS_COMPILE)"
> +	@echo "  nolibc-test       build the executable (uses \$$CC)"

I don't think this is correct. $CC itself depends on $CROSS_COMPILE
through tools/scripts/Makefile.include.

> 	@echo "  libc-test         build an executable using the compiler's default libc instead"
> 	@echo "  run-user          runs the executable under QEMU (uses \$$XARCH, \$$TEST)"
> 	@echo "  initramfs.cpio    prepare the initramfs archive with nolibc-test"
> 	@echo "  initramfs         prepare the initramfs tree with nolibc-test"
>  	@echo "  defconfig         create a fresh new default config (uses \$$XARCH)"
> -	@echo "  kernel            (re)build the kernel (uses \$$XARCH)"
> +	@echo "  kernel            (re)build the kernel (uses \$$XARCH, \$$CROSS_COMPILE)"

I'll fold this into the last commit of the series.


Thomas

Re: [PATCH 2/3] selftests/nolibc: don't pass CC to toplevel Makefile

[Willy Tarreau]

Hi Thomas!

On Sun, Aug 17, 2025 at 10:30:52AM +0200, Thomas Weißschuh wrote:
> Hi Willy,
> 
> On 2025-07-21 04:56:27+0200, Willy Tarreau wrote:
> > On Sat, Jul 19, 2025 at 05:38:28PM +0200, Thomas Weißschuh wrote:
> > > The toplevel Makefile is capable of calculating CC from CROSS_COMPILE
> > > and/or ARCH.
> > > 
> > > Stop passing the unnecessary variable.
> > (...) 
> > >  # Execute the toplevel kernel Makefile
> > > -KBUILD_MAKE = $(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE)
> > > +KBUILD_MAKE = $(MAKE) -C $(srctree) ARCH=$(ARCH) CROSS_COMPILE=$(CROSS_COMPILE)
> > 
> > Here the goal was not to help the toplevel Makefile figure CC, but rather
> > to permit the user to override it, and it's also listed in "make help",
> > and even used in cc-option.
> > 
> > I understnad that you're trying to avoid forcing CC to clang when
> > building, but in this case, what will CROSS_COMPILE contain ?  My
> > guess is that you intend to make CROSS_COMPILE point to the gcc-based
> > toolchain, and have CC point to clang for userland only. Is this the
> > case ?
> 
> Correct.
> 
> > I think I'd be fine with this, but then we need to make it
> > explicit in the help message and fix the current one, possibly just
> > with this:
> > 
> > -	@echo "  nolibc-test       build the executable (uses \$$CC and \$$CROSS_COMPILE)"
> > +	@echo "  nolibc-test       build the executable (uses \$$CC)"
> 
> I don't think this is correct. $CC itself depends on $CROSS_COMPILE
> through tools/scripts/Makefile.include.

I don't understand what you mean by "depends on" here. CC defaults
to ${CROSS_COMPILE}gcc and may override it if set. So if one sets
CC, CROSS_COMPILE will not be used for it. Or maybe we could change
it to this to indicate a precedence if that's the idea you want to
convey ?

 -	@echo "  nolibc-test       build the executable (uses \$$CC and \$$CROSS_COMPILE)"
 +	@echo "  nolibc-test       build the executable (uses \$$CC or \$$CROSS_COMPILE)"

Willy

Re: [PATCH 2/3] selftests/nolibc: don't pass CC to toplevel Makefile

[Thomas Weißschuh]

On 2025-08-17 11:39:05+0200, Willy Tarreau wrote:
> On Sun, Aug 17, 2025 at 10:30:52AM +0200, Thomas Weißschuh wrote:
> > On 2025-07-21 04:56:27+0200, Willy Tarreau wrote:
> > > On Sat, Jul 19, 2025 at 05:38:28PM +0200, Thomas Weißschuh wrote:

(...)

> > > I think I'd be fine with this, but then we need to make it
> > > explicit in the help message and fix the current one, possibly just
> > > with this:
> > > 
> > > -	@echo "  nolibc-test       build the executable (uses \$$CC and \$$CROSS_COMPILE)"
> > > +	@echo "  nolibc-test       build the executable (uses \$$CC)"
> > 
> > I don't think this is correct. $CC itself depends on $CROSS_COMPILE
> > through tools/scripts/Makefile.include.
> 
> I don't understand what you mean by "depends on" here. CC defaults
> to ${CROSS_COMPILE}gcc and may override it if set. So if one sets
> CC, CROSS_COMPILE will not be used for it. Or maybe we could change
> it to this to indicate a precedence if that's the idea you want to
> convey ?
> 
>  -	@echo "  nolibc-test       build the executable (uses \$$CC and \$$CROSS_COMPILE)"
>  +	@echo "  nolibc-test       build the executable (uses \$$CC or \$$CROSS_COMPILE)"

Yes, that is the idea. 
I pushed a commit to this effect to nolibc/for-next.


Thomas

Re: [PATCH 2/3] selftests/nolibc: don't pass CC to toplevel Makefile

[Willy Tarreau]

On Mon, Aug 18, 2025 at 04:07:21PM +0200, Thomas Weißschuh wrote:
> On 2025-08-17 11:39:05+0200, Willy Tarreau wrote:
> > On Sun, Aug 17, 2025 at 10:30:52AM +0200, Thomas Weißschuh wrote:
> > > On 2025-07-21 04:56:27+0200, Willy Tarreau wrote:
> > > > On Sat, Jul 19, 2025 at 05:38:28PM +0200, Thomas Weißschuh wrote:
> 
> (...)
> 
> > > > I think I'd be fine with this, but then we need to make it
> > > > explicit in the help message and fix the current one, possibly just
> > > > with this:
> > > > 
> > > > -	@echo "  nolibc-test       build the executable (uses \$$CC and \$$CROSS_COMPILE)"
> > > > +	@echo "  nolibc-test       build the executable (uses \$$CC)"
> > > 
> > > I don't think this is correct. $CC itself depends on $CROSS_COMPILE
> > > through tools/scripts/Makefile.include.
> > 
> > I don't understand what you mean by "depends on" here. CC defaults
> > to ${CROSS_COMPILE}gcc and may override it if set. So if one sets
> > CC, CROSS_COMPILE will not be used for it. Or maybe we could change
> > it to this to indicate a precedence if that's the idea you want to
> > convey ?
> > 
> >  -	@echo "  nolibc-test       build the executable (uses \$$CC and \$$CROSS_COMPILE)"
> >  +	@echo "  nolibc-test       build the executable (uses \$$CC or \$$CROSS_COMPILE)"
> 
> Yes, that is the idea. 
> I pushed a commit to this effect to nolibc/for-next.

OK perfect, thank you!
Willy

[PATCH 3/3] selftests/nolibc: always compile the kernel with GCC

[Thomas Weißschuh]

LLVM/clang can not build the kernel for all architectures supported by
nolibc. The current setup uses the same compiler to build the kernel as
is used for nolibc-test. This prevents using the full qemu-system tests
for LLVM builds.

Instead always build the kernel with GCC. For the nolibc testsuite the
kernel does not need to be built with LLVM.

Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
---
 tools/testing/selftests/nolibc/Makefile.nolibc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/nolibc/Makefile.nolibc b/tools/testing/selftests/nolibc/Makefile.nolibc
index 6e6dddaa2e3ed56886cd76c423297648d13027c7..3d8e314c2433a2d032cf4f58ea1d6a2f430faaae 100644
--- a/tools/testing/selftests/nolibc/Makefile.nolibc
+++ b/tools/testing/selftests/nolibc/Makefile.nolibc
@@ -263,7 +263,7 @@ REPORT  ?= awk '/\[OK\][\r]*$$/{p++} /\[FAIL\][\r]*$$/{if (!f) printf("\n"); f++
 		printf("\nSee all results in %s\n", ARGV[1]); }'
 
 # Execute the toplevel kernel Makefile
-KBUILD_MAKE = $(MAKE) -C $(srctree) ARCH=$(ARCH) CROSS_COMPILE=$(CROSS_COMPILE)
+KBUILD_MAKE = $(MAKE) -C $(srctree) ARCH=$(ARCH) CROSS_COMPILE=$(CROSS_COMPILE) LLVM=
 
 help:
 	@echo "Supported targets under selftests/nolibc:"

-- 
2.50.1

Re: [PATCH 0/3] selftests/nolibc: enable qemu-system tests with LLVM builds

[Willy Tarreau]

On Sat, Jul 19, 2025 at 05:38:26PM +0200, Thomas Weißschuh wrote:
> Currently the test setup does not support running nolibc-test built with
> LLVM in qemu-system. Enable this.
> 
> FYI, sparc32 on LLVM seems to be broken at the moment. To me this looks
> like a LLVM regression, emitting invalid object code.
> 
> Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>

For the series, once the help message is adjusted:

  Acked-by: Willy Tarreau <w@1wt.eu>

Thanks!
Willy