linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [RFC PATCH v2 0/2] tools/nolibc: install unified multi-arch headers
@ 2025-06-29 17:07 Willy Tarreau
  2025-06-29 17:07 ` [RFC PATCH v2 1/2] tools/nolibc: make the "headers" target install all supported archs Willy Tarreau
  2025-06-29 17:07 ` [RFC PATCH v2 2/2] tools/nolibc: add a new "install_all_archs" target Willy Tarreau
  0 siblings, 2 replies; 7+ messages in thread
From: Willy Tarreau @ 2025-06-29 17:07 UTC (permalink / raw)
  To: Thomas Weißschuh; +Cc: linux-kernel, Arnd Bergmann, Willy Tarreau

While the in-tree nolibc directory supports arch auto-detection,
installing it is only performed for a single architecture at once,
requiring as many sysroots as desired architectures. It is not
convenient because external tools rarely expect to change their
include path based on the target.

Furthermore, when installing it along with the UAPI headers for
use with the nolibc toolchains, the issue is reinforced as UAPI
may only be installed for a single architecture, and in this case
we cannot even count on the headers that would have been packaged
along with the compiler and its accompanying libc.

This patchset proposes a convenient approach to satisfy all this.

First, it updates nolibc's "headers" target so that it will install
the headers for all supported archs, so that they appear just like
in the original source tree, meaning that when uapi headers are
available (e.g. provided with the toolchain), the build will succeed
for any supported archs from a single installation directory.

Second, a new "install_all_archs" target iterates over all supported
archs to install the corresponding UAPI headers, and moves each
arch-specific "asm/" subdir to "asm-arch-$arch", then automatically
recreates the files under asm/ to include the corresponding files based
on the detected architecture. This results in a unified sysroot that
can be used by any of the supported architectures without changing the
include path.

I finally didn't use Arnd's proposal to try to build up the include
file name based on the architecture because I realized that it wouldn't
work for programs that include asm/foo first (or we'd have to go back
through a #include nolibc.h which is outside the scope of this change
that tries to remain libc-agnostic). Instead a template file containing
the #ifdefs for all archs is used for each file. It results in a slightly
larger set of asm/ files than the first version which only contained the
ifdefs for supported archs for each file, but compared to the unified
UAPI size the difference remains pretty small.

I'm marking this as RFC because the operations are entirely performed
inside the nolibc Makefile, and I'm wondering whether there could be
any interest in generalizing the principle and moving it to the
generic uapi installation itself. However I don't see how this could
be easily done in this case, because here we have no other option but
iterate over all supported architectures, and iterating over multiple
archs is not something standard in the kbuild system. But we could also
imagine having a script under scripts/ to install UAPI headers for all
archs at once for example, so ideas are welcome. Of course if there's
no perceived interest in generalizing this to uapi then it can stay
in nolibc where the maintenance cost should remain quite low anyway.

Changes in v2:
- merge of x86_64 and i386 extracted from the series
- dropped the aarch64->arm64 mapping since already handled separately
  by Thomas
- made the "headers" target install headers for all archs as suggested
  by Thomas.
- dropped the #ifdef generation from the makefile and run sed over
  a template instead.

Thanks for any comments!

---
Willy Tarreau (2):
  tools/nolibc: make the "headers" target install all supported archs
  tools/nolibc: add a new "install_all_archs" target

 tools/include/nolibc/Makefile       | 50 ++++++++++++++++++++++-------
 tools/include/nolibc/asm-template.h | 25 +++++++++++++++
 2 files changed, 63 insertions(+), 12 deletions(-)
 create mode 100644 tools/include/nolibc/asm-template.h

-- 
2.17.5

^ permalink raw reply	[flat|nested] 7+ messages in thread

* [RFC PATCH v2 1/2] tools/nolibc: make the "headers" target install all supported archs
  2025-06-29 17:07 [RFC PATCH v2 0/2] tools/nolibc: install unified multi-arch headers Willy Tarreau
@ 2025-06-29 17:07 ` Willy Tarreau
  2025-06-29 20:17   ` Thomas Weißschuh
  2025-06-29 17:07 ` [RFC PATCH v2 2/2] tools/nolibc: add a new "install_all_archs" target Willy Tarreau
  1 sibling, 1 reply; 7+ messages in thread
From: Willy Tarreau @ 2025-06-29 17:07 UTC (permalink / raw)
  To: Thomas Weißschuh; +Cc: linux-kernel, Arnd Bergmann, Willy Tarreau

The efforts we go through by installing a single arch are counter
productive when the base directory already supports them all, and
the arch-specific files are really small. Let's make the "headers"
target simply install headers for all supported archs and stop
trying to build a hybrid "arch.h" file on the fly, to instead keep
the generic one.

Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/include/nolibc/Makefile | 20 ++++++++------------
 1 file changed, 8 insertions(+), 12 deletions(-)

diff --git a/tools/include/nolibc/Makefile b/tools/include/nolibc/Makefile
index c69d7bf60c71e..d5be3d213c885 100644
--- a/tools/include/nolibc/Makefile
+++ b/tools/include/nolibc/Makefile
@@ -24,6 +24,8 @@ Q=@
 endif
 
 arch_file := arch-$(ARCH).h
+nolibc_supported_archs := arm64 arm loongarch m68k mips powerpc riscv s390 sparc x86
+
 all_files := \
 		compiler.h \
 		crt.h \
@@ -81,7 +83,7 @@ help:
 	@echo "Supported targets under nolibc:"
 	@echo "  all                 call \"headers\""
 	@echo "  clean               clean the sysroot"
-	@echo "  headers             prepare a sysroot in \$${OUTPUT}sysroot"
+	@echo "  headers             prepare a multi-arch sysroot in \$${OUTPUT}sysroot"
 	@echo "  headers_standalone  like \"headers\", and also install kernel headers"
 	@echo "  help                this help"
 	@echo ""
@@ -92,18 +94,12 @@ help:
 	@echo "  OUTPUT  = $(OUTPUT)"
 	@echo ""
 
+# installs headers for all archs at once.
 headers:
-	$(Q)mkdir -p $(OUTPUT)sysroot
-	$(Q)mkdir -p $(OUTPUT)sysroot/include
-	$(Q)cp --parents $(all_files) $(OUTPUT)sysroot/include/
-	$(Q)if [ "$(ARCH)" = "i386" -o "$(ARCH)" = "x86_64" ]; then \
-		cat arch-x86.h;                 \
-	elif [ -e "$(arch_file)" ]; then        \
-		cat $(arch_file);               \
-	else                                    \
-		echo "Fatal: architecture $(ARCH) not yet supported by nolibc." >&2; \
-		exit 1;                         \
-	fi > $(OUTPUT)sysroot/include/arch.h
+	$(Q)mkdir -p "$(OUTPUT)sysroot"
+	$(Q)mkdir -p "$(OUTPUT)sysroot/include"
+	$(Q)cp --parents $(all_files) arch.h "$(OUTPUT)sysroot/include/"
+	$(Q)cp $(addsuffix .h,$(addprefix arch-,$(nolibc_supported_archs))) "$(OUTPUT)sysroot/include/"
 
 headers_standalone: headers
 	$(Q)$(MAKE) -C $(srctree) headers
-- 
2.17.5


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [RFC PATCH v2 2/2] tools/nolibc: add a new "install_all_archs" target
  2025-06-29 17:07 [RFC PATCH v2 0/2] tools/nolibc: install unified multi-arch headers Willy Tarreau
  2025-06-29 17:07 ` [RFC PATCH v2 1/2] tools/nolibc: make the "headers" target install all supported archs Willy Tarreau
@ 2025-06-29 17:07 ` Willy Tarreau
  2025-06-29 20:44   ` Thomas Weißschuh
  1 sibling, 1 reply; 7+ messages in thread
From: Willy Tarreau @ 2025-06-29 17:07 UTC (permalink / raw)
  To: Thomas Weißschuh; +Cc: linux-kernel, Arnd Bergmann, Willy Tarreau

This installs all supported archs together, both from nolibc and kernel
headers. The arch-specific asm/ subdirs are renamed to asm-arch-$arch,
and asm/ is rebuilt from all these files in order to include the right
one depending on the build architecture. This is done by reusing a
template file (asm-template.h) for each file found under asm-arch-*,
and including the right sub-dir depending on the current arch.

This allows to use a single unified sysroot for all archs, and to only
change the compiler or the target architecture. This way, a complete
sysroot is much easier to use (a single directory is needed) and much
smaller.

Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/include/nolibc/Makefile       | 24 ++++++++++++++++++++++++
 tools/include/nolibc/asm-template.h | 25 +++++++++++++++++++++++++
 2 files changed, 49 insertions(+)
 create mode 100644 tools/include/nolibc/asm-template.h

diff --git a/tools/include/nolibc/Makefile b/tools/include/nolibc/Makefile
index d5be3d213c885..c47559a066f35 100644
--- a/tools/include/nolibc/Makefile
+++ b/tools/include/nolibc/Makefile
@@ -86,6 +86,7 @@ help:
 	@echo "  headers             prepare a multi-arch sysroot in \$${OUTPUT}sysroot"
 	@echo "  headers_standalone  like \"headers\", and also install kernel headers"
 	@echo "  help                this help"
+	@echo "  install_all_archs   install a multi-arch sysroot + kernel headers in \$${OUTPUT}sysroot"
 	@echo ""
 	@echo "These targets may also be called from tools as \"make nolibc_<target>\"."
 	@echo ""
@@ -105,6 +106,29 @@ headers_standalone: headers
 	$(Q)$(MAKE) -C $(srctree) headers
 	$(Q)$(MAKE) -C $(srctree) headers_install INSTALL_HDR_PATH=$(OUTPUT)sysroot
 
+install_all_archs: headers
+	@# install common headers for any arch, take them all. This will clear everything.
+	$(Q)$(MAKE) -C $(srctree) ARCH=x86 mrproper
+	$(Q)$(MAKE) -C $(srctree) ARCH=x86 headers_install no-export-headers= INSTALL_HDR_PATH="$(OUTPUT)sysroot"
+	@# remove the contents of the unused asm dir which we will rebuild from the arch ones
+	$(Q)rm -rf "$(OUTPUT)sysroot/include/asm"
+	$(Q)mkdir -p "$(OUTPUT)sysroot/include/asm"
+	@# Now install headers for all archs
+	$(Q)for arch in $(nolibc_supported_archs); do \
+		echo "# installing $$arch"; \
+		if ! [ -d $(OUTPUT)sysroot/include/asm-arch-$$arch ]; then \
+			$(MAKE) -C $(srctree) ARCH=$$arch mrproper; \
+			$(MAKE) -C $(srctree) ARCH=$$arch headers_install no-export-headers= \
+				INSTALL_HDR_PATH="$(OUTPUT)sysroot/include/$$arch" >/dev/null; \
+			mv "$(OUTPUT)sysroot/include/$$arch/include/asm" "$(OUTPUT)sysroot/include/asm-arch-$$arch"; \
+			rm -rf "$(OUTPUT)sysroot/include/$$arch"; \
+		fi;\
+	done; \
+	mkdir -p "$(OUTPUT)sysroot/include/asm"; \
+	for file in $$(find "$(OUTPUT)sysroot/include/"asm-arch-* -maxdepth 1 -name '*.h' -printf '%P\n'); do \
+		sed -e "s!_ASMFILE_!$$file!" asm-template.h > "$(OUTPUT)sysroot/include/asm/$$file"; \
+	done
+
 # GCC uses "s390", clang "systemz"
 CLANG_CROSS_FLAGS := $(subst --target=s390-linux,--target=systemz-linux,$(CLANG_CROSS_FLAGS))
 
diff --git a/tools/include/nolibc/asm-template.h b/tools/include/nolibc/asm-template.h
new file mode 100644
index 0000000000000..84930c4761d16
--- /dev/null
+++ b/tools/include/nolibc/asm-template.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: LGPL-2.1 OR MIT */
+
+#if defined(__x86_64__) || defined(__i386__) || defined(__i486__) || defined(__i586__) || defined(__i686__)
+#include "../asm-arch-x86/_ASMFILE_"
+#elif defined(__ARM_EABI__)
+#include "../asm-arch-arm/_ASMFILE_"
+#elif defined(__aarch64__)
+#include "../asm-arch-arm64/_ASMFILE_"
+#elif defined(__mips__)
+#include "../asm-arch-mips/_ASMFILE_"
+#elif defined(__powerpc__)
+#include "../asm-arch-powerpc/_ASMFILE_"
+#elif defined(__riscv)
+#include "../asm-arch-riscv/_ASMFILE_"
+#elif defined(__s390x__) || defined(__s390__)
+#include "../asm-arch-s390/_ASMFILE_"
+#elif defined(__loongarch__)
+#include "../asm-arch-loongarch/_ASMFILE_"
+#elif defined(__sparc__)
+#include "../asm-arch-sparc/_ASMFILE_"
+#elif defined(__m68k__)
+#include "../asm-arch-m68k/_ASMFILE_"
+#else
+#error Unsupported Architecture
+#endif
-- 
2.17.5


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* Re: [RFC PATCH v2 1/2] tools/nolibc: make the "headers" target install all supported archs
  2025-06-29 17:07 ` [RFC PATCH v2 1/2] tools/nolibc: make the "headers" target install all supported archs Willy Tarreau
@ 2025-06-29 20:17   ` Thomas Weißschuh
  0 siblings, 0 replies; 7+ messages in thread
From: Thomas Weißschuh @ 2025-06-29 20:17 UTC (permalink / raw)
  To: Willy Tarreau; +Cc: Thomas Weißschuh, linux-kernel, Arnd Bergmann

On 2025-06-29 19:07:31+0200, Willy Tarreau wrote:
> The efforts we go through by installing a single arch are counter
> productive when the base directory already supports them all, and
> the arch-specific files are really small. Let's make the "headers"
> target simply install headers for all supported archs and stop
> trying to build a hybrid "arch.h" file on the fly, to instead keep
> the generic one.
> 
> Signed-off-by: Willy Tarreau <w@1wt.eu>
> ---
>  tools/include/nolibc/Makefile | 20 ++++++++------------
>  1 file changed, 8 insertions(+), 12 deletions(-)
> 
> diff --git a/tools/include/nolibc/Makefile b/tools/include/nolibc/Makefile
> index c69d7bf60c71e..d5be3d213c885 100644
> --- a/tools/include/nolibc/Makefile
> +++ b/tools/include/nolibc/Makefile
> @@ -24,6 +24,8 @@ Q=@
>  endif
>  
>  arch_file := arch-$(ARCH).h

arch_file is unused now.

> +nolibc_supported_archs := arm64 arm loongarch m68k mips powerpc riscv s390 sparc x86

Nitpick:
Instead of having another variable, can we add the arch-foo.h files to
all_files instead? In patch 2 they can be extracted from there with $(filter)?

> +
>  all_files := \
>  		compiler.h \
>  		crt.h \
> @@ -81,7 +83,7 @@ help:
>  	@echo "Supported targets under nolibc:"
>  	@echo "  all                 call \"headers\""
>  	@echo "  clean               clean the sysroot"
> -	@echo "  headers             prepare a sysroot in \$${OUTPUT}sysroot"
> +	@echo "  headers             prepare a multi-arch sysroot in \$${OUTPUT}sysroot"

If multi-arch is the only remaining mode, do we need to mention it?

>  	@echo "  headers_standalone  like \"headers\", and also install kernel headers"
>  	@echo "  help                this help"
>  	@echo ""
> @@ -92,18 +94,12 @@ help:
>  	@echo "  OUTPUT  = $(OUTPUT)"
>  	@echo ""
>  
> +# installs headers for all archs at once.
>  headers:
> -	$(Q)mkdir -p $(OUTPUT)sysroot
> -	$(Q)mkdir -p $(OUTPUT)sysroot/include
> -	$(Q)cp --parents $(all_files) $(OUTPUT)sysroot/include/
> -	$(Q)if [ "$(ARCH)" = "i386" -o "$(ARCH)" = "x86_64" ]; then \
> -		cat arch-x86.h;                 \
> -	elif [ -e "$(arch_file)" ]; then        \
> -		cat $(arch_file);               \
> -	else                                    \
> -		echo "Fatal: architecture $(ARCH) not yet supported by nolibc." >&2; \
> -		exit 1;                         \
> -	fi > $(OUTPUT)sysroot/include/arch.h
> +	$(Q)mkdir -p "$(OUTPUT)sysroot"
> +	$(Q)mkdir -p "$(OUTPUT)sysroot/include"
> +	$(Q)cp --parents $(all_files) arch.h "$(OUTPUT)sysroot/include/"

Can we add arch.h to all_files, to avoid the special case here?

> +	$(Q)cp $(addsuffix .h,$(addprefix arch-,$(nolibc_supported_archs))) "$(OUTPUT)sysroot/include/"
>  
>  headers_standalone: headers
>  	$(Q)$(MAKE) -C $(srctree) headers
> -- 
> 2.17.5
> 

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [RFC PATCH v2 2/2] tools/nolibc: add a new "install_all_archs" target
  2025-06-29 17:07 ` [RFC PATCH v2 2/2] tools/nolibc: add a new "install_all_archs" target Willy Tarreau
@ 2025-06-29 20:44   ` Thomas Weißschuh
  2025-07-06  7:01     ` Willy Tarreau
  0 siblings, 1 reply; 7+ messages in thread
From: Thomas Weißschuh @ 2025-06-29 20:44 UTC (permalink / raw)
  To: Willy Tarreau; +Cc: Thomas Weißschuh, linux-kernel, Arnd Bergmann

On 2025-06-29 19:07:32+0200, Willy Tarreau wrote:
> This installs all supported archs together, both from nolibc and kernel
> headers. The arch-specific asm/ subdirs are renamed to asm-arch-$arch,
> and asm/ is rebuilt from all these files in order to include the right
> one depending on the build architecture. This is done by reusing a
> template file (asm-template.h) for each file found under asm-arch-*,
> and including the right sub-dir depending on the current arch.

Personally I would slightly prefer the shorter asm-$arch/ over
asm-arch-$arch/. Similar how the arch-*.h files are named.

> This allows to use a single unified sysroot for all archs, and to only
> change the compiler or the target architecture. This way, a complete
> sysroot is much easier to use (a single directory is needed) and much
> smaller.

What is the base commit for this series?

> 
> Signed-off-by: Willy Tarreau <w@1wt.eu>
> ---
>  tools/include/nolibc/Makefile       | 24 ++++++++++++++++++++++++
>  tools/include/nolibc/asm-template.h | 25 +++++++++++++++++++++++++
>  2 files changed, 49 insertions(+)
>  create mode 100644 tools/include/nolibc/asm-template.h
> 
> diff --git a/tools/include/nolibc/Makefile b/tools/include/nolibc/Makefile
> index d5be3d213c885..c47559a066f35 100644
> --- a/tools/include/nolibc/Makefile
> +++ b/tools/include/nolibc/Makefile
> @@ -86,6 +86,7 @@ help:
>  	@echo "  headers             prepare a multi-arch sysroot in \$${OUTPUT}sysroot"
>  	@echo "  headers_standalone  like \"headers\", and also install kernel headers"
>  	@echo "  help                this help"
> +	@echo "  install_all_archs   install a multi-arch sysroot + kernel headers in \$${OUTPUT}sysroot"

"headers_multiarch" Would be more consistent with the other targets.
Also should be above "help".

>  	@echo ""
>  	@echo "These targets may also be called from tools as \"make nolibc_<target>\"."
>  	@echo ""
> @@ -105,6 +106,29 @@ headers_standalone: headers
>  	$(Q)$(MAKE) -C $(srctree) headers
>  	$(Q)$(MAKE) -C $(srctree) headers_install INSTALL_HDR_PATH=$(OUTPUT)sysroot
>  
> +install_all_archs: headers
> +	@# install common headers for any arch, take them all. This will clear everything.
> +	$(Q)$(MAKE) -C $(srctree) ARCH=x86 mrproper
> +	$(Q)$(MAKE) -C $(srctree) ARCH=x86 headers_install no-export-headers= INSTALL_HDR_PATH="$(OUTPUT)sysroot"
> +	@# remove the contents of the unused asm dir which we will rebuild from the arch ones
> +	$(Q)rm -rf "$(OUTPUT)sysroot/include/asm"
> +	$(Q)mkdir -p "$(OUTPUT)sysroot/include/asm"
> +	@# Now install headers for all archs
> +	$(Q)for arch in $(nolibc_supported_archs); do \
> +		echo "# installing $$arch"; \
> +		if ! [ -d $(OUTPUT)sysroot/include/asm-arch-$$arch ]; then \

I don't understand this check. If used with an existing sysroot, the
files won't be updated.

> +			$(MAKE) -C $(srctree) ARCH=$$arch mrproper; \

For the defconfig target we tried to avoid implicitly deleting files.
Maybe letting make erroring out would be less surprising.
A clean source tree is necessary for the testsuite anyways.

> +			$(MAKE) -C $(srctree) ARCH=$$arch headers_install no-export-headers= \
> +				INSTALL_HDR_PATH="$(OUTPUT)sysroot/include/$$arch" >/dev/null; \
> +			mv "$(OUTPUT)sysroot/include/$$arch/include/asm" "$(OUTPUT)sysroot/include/asm-arch-$$arch"; \
> +			rm -rf "$(OUTPUT)sysroot/include/$$arch"; \
> +		fi;\
> +	done; \
> +	mkdir -p "$(OUTPUT)sysroot/include/asm"; \
> +	for file in $$(find "$(OUTPUT)sysroot/include/"asm-arch-* -maxdepth 1 -name '*.h' -printf '%P\n'); do \
> +		sed -e "s!_ASMFILE_!$$file!" asm-template.h > "$(OUTPUT)sysroot/include/asm/$$file"; \
> +	done

I think it should be possible to use the existing arch.h as template.
That would avoid some duplication and busywork.

> +
>  # GCC uses "s390", clang "systemz"
>  CLANG_CROSS_FLAGS := $(subst --target=s390-linux,--target=systemz-linux,$(CLANG_CROSS_FLAGS))
>  
> diff --git a/tools/include/nolibc/asm-template.h b/tools/include/nolibc/asm-template.h
> new file mode 100644
> index 0000000000000..84930c4761d16
> --- /dev/null
> +++ b/tools/include/nolibc/asm-template.h
> @@ -0,0 +1,25 @@
> +/* SPDX-License-Identifier: LGPL-2.1 OR MIT */
> +
> +#if defined(__x86_64__) || defined(__i386__) || defined(__i486__) || defined(__i586__) || defined(__i686__)
> +#include "../asm-arch-x86/_ASMFILE_"
> +#elif defined(__ARM_EABI__)
> +#include "../asm-arch-arm/_ASMFILE_"
> +#elif defined(__aarch64__)
> +#include "../asm-arch-arm64/_ASMFILE_"
> +#elif defined(__mips__)
> +#include "../asm-arch-mips/_ASMFILE_"
> +#elif defined(__powerpc__)
> +#include "../asm-arch-powerpc/_ASMFILE_"
> +#elif defined(__riscv)
> +#include "../asm-arch-riscv/_ASMFILE_"
> +#elif defined(__s390x__) || defined(__s390__)
> +#include "../asm-arch-s390/_ASMFILE_"
> +#elif defined(__loongarch__)
> +#include "../asm-arch-loongarch/_ASMFILE_"
> +#elif defined(__sparc__)
> +#include "../asm-arch-sparc/_ASMFILE_"
> +#elif defined(__m68k__)
> +#include "../asm-arch-m68k/_ASMFILE_"
> +#else
> +#error Unsupported Architecture
> +#endif
> -- 
> 2.17.5
> 

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [RFC PATCH v2 2/2] tools/nolibc: add a new "install_all_archs" target
  2025-06-29 20:44   ` Thomas Weißschuh
@ 2025-07-06  7:01     ` Willy Tarreau
  2025-07-07 15:41       ` Thomas Weißschuh
  0 siblings, 1 reply; 7+ messages in thread
From: Willy Tarreau @ 2025-07-06  7:01 UTC (permalink / raw)
  To: Thomas Weißschuh; +Cc: Thomas Weißschuh, linux-kernel, Arnd Bergmann

On Sun, Jun 29, 2025 at 10:44:31PM +0200, Thomas Weißschuh wrote:
> On 2025-06-29 19:07:32+0200, Willy Tarreau wrote:
> > This installs all supported archs together, both from nolibc and kernel
> > headers. The arch-specific asm/ subdirs are renamed to asm-arch-$arch,
> > and asm/ is rebuilt from all these files in order to include the right
> > one depending on the build architecture. This is done by reusing a
> > template file (asm-template.h) for each file found under asm-arch-*,
> > and including the right sub-dir depending on the current arch.
> 
> Personally I would slightly prefer the shorter asm-$arch/ over
> asm-arch-$arch/. Similar how the arch-*.h files are named.

That was my first intent but I wanted to avoid special casing asm-generic
if the list of archs to be processed came from asm-*, as my initial
approach was to have a method that was totally independent of nolibc. I'm
of course fine with that approach as well, just not sold on any of them
at the moment.

> > This allows to use a single unified sysroot for all archs, and to only
> > change the compiler or the target architecture. This way, a complete
> > sysroot is much easier to use (a single directory is needed) and much
> > smaller.
> 
> What is the base commit for this series?

Just forgot, will check, though at the moment it's not important since
the purpose is to discuss about the installation method.

> > --- a/tools/include/nolibc/Makefile
> > +++ b/tools/include/nolibc/Makefile
> > @@ -86,6 +86,7 @@ help:
> >  	@echo "  headers             prepare a multi-arch sysroot in \$${OUTPUT}sysroot"
> >  	@echo "  headers_standalone  like \"headers\", and also install kernel headers"
> >  	@echo "  help                this help"
> > +	@echo "  install_all_archs   install a multi-arch sysroot + kernel headers in \$${OUTPUT}sysroot"
> 
> "headers_multiarch" Would be more consistent with the other targets.

I'm still not fond of any of the names in the list, they're all confusing
because the same terminology is used to install uapi headers and nolibc
headers. Ideally I'd like to have this:
  - a method to install only uapi headers (similar to make headers_install
    but for all archs supported by the kernel).
  - a method to install nolibc headers only (now it's for all archs so
    that's probably covered).
  - a method to install uapi+nolibc only for one arch (currently
    make headers_standalone)
  - a method to install uapi+nolibc for all archs supported by nolibc

If it is made clear that there is no interest in packaging uapi headers
for archs not supported by nolibc, then this means that nolibc can be
the place where the archs to be packaged is decided, and the two can
be more tightly coupled, but as of now, this lack of interest is not
yet obvious to me.

In addition, I think it can remain desirable to be able to produce headers
packages for a given kernel version, and independent nolibc headers, so
that it beacomes easy to simply assemble the two depending on what's
desired. And as a convenience, having a single target that does everything
is obviously nice.

> Also should be above "help".

If renamed, yes, but I was respecting alphabetical ordering ;-)

> >  	@echo ""
> >  	@echo "These targets may also be called from tools as \"make nolibc_<target>\"."
> >  	@echo ""
> > @@ -105,6 +106,29 @@ headers_standalone: headers
> >  	$(Q)$(MAKE) -C $(srctree) headers
> >  	$(Q)$(MAKE) -C $(srctree) headers_install INSTALL_HDR_PATH=$(OUTPUT)sysroot
> >  
> > +install_all_archs: headers
> > +	@# install common headers for any arch, take them all. This will clear everything.
> > +	$(Q)$(MAKE) -C $(srctree) ARCH=x86 mrproper
> > +	$(Q)$(MAKE) -C $(srctree) ARCH=x86 headers_install no-export-headers= INSTALL_HDR_PATH="$(OUTPUT)sysroot"
> > +	@# remove the contents of the unused asm dir which we will rebuild from the arch ones
> > +	$(Q)rm -rf "$(OUTPUT)sysroot/include/asm"
> > +	$(Q)mkdir -p "$(OUTPUT)sysroot/include/asm"
> > +	@# Now install headers for all archs
> > +	$(Q)for arch in $(nolibc_supported_archs); do \
> > +		echo "# installing $$arch"; \
> > +		if ! [ -d $(OUTPUT)sysroot/include/asm-arch-$$arch ]; then \
> 
> I don't understand this check. If used with an existing sysroot, the
> files won't be updated.

It comes from my initial intent not to depend on onlibc and instead seek
a method to incrementally install archs one at a time in the target
directory.

> > +			$(MAKE) -C $(srctree) ARCH=$$arch mrproper; \
> 
> For the defconfig target we tried to avoid implicitly deleting files.
> Maybe letting make erroring out would be less surprising.
> A clean source tree is necessary for the testsuite anyways.

My problem for now is that we don't have an explicit "clean all archs".
I've been spending a lot of time running and re-running the installation
to see it fail in the middle after a minute, it was really a pain that I
imagine nobody would accept. We could have a "clean-all-archs" target to
avoid the surprise and mention it in the help command as a prerequisite.
I've also been wondering if using O= could alleviate the need for mrproper,
but I don't know (not tried yet).

> > +	for file in $$(find "$(OUTPUT)sysroot/include/"asm-arch-* -maxdepth 1 -name '*.h' -printf '%P\n'); do \
> > +		sed -e "s!_ASMFILE_!$$file!" asm-template.h > "$(OUTPUT)sysroot/include/asm/$$file"; \
> > +	done
> 
> I think it should be possible to use the existing arch.h as template.
> That would avoid some duplication and busywork.

Yes it could, but see my comments above, arch.h is nolibc and I still
suspect there can be interest in installing headers for all supported
archs outside of nolibc.

All of this means that such points are not yet clarified. I'll leave
that at rest for a while, seeing if others chime in any direction, so
as to try to propose something that better matches everyone's needs,

Thanks,
Willy

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [RFC PATCH v2 2/2] tools/nolibc: add a new "install_all_archs" target
  2025-07-06  7:01     ` Willy Tarreau
@ 2025-07-07 15:41       ` Thomas Weißschuh
  0 siblings, 0 replies; 7+ messages in thread
From: Thomas Weißschuh @ 2025-07-07 15:41 UTC (permalink / raw)
  To: Willy Tarreau; +Cc: Thomas Weißschuh, linux-kernel, Arnd Bergmann

On 2025-07-06 09:01:24+0200, Willy Tarreau wrote:

<snip>

> If it is made clear that there is no interest in packaging uapi headers
> for archs not supported by nolibc, then this means that nolibc can be
> the place where the archs to be packaged is decided, and the two can
> be more tightly coupled, but as of now, this lack of interest is not
> yet obvious to me.

Given that this functionality was removed from kbuild recently I assume
that it there is limited interest to get it back.
Also in v2 none of the kbuild maintainers or list are Cc-ed anymore.

IMO we can apply patch 1 first and on its own.


Thomas

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2025-07-07 15:41 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-06-29 17:07 [RFC PATCH v2 0/2] tools/nolibc: install unified multi-arch headers Willy Tarreau
2025-06-29 17:07 ` [RFC PATCH v2 1/2] tools/nolibc: make the "headers" target install all supported archs Willy Tarreau
2025-06-29 20:17   ` Thomas Weißschuh
2025-06-29 17:07 ` [RFC PATCH v2 2/2] tools/nolibc: add a new "install_all_archs" target Willy Tarreau
2025-06-29 20:44   ` Thomas Weißschuh
2025-07-06  7:01     ` Willy Tarreau
2025-07-07 15:41       ` Thomas Weißschuh

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).