[PATCH 0/2] kbuild: userprogs: two fixes for LLVM=1

[PATCH 0/2] kbuild: userprogs: two fixes for LLVM=1

[Thomas Weißschuh]

Fix two issues when cross-building userprogs with clang.

Reproducer, using nolibc to avoid libc requirements for cross building:

$ tail -2 init/Makefile
userprogs-always-y += test-llvm
test-llvm-userccflags += -nostdlib -nolibc -static -isystem usr/ -include $(srctree)/tools/include/nolibc/nolibc.h

$ cat init/test-llvm.c
int main(void)
{
	return 0;
}

$ make ARCH=arm64 LLVM=1 allnoconfig headers_install init/

Validate that init/test-llvm builds and has the correct binary format.

Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
---
Thomas Weißschuh (2):
      kbuild: userprogs: fix bitsize and target detection on clang
      kbuild: userprogs: use lld to link through clang

 Makefile | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
---
base-commit: 2014c95afecee3e76ca4a56956a936e23283f05b
change-id: 20250213-kbuild-userprog-fixes-4f07b62ae818

Best regards,
-- 
Thomas Weißschuh <thomas.weissschuh@linutronix.de>

[PATCH 1/2] kbuild: userprogs: fix bitsize and target detection on clang

[Thomas Weißschuh]

scripts/Makefile.clang was changed in the linked commit to move --target from
KBUILD_CFLAGS to KBUILD_CPPFLAGS, as that generally has a broader scope.
However that variable is not inspected by the userprogs logic,
breaking cross compilation on clang.

Use both variables to detect bitsize and target arguments for userprogs.

Fixes: feb843a469fb ("kbuild: add $(CLANG_FLAGS) to KBUILD_CPPFLAGS")
Cc: stable@vger.kernel.org
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
---
 Makefile | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 9e0d63d9d94b90672f91929e5e148e5a0c346cb6..bb5737ce7f9e79f4023c9c1f578a49a951d1e239 100644
--- a/Makefile
+++ b/Makefile
@@ -1120,8 +1120,8 @@ LDFLAGS_vmlinux += --orphan-handling=$(CONFIG_LD_ORPHAN_WARN_LEVEL)
 endif
 
 # Align the bit size of userspace programs with the kernel
-KBUILD_USERCFLAGS  += $(filter -m32 -m64 --target=%, $(KBUILD_CFLAGS))
-KBUILD_USERLDFLAGS += $(filter -m32 -m64 --target=%, $(KBUILD_CFLAGS))
+KBUILD_USERCFLAGS  += $(filter -m32 -m64 --target=%, $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS))
+KBUILD_USERLDFLAGS += $(filter -m32 -m64 --target=%, $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS))
 
 # make the checker run with the right architecture
 CHECKFLAGS += --arch=$(ARCH)

-- 
2.48.1

Re: [PATCH 1/2] kbuild: userprogs: fix bitsize and target detection on clang

[Nathan Chancellor]

On Thu, Feb 13, 2025 at 03:55:17PM +0100, Thomas Weißschuh wrote:
> scripts/Makefile.clang was changed in the linked commit to move --target from
> KBUILD_CFLAGS to KBUILD_CPPFLAGS, as that generally has a broader scope.
> However that variable is not inspected by the userprogs logic,
> breaking cross compilation on clang.
> 
> Use both variables to detect bitsize and target arguments for userprogs.
> 
> Fixes: feb843a469fb ("kbuild: add $(CLANG_FLAGS) to KBUILD_CPPFLAGS")
> Cc: stable@vger.kernel.org
> Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>

Reviewed-by: Nathan Chancellor <nathan@kernel.org>

> ---
>  Makefile | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 9e0d63d9d94b90672f91929e5e148e5a0c346cb6..bb5737ce7f9e79f4023c9c1f578a49a951d1e239 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1120,8 +1120,8 @@ LDFLAGS_vmlinux += --orphan-handling=$(CONFIG_LD_ORPHAN_WARN_LEVEL)
>  endif
>  
>  # Align the bit size of userspace programs with the kernel
> -KBUILD_USERCFLAGS  += $(filter -m32 -m64 --target=%, $(KBUILD_CFLAGS))
> -KBUILD_USERLDFLAGS += $(filter -m32 -m64 --target=%, $(KBUILD_CFLAGS))
> +KBUILD_USERCFLAGS  += $(filter -m32 -m64 --target=%, $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS))
> +KBUILD_USERLDFLAGS += $(filter -m32 -m64 --target=%, $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS))
>  
>  # make the checker run with the right architecture
>  CHECKFLAGS += --arch=$(ARCH)
> 
> -- 
> 2.48.1
> 

Re: [PATCH 1/2] kbuild: userprogs: fix bitsize and target detection on clang

[Masahiro Yamada]

On Thu, Feb 13, 2025 at 11:55 PM Thomas Weißschuh
<thomas.weissschuh@linutronix.de> wrote:
>
> scripts/Makefile.clang was changed in the linked commit to move --target from
> KBUILD_CFLAGS to KBUILD_CPPFLAGS, as that generally has a broader scope.
> However that variable is not inspected by the userprogs logic,
> breaking cross compilation on clang.
>
> Use both variables to detect bitsize and target arguments for userprogs.
>
> Fixes: feb843a469fb ("kbuild: add $(CLANG_FLAGS) to KBUILD_CPPFLAGS")
> Cc: stable@vger.kernel.org
> Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
> ---

Applied to linux-kbuild/fixes.
Thanks!




-- 
Best Regards
Masahiro Yamada

[PATCH 2/2] kbuild: userprogs: use lld to link through clang

[Thomas Weißschuh]

The userprog infrastructure links objects files through $(CC).
Either explicitly by manually calling $(CC) on multiple object files or
implicitly by directly compiling a source file to an executable.
The documentation at Documentation/kbuild/llvm.rst indicates that ld.lld would
be used for linking if LLVM=1 is specified.
However clang instead will use either a globally installed cross linker from
$PATH called ${target}-ld or fall back to the system linker, which probably
does not support crosslinking.
For the normal kernel build this is not an issue because the linker is always
executed directly, without the compiler being involved.

Fix this by passing -fuse-lld and let clang find its matching lld.

Fixes: 7f3a59db274c ("kbuild: add infrastructure to build userspace programs")
Cc: stable@vger.kernel.org
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
---
 Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Makefile b/Makefile
index bb5737ce7f9e79f4023c9c1f578a49a951d1e239..b4c208ae4041c1f4e32c2a158322422ce7353d06 100644
--- a/Makefile
+++ b/Makefile
@@ -510,6 +510,7 @@ OBJCOPY		= $(LLVM_PREFIX)llvm-objcopy$(LLVM_SUFFIX)
 OBJDUMP		= $(LLVM_PREFIX)llvm-objdump$(LLVM_SUFFIX)
 READELF		= $(LLVM_PREFIX)llvm-readelf$(LLVM_SUFFIX)
 STRIP		= $(LLVM_PREFIX)llvm-strip$(LLVM_SUFFIX)
+KBUILD_USERLDFLAGS += -fuse-ld=lld
 else
 CC		= $(CROSS_COMPILE)gcc
 LD		= $(CROSS_COMPILE)ld

-- 
2.48.1

Re: [PATCH 2/2] kbuild: userprogs: use lld to link through clang

[Nathan Chancellor]

Hi Thomas,

On Thu, Feb 13, 2025 at 03:55:18PM +0100, Thomas Weißschuh wrote:
> The userprog infrastructure links objects files through $(CC).
> Either explicitly by manually calling $(CC) on multiple object files or
> implicitly by directly compiling a source file to an executable.
> The documentation at Documentation/kbuild/llvm.rst indicates that ld.lld would
> be used for linking if LLVM=1 is specified.
> However clang instead will use either a globally installed cross linker from
> $PATH called ${target}-ld or fall back to the system linker, which probably
> does not support crosslinking.
> For the normal kernel build this is not an issue because the linker is always
> executed directly, without the compiler being involved.
> 
> Fix this by passing -fuse-lld and let clang find its matching lld.
> 
> Fixes: 7f3a59db274c ("kbuild: add infrastructure to build userspace programs")
> Cc: stable@vger.kernel.org
> Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>

First of all, thank you for catching and noticing this!

> ---
>  Makefile | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/Makefile b/Makefile
> index bb5737ce7f9e79f4023c9c1f578a49a951d1e239..b4c208ae4041c1f4e32c2a158322422ce7353d06 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -510,6 +510,7 @@ OBJCOPY		= $(LLVM_PREFIX)llvm-objcopy$(LLVM_SUFFIX)
>  OBJDUMP		= $(LLVM_PREFIX)llvm-objdump$(LLVM_SUFFIX)
>  READELF		= $(LLVM_PREFIX)llvm-readelf$(LLVM_SUFFIX)
>  STRIP		= $(LLVM_PREFIX)llvm-strip$(LLVM_SUFFIX)
> +KBUILD_USERLDFLAGS += -fuse-ld=lld

Now that our minimum supported version upstream is 13.0.1, I think we
can make this

  KBUILD_USERLDFLAGS += --ld-path=$(LD)

as it should respect the user's choice of linker a little bit more, such
as if they specific LLVM=<prefix>/bin/ or LLVM=-20. That reminds me that
I can clean up what I did in commit 4406b12214f6 ("powerpc/vdso: Link
with ld.lld when requested").

Additionally, this would not fix someone using CC=clang and LD=ld.lld
(it is uncommon but still techincally supported) so could we use a check
like

  ifeq ($(CONFIG_CC_IS_CLANG)$(CONFIG_LD_IS_LLD),yy)
  KBUILD_USERLDFLAGS += --ld-path=$(LD)
  endif

further down in Makefile to make it more robust?

The stable backport may want to use cc-option like I did for the powerpc
vdso since there is a lower minimum supported version of LLVM there.

>  else
>  CC		= $(CROSS_COMPILE)gcc
>  LD		= $(CROSS_COMPILE)ld
> 
> -- 
> 2.48.1
> 

Cheers,
Nathan

Re: [PATCH 2/2] kbuild: userprogs: use lld to link through clang

[Thomas Weißschuh]

On Thu, Feb 13, 2025 at 10:54:37AM -0700, Nathan Chancellor wrote:
> Hi Thomas,
> 
> On Thu, Feb 13, 2025 at 03:55:18PM +0100, Thomas Weißschuh wrote:
> > The userprog infrastructure links objects files through $(CC).
> > Either explicitly by manually calling $(CC) on multiple object files or
> > implicitly by directly compiling a source file to an executable.
> > The documentation at Documentation/kbuild/llvm.rst indicates that ld.lld would
> > be used for linking if LLVM=1 is specified.
> > However clang instead will use either a globally installed cross linker from
> > $PATH called ${target}-ld or fall back to the system linker, which probably
> > does not support crosslinking.
> > For the normal kernel build this is not an issue because the linker is always
> > executed directly, without the compiler being involved.
> > 
> > Fix this by passing -fuse-lld and let clang find its matching lld.
> > 
> > Fixes: 7f3a59db274c ("kbuild: add infrastructure to build userspace programs")
> > Cc: stable@vger.kernel.org
> > Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
> 
> First of all, thank you for catching and noticing this!
> 
> > ---
> >  Makefile | 1 +
> >  1 file changed, 1 insertion(+)
> > 
> > diff --git a/Makefile b/Makefile
> > index bb5737ce7f9e79f4023c9c1f578a49a951d1e239..b4c208ae4041c1f4e32c2a158322422ce7353d06 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -510,6 +510,7 @@ OBJCOPY		= $(LLVM_PREFIX)llvm-objcopy$(LLVM_SUFFIX)
> >  OBJDUMP		= $(LLVM_PREFIX)llvm-objdump$(LLVM_SUFFIX)
> >  READELF		= $(LLVM_PREFIX)llvm-readelf$(LLVM_SUFFIX)
> >  STRIP		= $(LLVM_PREFIX)llvm-strip$(LLVM_SUFFIX)
> > +KBUILD_USERLDFLAGS += -fuse-ld=lld
> 
> Now that our minimum supported version upstream is 13.0.1, I think we
> can make this
> 
>   KBUILD_USERLDFLAGS += --ld-path=$(LD)
> 
> as it should respect the user's choice of linker a little bit more, such
> as if they specific LLVM=<prefix>/bin/ or LLVM=-20. That reminds me that
> I can clean up what I did in commit 4406b12214f6 ("powerpc/vdso: Link
> with ld.lld when requested").

That is much better, thanks!
I looked for something like this but could't find it.

> Additionally, this would not fix someone using CC=clang and LD=ld.lld
> (it is uncommon but still techincally supported) so could we use a check
> like
> 
>   ifeq ($(CONFIG_CC_IS_CLANG)$(CONFIG_LD_IS_LLD),yy)
>   KBUILD_USERLDFLAGS += --ld-path=$(LD)
>   endif
> 
> further down in Makefile to make it more robust?

Ack.

> The stable backport may want to use cc-option like I did for the powerpc
> vdso since there is a lower minimum supported version of LLVM there.

Ack.


FYI, in case you want to have a look:
The kunit config for powerpc is currently broken on LLVM=1.

$ ./tools/testing/kunit/kunit.py run --arch powerpc --make_options LLVM=1

arch/powerpc/boot/wrapper manually inspects with CROSS_COMPILE, which does not
exist when LLVM=1. Instead it falls back to the system objcopy, etc.

> >  else
> >  CC		= $(CROSS_COMPILE)gcc
> >  LD		= $(CROSS_COMPILE)ld

Re: [PATCH 2/2] kbuild: userprogs: use lld to link through clang

[Nathan Chancellor]

On Fri, Feb 14, 2025 at 08:40:22AM +0100, Thomas Weißschuh wrote:
> FYI, in case you want to have a look:
> The kunit config for powerpc is currently broken on LLVM=1.
> 
> $ ./tools/testing/kunit/kunit.py run --arch powerpc --make_options LLVM=1
> 
> arch/powerpc/boot/wrapper manually inspects with CROSS_COMPILE, which does not
> exist when LLVM=1. Instead it falls back to the system objcopy, etc.

Thanks for reminding me, it is a known papercut:

https://github.com/ClangBuiltLinux/linux/issues/1601

I have a WIP series but there are still some errors in the
configurations that I test that I have not investigated.

https://git.kernel.org/pub/scm/linux/kernel/git/nathan/linux.git/log/?h=b4/ppc-boot-llvm-1

Cheers,
Nathan