Re: [PATCH 2/2] kbuild: userprogs: also inherit byte order and ABI from kernel

Thomas Weißschuh
2025-10-02T13:23:10Z

Hi Nicolas and Nathan,

On Wed, Sep 03, 2025 at 03:31:31PM -0700, Nathan Chancellor wrote:
> On Mon, Sep 01, 2025 at 11:51:03AM +0200, Thomas Weißschuh wrote:
> > Exactly. The normal cases can be handled generically. For example the kconfig
> > below works for architectures which only differ in byte order and 32bit/64bit,
> > which are most of them. MIPS should require more logic.
> > Also I'm ignoring x32, as it is never the kernel's native ABI.
> >
> > config CC_CAN_LINK
> > bool
> > + default $(cc_can_link_user,$(m64-flag) -mlittle-endian) if 64BIT && CPU_LITTLE_ENDIAN
> > + default $(cc_can_link_user,$(m64-flag) -mbig-endian) if 64BIT && CPU_BIG_ENDIAN
> > default $(cc_can_link_user,$(m64-flag)) if 64BIT
> > + default $(cc_can_link_user,$(m32-flag) -mlittle-endian) if CPU_LITTLE_ENDIAN
> > + default $(cc_can_link_user,$(m32-flag) -mbig-endian) if CPU_BIG_ENDIAN
> > default $(cc_can_link_user,$(m32-flag))
> >
> >
> > > Feels like that could get complicated quickly but this would probably be
> > > the objectively most robust and "hands off" option.
> >
> > Agreed.
>
> Nicolas might feel differently but this does not seem terrible to me,
> especially with a macro to wrap the common logic, which is where I felt
> like things could get unwieldy. Feel free to send an RFC if it is not
> too much work.

I investigated this some more and didn't really like the end result. The
problem is that $(m32-flag) and $(m64-flag) will expand to nothing if the
compiler does not support -m32/-m64. So for architectures which use
different flags the current logic will just ignore the bitness. One way
around this would be a mapping from -m32/-m64 to architecture-specific
flags inside cc-can-link.sh, similar to what I already did before for
the mapping of -mlittle-endian to -EL on MIPS. But we'll end up with a
bunch of architecture-specific details hidden away in a non-generic
shellscript. And the interactions are very non-obvious and brittle.
Then I'd rather have the architecture-specific bits openly in proper
architecture code.

See my current proposal, using x86 as example below. It will require
code for each architecture, but there are not that many of them.
And the configuration matrix for each architecture only contains a
relative small set of actually supported configurations.
Unfortunately I don't see a generic way to deduplicate the flag values
between ARCH_CC_CAN_LINK ARCH_USERPROGS_CFLAGS. Each architecture can
use a macro if they so prefer.

When the "interesting" architectures are done we can also slim down the
generic implementation to not use any special arguments and that would
be enough for the simple architectures.

For the future I would like to introduce CC_CAN_LINK_STATIC again.
With the scheme from below this would mean to duplicate all the kconfig
symbols for each architecture again. One way around would be to change
ARCH_CC_CAN_LINK from bool to string. And then let cc-can-link.sh test
for both static and dynamic linking in one go and return either
"dynamic,static", "dynamic" or "static" which then can be mapped to
CC_CAN_LINK and CC_CAN_LINK_STATIC by generic logic.

What do you think?

diff --git a/Makefile b/Makefile
index d37dca7850b3..17123948a4fa 100644
--- a/Makefile
+++ b/Makefile
@@ -1135,7 +1135,15 @@ LDFLAGS_vmlinux += --emit-relocs --discard-none
endif

# Align the bit size of userspace programs with the kernel
-USERFLAGS_FROM_KERNEL := -m32 -m64 --target=%
+USERFLAGS_FROM_KERNEL := --target=%
+
+ifdef CONFIG_ARCH_USERPROGS_CFLAGS
+KBUILD_USERCFLAGS += $(CONFIG_ARCH_USERPROGS_CFLAGS)
+KBUILD_USERLDFLAGS += $(CONFIG_ARCH_USERPROGS_CFLAGS)
+else
+USERFLAGS_FROM_KERNEL += -m32 -m64
+endif
+
KBUILD_USERCFLAGS += $(filter $(USERFLAGS_FROM_KERNEL), $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS))
KBUILD_USERLDFLAGS += $(filter $(USERFLAGS_FROM_KERNEL), $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS))

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 75f3de70df51..162c71c117bc 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -332,6 +332,17 @@ config X86
select SCHED_SMT if SMP
select ARCH_SUPPORTS_SCHED_CLUSTER if SMP
select ARCH_SUPPORTS_SCHED_MC if SMP
+ select ARCH_HAS_CC_CAN_LINK
+
+config ARCH_CC_CAN_LINK
+ bool
+ default $(cc_can_link_user,-m64) if 64BIT
+ default $(cc_can_link_user,-m32) if !64BIT
+
+config ARCH_USERPROGS_CFLAGS
+ string
+ default "-m64" if 64BIT
+ default "-m32" if !64BIT

config INSTRUCTION_DECODER
def_bool y
diff --git a/init/Kconfig b/init/Kconfig
index f3b13463ec26..5ca2f3289020 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -82,8 +82,13 @@ config RUSTC_LLVM_VERSION
int
default $(rustc-llvm-version)

+# Might be removed when all architectures are migrated
+config ARCH_HAS_CC_CAN_LINK
+ bool
+
config CC_CAN_LINK
bool
+ default ARCH_CC_CAN_LINK if ARCH_HAS_CC_CAN_LINK
default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(USERCFLAGS) $(USERLDFLAGS) $(m64-flag)) if 64BIT
default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(USERCFLAGS) $(USERLDFLAGS) $(m32-flag))

diff --git a/scripts/Kconfig.include b/scripts/Kconfig.include
index 33193ca6e803..0c8dbfbce415 100644
--- a/scripts/Kconfig.include
+++ b/scripts/Kconfig.include
@@ -75,3 +75,6 @@ rustc-llvm-version := $(shell,$(srctree)/scripts/rustc-llvm-version.sh $(RUSTC))
# If you are testing for unstable features, consider testing RUSTC_VERSION
# instead, as features may have different completeness while available.
rustc-option = $(success,trap "rm -rf .tmp_$$" EXIT; mkdir .tmp_$$; $(RUSTC) $(1) --crate-type=rlib /dev/null --out-dir=.tmp_$$ -o .tmp_$$/tmp.rlib)
+
+# Test whether the compiler can link userspace applications
+cc_can_link_user = $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(USERCFLAGS) $(USERLDFLAGS) $(1))