All of lore.kernel.org
 help / color / mirror / Atom feed
From: Masahiro Yamada <masahiroy@kernel.org>
To: linux-kbuild@vger.kernel.org
Cc: linux-kernel@vger.kernel.org, Kees Cook <kees@kernel.org>,
	Nathan Chancellor <nathan@kernel.org>,
	Nicolas Schier <nicolas@fjasle.eu>,
	Ben Hutchings <ben@decadent.org.uk>,
	Masahiro Yamada <masahiroy@kernel.org>
Subject: [PATCH 4/4] kbuild: cross-compile linux-headers package when possible
Date: Sat, 27 Jul 2024 16:42:04 +0900	[thread overview]
Message-ID: <20240727074526.1771247-5-masahiroy@kernel.org> (raw)
In-Reply-To: <20240727074526.1771247-1-masahiroy@kernel.org>

A long standing issue in the upstream kernel packaging is that the
linux-headers package is not cross-compiled.

For example, you can cross-build Debian packages for arm64 by running
the following command:

  $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bindeb-pkg

However, the generated linux-headers-*_arm64.deb is useless because the
host programs in it were built for your build machine architecture
(likely x86), not arm64.

The Debian kernel maintains its own Makefiles to cross-compile host
tools without relying on Kbuild. [1]

Instead of adding such full custom Makefiles, this commit adds a small
piece of code to cross-compile host programs located under the scripts/
directory.

A straightforward solution is to pass HOSTCC=${CROSS_COMPILE}gcc, but it
would also cross-compile scripts/basic/fixdep, which needs to be native
to process the if_changed_dep macro. (This approach may work under some
circumstances; you can execute foreign architecture programs with the
help of binfmt_misc because Debian systems enable CONFIG_BINFMT_MISC,
but it would require installing QEMU and libc for that architecture.)

A trick is to use the external module build (KBUILD_EXTMOD=), which
does not rebuild scripts/basic/fixdep. ${CC} needs to be able to link
userspace programs (CONFIG_CC_CAN_LINK=y).

There are known limitations:

 - GCC plugins

   It would possible to rebuild GCC plugins for the target architecture
   by passing HOSTCXX=${CROSS_COMPILE}g++ with necessary packages
   installed, but gcc on the installed system emits
   "cc1: error: incompatible gcc/plugin versions". I did not find a
   solution for this because 'gcc' on a foreign architecture is a
   different compiler after all.

 - objtool and resolve_btfids

   These are built by the tools build system. They are not covered by
   the current solution.

I only tested this with Debian, but it should work for other package
systems as well.

[1]: https://salsa.debian.org/kernel-team/linux/-/blob/debian/6.9.9-1/debian/rules.real#L586

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

 scripts/package/install-extmod-build | 34 ++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/scripts/package/install-extmod-build b/scripts/package/install-extmod-build
index cc335945dfbc..0b56d3d7b48f 100755
--- a/scripts/package/install-extmod-build
+++ b/scripts/package/install-extmod-build
@@ -43,4 +43,38 @@ mkdir -p "${destdir}"
 	fi
 } | tar -c -f - -T - | tar -xf - -C "${destdir}"
 
+# When ${CC} and ${HOSTCC} differ, we are likely cross-compiling. Rebuild host
+# programs using ${CC}. This assumes CC=${CROSS_COMPILE}gcc, which is usually
+# the case for package building. It does not cross-compile when CC=clang.
+#
+# This caters to host programs that participate in Kbuild. objtool and
+# resolve_btfids are out of scope.
+if [ "${CC}" != "${HOSTCC}" ] && is_enabled CONFIG_CC_CAN_LINK; then
+	echo "Rebuilding host programs with ${CC}..."
+
+	cat <<-'EOF' >  "${destdir}/Kbuild"
+	subdir-y := scripts
+	EOF
+
+	# HOSTCXX is not overridden. The C++ compiler is used to build:
+	# - scripts/kconfig/qconf, which is unneeded for external module builds
+	# - GCC plugins, which will not work on the installed system even with
+	#   being rebuilt.
+	#
+	# Use the single-target build to avoid the modpost invocation, which
+	# would overwrite Module.symvers.
+	"${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/
+
+	cat <<-'EOF' >  "${destdir}/scripts/Kbuild"
+	subdir-y := basic
+	hostprogs-always-y := mod/modpost
+	mod/modpost-objs := $(addprefix mod/, modpost.o file2alias.o sumversion.o symsearch.o)
+	EOF
+
+	# Run once again to rebuild scripts/basic/ and scripts/mod/modpost.
+	"${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/
+
+	rm -f "${destdir}/Kbuild" "${destdir}/scripts/Kbuild"
+fi
+
 find "${destdir}" \( -name '.*.cmd' -o -name '*.o' \) -delete
-- 
2.43.0


  parent reply	other threads:[~2024-07-27  7:45 UTC|newest]

Thread overview: 31+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-07-27  7:42 [PATCH 0/4] kbuild: cross-compile linux-headers package Masahiro Yamada
2024-07-27  7:42 ` [PATCH 1/4] modpost: remove unused HOST_ELFCLASS Masahiro Yamada
2024-07-31 20:43   ` Nicolas Schier
2024-07-27  7:42 ` [PATCH 2/4] modpost: detect endianness on run-time Masahiro Yamada
2024-07-31 20:47   ` Nicolas Schier
2024-07-27  7:42 ` [PATCH 3/4] kbuild: slim down package for building external modules Masahiro Yamada
2024-07-31 21:01   ` Nicolas Schier
2024-08-24 12:27   ` Thomas Weißschuh
2024-08-24 16:58     ` Masahiro Yamada
2025-02-18 20:25   ` Jeffrey Hugo
2025-02-20 10:03     ` Nicolas Schier
2025-02-20 15:03       ` Jeffrey Hugo
2025-02-20 15:54         ` Nicolas Schier
2025-02-20 16:31           ` Jeffrey Hugo
2025-02-20 16:49             ` Greg KH
2025-02-20 17:24               ` Jeffrey Hugo
2025-02-20 17:43                 ` Greg KH
2025-02-20 18:12                   ` Masahiro Yamada
2025-02-28 22:04                   ` Jeffrey Hugo
2025-02-20 17:57                 ` Masahiro Yamada
2024-07-27  7:42 ` Masahiro Yamada [this message]
2024-07-30  1:03   ` [PATCH 4/4] kbuild: cross-compile linux-headers package when possible Kees Cook
2024-08-01  2:26     ` Masahiro Yamada
2024-07-31 21:10   ` Nicolas Schier
2024-08-01  2:37     ` Masahiro Yamada
2024-08-01  7:04       ` Nicolas Schier
2024-10-17 14:45   ` Ron Economos
2024-10-17 19:24     ` Nicolas Schier
2024-10-17 19:34       ` Ron Economos
2024-11-14 20:57         ` Charlie Jenkins
2024-11-16  8:03           ` Masahiro Yamada

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240727074526.1771247-5-masahiroy@kernel.org \
    --to=masahiroy@kernel.org \
    --cc=ben@decadent.org.uk \
    --cc=kees@kernel.org \
    --cc=linux-kbuild@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=nathan@kernel.org \
    --cc=nicolas@fjasle.eu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.