Re: [PATCH 4/4] kbuild: cross-compile linux-headers package when possible

[Ron Economos <re@w6rz.net>]

On 10/17/24 12:24 PM, Nicolas Schier wrote:
> On Thu, Oct 17, 2024 at 07:45:57AM -0700 Ron Economos wrote:
>> On 7/27/24 12:42 AM, Masahiro Yamada wrote:
>>> A long standing issue in the upstream kernel packaging is that the
>>> linux-headers package is not cross-compiled.
>>>
>>> For example, you can cross-build Debian packages for arm64 by running
>>> the following command:
>>>
>>>     $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bindeb-pkg
>>>
>>> However, the generated linux-headers-*_arm64.deb is useless because the
>>> host programs in it were built for your build machine architecture
>>> (likely x86), not arm64.
>>>
>>> The Debian kernel maintains its own Makefiles to cross-compile host
>>> tools without relying on Kbuild. [1]
>>>
>>> Instead of adding such full custom Makefiles, this commit adds a small
>>> piece of code to cross-compile host programs located under the scripts/
>>> directory.
>>>
>>> A straightforward solution is to pass HOSTCC=${CROSS_COMPILE}gcc, but it
>>> would also cross-compile scripts/basic/fixdep, which needs to be native
>>> to process the if_changed_dep macro. (This approach may work under some
>>> circumstances; you can execute foreign architecture programs with the
>>> help of binfmt_misc because Debian systems enable CONFIG_BINFMT_MISC,
>>> but it would require installing QEMU and libc for that architecture.)
>>>
>>> A trick is to use the external module build (KBUILD_EXTMOD=), which
>>> does not rebuild scripts/basic/fixdep. ${CC} needs to be able to link
>>> userspace programs (CONFIG_CC_CAN_LINK=y).
>>>
>>> There are known limitations:
>>>
>>>    - GCC plugins
>>>
>>>      It would possible to rebuild GCC plugins for the target architecture
>>>      by passing HOSTCXX=${CROSS_COMPILE}g++ with necessary packages
>>>      installed, but gcc on the installed system emits
>>>      "cc1: error: incompatible gcc/plugin versions". I did not find a
>>>      solution for this because 'gcc' on a foreign architecture is a
>>>      different compiler after all.
>>>
>>>    - objtool and resolve_btfids
>>>
>>>      These are built by the tools build system. They are not covered by
>>>      the current solution.
>>>
>>> I only tested this with Debian, but it should work for other package
>>> systems as well.
>>>
>>> [1]: https://salsa.debian.org/kernel-team/linux/-/blob/debian/6.9.9-1/debian/rules.real#L586
>>>
>>> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
>>> ---
>>>
>>>    scripts/package/install-extmod-build | 34 ++++++++++++++++++++++++++++
>>>    1 file changed, 34 insertions(+)
>>>
>>> diff --git a/scripts/package/install-extmod-build b/scripts/package/install-extmod-build
>>> index cc335945dfbc..0b56d3d7b48f 100755
>>> --- a/scripts/package/install-extmod-build
>>> +++ b/scripts/package/install-extmod-build
>>> @@ -43,4 +43,38 @@ mkdir -p "${destdir}"
>>>    	fi
>>>    } | tar -c -f - -T - | tar -xf - -C "${destdir}"
>>> +# When ${CC} and ${HOSTCC} differ, we are likely cross-compiling. Rebuild host
>>> +# programs using ${CC}. This assumes CC=${CROSS_COMPILE}gcc, which is usually
>>> +# the case for package building. It does not cross-compile when CC=clang.
>>> +#
>>> +# This caters to host programs that participate in Kbuild. objtool and
>>> +# resolve_btfids are out of scope.
>>> +if [ "${CC}" != "${HOSTCC}" ] && is_enabled CONFIG_CC_CAN_LINK; then
>>> +	echo "Rebuilding host programs with ${CC}..."
>>> +
>>> +	cat <<-'EOF' >  "${destdir}/Kbuild"
>>> +	subdir-y := scripts
>>> +	EOF
>>> +
>>> +	# HOSTCXX is not overridden. The C++ compiler is used to build:
>>> +	# - scripts/kconfig/qconf, which is unneeded for external module builds
>>> +	# - GCC plugins, which will not work on the installed system even with
>>> +	#   being rebuilt.
>>> +	#
>>> +	# Use the single-target build to avoid the modpost invocation, which
>>> +	# would overwrite Module.symvers.
>>> +	"${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/
>>> +
>>> +	cat <<-'EOF' >  "${destdir}/scripts/Kbuild"
>>> +	subdir-y := basic
>>> +	hostprogs-always-y := mod/modpost
>>> +	mod/modpost-objs := $(addprefix mod/, modpost.o file2alias.o sumversion.o symsearch.o)
>>> +	EOF
>>> +
>>> +	# Run once again to rebuild scripts/basic/ and scripts/mod/modpost.
>>> +	"${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/
>>> +
>>> +	rm -f "${destdir}/Kbuild" "${destdir}/scripts/Kbuild"
>>> +fi
>>> +
>>>    find "${destdir}" \( -name '.*.cmd' -o -name '*.o' \) -delete
>> This patch causes a build error when cross-compiling for RISC-V. I'm using
>> the cross-compiler from https://github.com/riscv-collab/riscv-gnu-toolchain.
>> When trying to build .debs with:
>>
>> make CROSS_COMPILE=riscv64-unknown-linux-gnu- ARCH=riscv INSTALL_MOD_STRIP=1
>> "KCFLAGS=-mtune=sifive-7-series" LOCALVERSION= bindeb-pkg
>>
>> I get the following error:
>>
>> Rebuilding host programs with riscv64-unknown-linux-gnu-gcc...
>>    HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/genksyms.o
>>    YACC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/parse.tab.[ch]
>>    HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/parse.tab.o
>>    LEX debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/lex.lex.c
>>    HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/lex.lex.o
>>    HOSTLD debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/genksyms
>>    HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/selinux/genheaders/genheaders
>>    HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/selinux/mdp/mdp
>>    HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/kallsyms
>>    HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/sorttable
>>    HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/asn1_compiler
>>    HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/sign-file
>>
>> debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/sign-file.c:25:10:
>> fatal error: openssl/opensslv.h: No such file or directory
>>     25 | #include <openssl/opensslv.h>
>>        |          ^~~~~~~~~~~~~~~~~~~~
>> compilation terminated.
> I guess you have openssl/opensslv.h available on your system, do you?  (In
> Debian/Ubuntu package libssl-dev or similar)
>
> Can you natively build a kernel with a similar kernel config?
>
> Kind regards,
> Nicolas

Yes, I have /usr/include/openssl/opensslv.h on my system. But that's the 
x86 version. The cross compiler can't use that.

A native build works fine.

Ron