[PATCH 0/2] ARM: Fix ARM_VECTORS with CONFIG_LD_DEAD_CODE_DATA_ELIMINATION

[PATCH 0/2] ARM: Fix ARM_VECTORS with CONFIG_LD_DEAD_CODE_DATA_ELIMINATION

[Nathan Chancellor]

Hi all,

Christian sent a fix [1] for ARM_VECTORS with
CONFIG_LD_DEAD_CODE_DATA_ELIMINATION that exposed a deficiency in ld.lld
with regards to KEEP() within an OVERLAY description. I have fixed that
in ld.lld [2] and added a patch before Christian's to disallow
CONFIG_LD_DEAD_CODE_DATA_ELIMINATION when KEEP() cannot be used within
OVERLAY to keep everything working for all linkers.

[1]: https://lore.kernel.org/20250221125520.14035-1-ceggers@arri.de/
[2]: https://github.com/llvm/llvm-project/commit/381599f1fe973afad3094e55ec99b1620dba7d8c

---
Christian Eggers (1):
      ARM: add KEEP() keyword to ARM_VECTORS

Nathan Chancellor (1):
      ARM: Require linker to support KEEP within OVERLAY for DCE

 arch/arm/Kconfig                   |  2 +-
 arch/arm/include/asm/vmlinux.lds.h | 12 +++++++++---
 init/Kconfig                       |  5 +++++
 3 files changed, 15 insertions(+), 4 deletions(-)
---
base-commit: 80e54e84911a923c40d7bee33a34c1b4be148d7a
change-id: 20250311-arm-fix-vectors-with-linker-dce-83475b0b8f5b

Best regards,
-- 
Nathan Chancellor <nathan@kernel.org>

[PATCH 1/2] ARM: Require linker to support KEEP within OVERLAY for DCE

[Nathan Chancellor]

ld.lld prior to 21.0.0 does not support using the KEEP keyword within an
overlay description, which may be needed to avoid discarding necessary
sections within an overlay with '--gc-sections', which can be enabled
for the kernel via CONFIG_LD_DEAD_CODE_DATA_ELIMINATION.

Disallow CONFIG_LD_DEAD_CODE_DATA_ELIMINATION without support for KEEP
within OVERLAY and introduce a macro, OVERLAY_KEEP, that can be used to
conditionally add KEEP when it is properly supported to avoid breaking
old versions of ld.lld.

Cc: stable@vger.kernel.org
Link: https://github.com/llvm/llvm-project/commit/381599f1fe973afad3094e55ec99b1620dba7d8c
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
---
 arch/arm/Kconfig                   | 2 +-
 arch/arm/include/asm/vmlinux.lds.h | 6 ++++++
 init/Kconfig                       | 5 +++++
 3 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 835b5f100e92..f3f6b7a33b79 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -121,7 +121,7 @@ config ARM
 	select HAVE_KERNEL_XZ
 	select HAVE_KPROBES if !XIP_KERNEL && !CPU_ENDIAN_BE32 && !CPU_V7M
 	select HAVE_KRETPROBES if HAVE_KPROBES
-	select HAVE_LD_DEAD_CODE_DATA_ELIMINATION if (LD_VERSION >= 23600 || LD_IS_LLD)
+	select HAVE_LD_DEAD_CODE_DATA_ELIMINATION if (LD_VERSION >= 23600 || LD_CAN_USE_KEEP_IN_OVERLAY)
 	select HAVE_MOD_ARCH_SPECIFIC
 	select HAVE_NMI
 	select HAVE_OPTPROBES if !THUMB2_KERNEL
diff --git a/arch/arm/include/asm/vmlinux.lds.h b/arch/arm/include/asm/vmlinux.lds.h
index d60f6e83a9f7..0f8ef1ed725e 100644
--- a/arch/arm/include/asm/vmlinux.lds.h
+++ b/arch/arm/include/asm/vmlinux.lds.h
@@ -34,6 +34,12 @@
 #define NOCROSSREFS
 #endif
 
+#ifdef CONFIG_LD_CAN_USE_KEEP_IN_OVERLAY
+#define OVERLAY_KEEP(x)		KEEP(x)
+#else
+#define OVERLAY_KEEP(x)		x
+#endif
+
 /* Set start/end symbol names to the LMA for the section */
 #define ARM_LMA(sym, section)						\
 	sym##_start = LOADADDR(section);				\
diff --git a/init/Kconfig b/init/Kconfig
index d0d021b3fa3b..fc994f5cd5db 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -129,6 +129,11 @@ config CC_HAS_COUNTED_BY
 	# https://github.com/llvm/llvm-project/pull/112636
 	depends on !(CC_IS_CLANG && CLANG_VERSION < 190103)
 
+config LD_CAN_USE_KEEP_IN_OVERLAY
+	# ld.lld prior to 21.0.0 did not support KEEP within an overlay description
+	# https://github.com/llvm/llvm-project/pull/130661
+	def_bool LD_IS_BFD || LLD_VERSION >= 210000
+
 config RUSTC_HAS_COERCE_POINTEE
 	def_bool RUSTC_VERSION >= 108400
 

-- 
2.48.1

[PATCH 2/2] ARM: add KEEP() keyword to ARM_VECTORS

[Nathan Chancellor]

From: Christian Eggers <ceggers@arri.de>

Without this, the vectors are removed if LD_DEAD_CODE_DATA_ELIMINATION
is enabled.  At startup, the CPU (silently) hangs in the undefined
instruction exception as soon as the first timer interrupt arrives.

On my setup, the system also boots fine without the 2nd and 3rd KEEP()
statements, so I cannot tell whether these are actually required.

Cc: stable@vger.kernel.org
Fixes: ed0f94102251 ("ARM: 9404/1: arm32: enable HAVE_LD_DEAD_CODE_DATA_ELIMINATION")
Signed-off-by: Christian Eggers <ceggers@arri.de>
[nathan: Use OVERLAY_KEEP() to avoid breaking old ld.lld versions]
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
---
 arch/arm/include/asm/vmlinux.lds.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/arm/include/asm/vmlinux.lds.h b/arch/arm/include/asm/vmlinux.lds.h
index 0f8ef1ed725e..14811b4f48ec 100644
--- a/arch/arm/include/asm/vmlinux.lds.h
+++ b/arch/arm/include/asm/vmlinux.lds.h
@@ -131,13 +131,13 @@
 	__vectors_lma = .;						\
 	OVERLAY 0xffff0000 : NOCROSSREFS AT(__vectors_lma) {		\
 		.vectors {						\
-			*(.vectors)					\
+			OVERLAY_KEEP(*(.vectors))			\
 		}							\
 		.vectors.bhb.loop8 {					\
-			*(.vectors.bhb.loop8)				\
+			OVERLAY_KEEP(*(.vectors.bhb.loop8))		\
 		}							\
 		.vectors.bhb.bpiall {					\
-			*(.vectors.bhb.bpiall)				\
+			OVERLAY_KEEP(*(.vectors.bhb.bpiall))		\
 		}							\
 	}								\
 	ARM_LMA(__vectors, .vectors);					\

-- 
2.48.1

Re: [PATCH 1/2] ARM: Require linker to support KEEP within OVERLAY for DCE

[Linus Walleij]

On Tue, Mar 11, 2025 at 8:43 PM Nathan Chancellor <nathan@kernel.org> wrote:

> ld.lld prior to 21.0.0 does not support using the KEEP keyword within an
> overlay description, which may be needed to avoid discarding necessary
> sections within an overlay with '--gc-sections', which can be enabled
> for the kernel via CONFIG_LD_DEAD_CODE_DATA_ELIMINATION.
>
> Disallow CONFIG_LD_DEAD_CODE_DATA_ELIMINATION without support for KEEP
> within OVERLAY and introduce a macro, OVERLAY_KEEP, that can be used to
> conditionally add KEEP when it is properly supported to avoid breaking
> old versions of ld.lld.
>
> Cc: stable@vger.kernel.org
> Link: https://github.com/llvm/llvm-project/commit/381599f1fe973afad3094e55ec99b1620dba7d8c
> Signed-off-by: Nathan Chancellor <nathan@kernel.org>

Very clear and easy to follow.
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>

Yours,
Linus Walleij

Re: [PATCH 2/2] ARM: add KEEP() keyword to ARM_VECTORS

[Linus Walleij]

On Tue, Mar 11, 2025 at 8:44 PM Nathan Chancellor <nathan@kernel.org> wrote:

> From: Christian Eggers <ceggers@arri.de>
>
> Without this, the vectors are removed if LD_DEAD_CODE_DATA_ELIMINATION
> is enabled.  At startup, the CPU (silently) hangs in the undefined
> instruction exception as soon as the first timer interrupt arrives.
>
> On my setup, the system also boots fine without the 2nd and 3rd KEEP()
> statements, so I cannot tell whether these are actually required.
>
> Cc: stable@vger.kernel.org
> Fixes: ed0f94102251 ("ARM: 9404/1: arm32: enable HAVE_LD_DEAD_CODE_DATA_ELIMINATION")
> Signed-off-by: Christian Eggers <ceggers@arri.de>
> [nathan: Use OVERLAY_KEEP() to avoid breaking old ld.lld versions]
> Signed-off-by: Nathan Chancellor <nathan@kernel.org>

Reviewed-by: Linus Walleij <linus.walleij@linaro.org>

Yours,
Linus Walleij

Re: [PATCH 1/2] ARM: Require linker to support KEEP within OVERLAY for DCE

[Rob Landley]

[-- Attachment #1: Type: text/plain, Size: 1916 bytes --]

On 3/13/25 03:48, Linus Walleij wrote:
> On Tue, Mar 11, 2025 at 8:43 PM Nathan Chancellor <nathan@kernel.org> wrote:
> 
>> ld.lld prior to 21.0.0 does not support using the KEEP keyword within an
>> overlay description, which may be needed to avoid discarding necessary
>> sections within an overlay with '--gc-sections', which can be enabled
>> for the kernel via CONFIG_LD_DEAD_CODE_DATA_ELIMINATION.
>>
>> Disallow CONFIG_LD_DEAD_CODE_DATA_ELIMINATION without support for KEEP
>> within OVERLAY and introduce a macro, OVERLAY_KEEP, that can be used to
>> conditionally add KEEP when it is properly supported to avoid breaking
>> old versions of ld.lld.

I bisected the 6.15 armv7l build break my mkroot project hit to this 
commit (e7607f7d6d81):

   LD      .tmp_vmlinux1
Segmentation fault
make[2]: *** [scripts/Makefile.vmlinux:77: vmlinux] Error 139
make[1]: *** 
[/home/landley/toybox/clean/root/build/armv7l-tmp/linux/Makefile:1226: 
vmlinux] Error 2
make: *** [Makefile:251: __sub-make] Error 2

The toolchain in question was built from gcc 11.4.0 and binutils 2.33.1 
which were the newest versions supported by 
https://github.com/richfelker/musl-cross-make when the still-current 
musl release (1.2.5) came out.

You can grab a binary toolchain to smoketest with from 
https://landley.net/bin/toolchains/latest/armv7l-linux-musleabihf-cross.tar.xz 
and build using the attached miniconfig ala:

for i in distclean allnoconfig ""
do
   CROSS_COMPILE=armv7l-linux-musleabihf- make ARCH=arm \
     KCONFIG_ALLCONFIG=linux-miniconfig -j4 $i
done

This _just_ seems to affect armv7l: armv5l and aarch64 still build fine.

Rob

P.S.  Rich has since added newer gcc version support to MCM, but:

A) the binutils in MCM is still stuck at the old version and if you 
upgrade the gcc without the binutils it hits 
https://lkml.org/lkml/2023/8/9/890

B) I'm waiting for musl-1.2.6 before updating the toolchain.

[-- Attachment #2: linux-miniconfig --]
[-- Type: text/plain, Size: 1580 bytes --]

# make ARCH=arm allnoconfig KCONFIG_ALLCONFIG=linux-miniconfig
# make ARCH=arm -j $(nproc)
# boot zImage console=ttyAMA0

# architecture independent
CONFIG_PANIC_TIMEOUT=1
CONFIG_NO_HZ_IDLE=y
CONFIG_HIGH_RES_TIMERS=y
CONFIG_RD_GZIP=y
CONFIG_BINFMT_ELF=y
CONFIG_BINFMT_SCRIPT=y
CONFIG_BLK_DEV=y
CONFIG_BLK_DEV_INITRD=y
CONFIG_BLK_DEV_LOOP=y
CONFIG_EXT4_FS=y
CONFIG_EXT4_USE_FOR_EXT2=y
CONFIG_VFAT_FS=y
CONFIG_FAT_DEFAULT_UTF8=y
CONFIG_MISC_FILESYSTEMS=y
CONFIG_NLS_CODEPAGE_437=y
CONFIG_NLS_ISO8859_1=y
CONFIG_SQUASHFS=y
CONFIG_SQUASHFS_XATTR=y
CONFIG_SQUASHFS_ZLIB=y
CONFIG_TMPFS=y
CONFIG_TMPFS_POSIX_ACL=y
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
CONFIG_NET=y
CONFIG_NETDEVICES=y
CONFIG_NET_CORE=y
CONFIG_NETCONSOLE=y
CONFIG_PACKET=y
CONFIG_UNIX=y
CONFIG_INET=y
CONFIG_IPV6=y
CONFIG_ETHERNET=y
CONFIG_COMPAT_32BIT_TIME=y
CONFIG_EARLY_PRINTK=y
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y

# architecture specific
CONFIG_MMU=y
CONFIG_SOC_DRA7XX=y
CONFIG_VDSO=y
CONFIG_CPU_IDLE=y
CONFIG_KERNEL_MODE_NEON=y
CONFIG_ARCH_MULTI_V7=y
CONFIG_ARCH_VIRT=y
CONFIG_ARCH_OMAP2PLUS_TYPICAL=y
CONFIG_ARCH_ALPINE=y
CONFIG_ARM_THUMB=y
CONFIG_ARM_CPUIDLE=y
CONFIG_ARM_LPAE=y
CONFIG_ATA=y
CONFIG_ATA_SFF=y
CONFIG_ATA_BMDMA=y
CONFIG_ATA_PIIX=y
CONFIG_ATA_GENERIC=y
CONFIG_VIRTIO_MENU=y
CONFIG_VIRTIO_NET=y
CONFIG_VIRTIO_BLK=y
CONFIG_VIRTIO_PCI=y
CONFIG_VIRTIO_MMIO=y
CONFIG_SERIAL_AMBA_PL011=y
CONFIG_SERIAL_AMBA_PL011_CONSOLE=y
CONFIG_RTC_CLASS=y
CONFIG_RTC_HCTOSYS=y
CONFIG_RTC_DRV_PL031=y
CONFIG_PATA_PLATFORM=y
CONFIG_PATA_OF_PLATFORM=y
CONFIG_PCI=y
CONFIG_PCI_HOST_GENERIC=y

# architecture extra

Re: [PATCH 1/2] ARM: Require linker to support KEEP within OVERLAY for DCE

[Nathan Chancellor]

On Fri, Jul 04, 2025 at 02:15:18PM -0500, Rob Landley wrote:
> On 3/13/25 03:48, Linus Walleij wrote:
> > On Tue, Mar 11, 2025 at 8:43 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > 
> > > ld.lld prior to 21.0.0 does not support using the KEEP keyword within an
> > > overlay description, which may be needed to avoid discarding necessary
> > > sections within an overlay with '--gc-sections', which can be enabled
> > > for the kernel via CONFIG_LD_DEAD_CODE_DATA_ELIMINATION.
> > > 
> > > Disallow CONFIG_LD_DEAD_CODE_DATA_ELIMINATION without support for KEEP
> > > within OVERLAY and introduce a macro, OVERLAY_KEEP, that can be used to
> > > conditionally add KEEP when it is properly supported to avoid breaking
> > > old versions of ld.lld.
> 
> I bisected the 6.15 armv7l build break my mkroot project hit to this commit
> (e7607f7d6d81):
> 
>   LD      .tmp_vmlinux1
> Segmentation fault
> make[2]: *** [scripts/Makefile.vmlinux:77: vmlinux] Error 139
> make[1]: ***
> [/home/landley/toybox/clean/root/build/armv7l-tmp/linux/Makefile:1226:
> vmlinux] Error 2
> make: *** [Makefile:251: __sub-make] Error 2
> 
> The toolchain in question was built from gcc 11.4.0 and binutils 2.33.1
> which were the newest versions supported by
> https://github.com/richfelker/musl-cross-make when the still-current musl
> release (1.2.5) came out.
> 
> You can grab a binary toolchain to smoketest with from https://landley.net/bin/toolchains/latest/armv7l-linux-musleabihf-cross.tar.xz
> and build using the attached miniconfig ala:
> 
> for i in distclean allnoconfig ""
> do
>   CROSS_COMPILE=armv7l-linux-musleabihf- make ARCH=arm \
>     KCONFIG_ALLCONFIG=linux-miniconfig -j4 $i
> done
> 
> This _just_ seems to affect armv7l: armv5l and aarch64 still build fine.

Hmm, I do see a bug in that change, as it allows DCE to be turned on
with binutils older than 2.36, which should be avoided with something
like

diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 3072731fe09c..962451e54fdd 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -121,7 +121,7 @@ config ARM
 	select HAVE_KERNEL_XZ
 	select HAVE_KPROBES if !XIP_KERNEL && !CPU_ENDIAN_BE32 && !CPU_V7M
 	select HAVE_KRETPROBES if HAVE_KPROBES
-	select HAVE_LD_DEAD_CODE_DATA_ELIMINATION if (LD_VERSION >= 23600 || LD_CAN_USE_KEEP_IN_OVERLAY)
+	select HAVE_LD_DEAD_CODE_DATA_ELIMINATION if (LD_VERSION >= 23600 || LD_IS_LLD) && LD_CAN_USE_KEEP_IN_OVERLAY
 	select HAVE_MOD_ARCH_SPECIFIC
 	select HAVE_NMI
 	select HAVE_OPTPROBES if !THUMB2_KERNEL

but it does not seem like your test configuration enables
CONFIG_LD_DEAD_CODE_DATA_ELIMINATION so I would expect this change to be
a no-op in that case? Does the above change work for you? I won't be
free to validate this until after the weekend.

Cheers,
Nathan

Re: [PATCH 1/2] ARM: Require linker to support KEEP within OVERLAY for DCE

[Rob Landley]

On 7/4/25 19:24, Nathan Chancellor wrote:
> Hmm, I do see a bug in that change, as it allows DCE to be turned on
> with binutils older than 2.36, which should be avoided with something
> like
> 
> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
> index 3072731fe09c..962451e54fdd 100644
> --- a/arch/arm/Kconfig
> +++ b/arch/arm/Kconfig
> @@ -121,7 +121,7 @@ config ARM
>   	select HAVE_KERNEL_XZ
>   	select HAVE_KPROBES if !XIP_KERNEL && !CPU_ENDIAN_BE32 && !CPU_V7M
>   	select HAVE_KRETPROBES if HAVE_KPROBES
> -	select HAVE_LD_DEAD_CODE_DATA_ELIMINATION if (LD_VERSION >= 23600 || LD_CAN_USE_KEEP_IN_OVERLAY)
> +	select HAVE_LD_DEAD_CODE_DATA_ELIMINATION if (LD_VERSION >= 23600 || LD_IS_LLD) && LD_CAN_USE_KEEP_IN_OVERLAY
>   	select HAVE_MOD_ARCH_SPECIFIC
>   	select HAVE_NMI
>   	select HAVE_OPTPROBES if !THUMB2_KERNEL
> 
> but it does not seem like your test configuration enables
> CONFIG_LD_DEAD_CODE_DATA_ELIMINATION so I would expect this change to be
> a no-op in that case? Does the above change work for you? I won't be
> free to validate this until after the weekend.

That fixed it, thanks.

   AS      arch/arm/boot/compressed/piggy.o
   LD      arch/arm/boot/compressed/vmlinux
   OBJCOPY arch/arm/boot/zImage
   Kernel: arch/arm/boot/zImage is ready
$ qemu-system-arm -M virt -nographic -no-reboot -kernel $(find . -name 
zImage) -append console=ttyAMA0
Booting Linux on physical CPU 0x0
Linux version 6.15.0-dirty (landley@driftwood) 
(armv7l-linux-musleabihf-gcc (GCC) 11.4.0, GNU ld (GNU Binutils) 2.33.1) 
#1 Sat Jul  5 14:40:16 CDT 2025
CPU: ARMv7 Processor [414fc0f0] revision 0 (ARMv7), cr=30c53c7d
CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
OF: fdt: Machine model: linux,dummy-virt
...

I note it was a host tool segfaulting. It seems unlikely the host was 
running an arm binary, maybe something went off into la-la land trying 
to parse unexpected ELF tables?

Appropriately-tagged-by: Rob Landley <rob@landley.net>

> Cheers,
> Nathan

Thanks,

Rob