[RFT PATCH] arm64: add support for building the KASLR kernel with LLVM lld

[RFT PATCH] arm64: add support for building the KASLR kernel with LLVM lld

[Ard Biesheuvel]

Work around some differences in the behavior of ld.lld as compared
to lb.bfd:
- pass the -z notext and -z norelro options to convince the linker to
  permit text relocations, and relro sections that are non-adjacent
  (both of which are irrelevant for bare metal executables such as the
  kernel)
- move the definition of some __efistub_ decorated section markers to
  the linker script, which permits us to assign them as relative
  quantities (since using an intermediate assignment loses the section
  relative property when using ld.lld)
- handle .eh_frame and .gnu.hash sections to avoid them from being
  emitted between .head.text and .text, screwing up the section layout.

Cc: Nick Desaulniers <ndesaulniers@google.com>
Cc: Peter Smith <peter.smith@linaro.org>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---

With this patch and [0] applied on top of today's arm64/for-next/core [1],
I can create a working KASLR kernel using ld.lld (using LLD 6 from
Debian Buster)

[0] https://marc.info/?l=linux-arm-kernel&m=154366528003912
[1] https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git/log/?h=for-next/core

 arch/arm64/Makefile             |  2 +-
 arch/arm64/kernel/image.h       |  5 -----
 arch/arm64/kernel/vmlinux.lds.S | 10 ++++++++++
 3 files changed, 11 insertions(+), 6 deletions(-)

diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 7221494bcf60..8978f60779c4 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -18,7 +18,7 @@ ifeq ($(CONFIG_RELOCATABLE), y)
 # Pass --no-apply-dynamic-relocs to restore pre-binutils-2.27 behaviour
 # for relative relocs, since this leads to better Image compression
 # with the relocation offsets always being zero.
-LDFLAGS_vmlinux		+= -shared -Bsymbolic \
+LDFLAGS_vmlinux		+= -shared -Bsymbolic -z notext -z norelro \
 			$(call ld-option, --no-apply-dynamic-relocs)
 endif
 
diff --git a/arch/arm64/kernel/image.h b/arch/arm64/kernel/image.h
index 8da289dc843a..628c2fbf8939 100644
--- a/arch/arm64/kernel/image.h
+++ b/arch/arm64/kernel/image.h
@@ -73,8 +73,6 @@
 
 #ifdef CONFIG_EFI
 
-__efistub_stext_offset = stext - _text;
-
 /*
  * The EFI stub has its own symbol namespace prefixed by __efistub_, to
  * isolate it from the kernel proper. The following symbols are legally
@@ -102,9 +100,6 @@ __efistub___memmove		= __pi_memmove;
 __efistub___memset		= __pi_memset;
 #endif
 
-__efistub__text			= _text;
-__efistub__end			= _end;
-__efistub__edata		= _edata;
 __efistub_screen_info		= screen_info;
 
 #endif
diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
index 9ba4016090b1..cf3ad76f339f 100644
--- a/arch/arm64/kernel/vmlinux.lds.S
+++ b/arch/arm64/kernel/vmlinux.lds.S
@@ -98,12 +98,15 @@ SECTIONS
 		EXIT_CALL
 		*(.discard)
 		*(.discard.*)
+		*(.eh_frame)
 	}
 
 	. = KIMAGE_VADDR + TEXT_OFFSET;
 
 	.head.text : {
 		_text = .;
+		PROVIDE(__efistub__text = .);
+
 		HEAD_TEXT
 	}
 	.text : {			/* Real text segment		*/
@@ -206,6 +209,9 @@ SECTIONS
 	.hash : {
 		*(.hash)
 	}
+	.gnu.hash : {
+		*(.gnu.hash)
+	}
 	__rela_offset	= ABSOLUTE(ADDR(.rela.dyn) - KIMAGE_VADDR);
 	__rela_size	= SIZEOF(.rela.dyn);
 #endif
@@ -239,6 +245,7 @@ SECTIONS
 	PECOFF_EDATA_PADDING
 	__pecoff_data_rawsize = ABSOLUTE(. - __initdata_begin);
 	_edata = .;
+	PROVIDE(__efistub__edata = .);
 
 	BSS_SECTION(0, 0, 0)
 
@@ -249,12 +256,15 @@ SECTIONS
 
 	__pecoff_data_size = ABSOLUTE(. - __initdata_begin);
 	_end = .;
+	PROVIDE(__efistub__end = .);
 
 	STABS_DEBUG
 
 	HEAD_SYMBOLS
 }
 
+PROVIDE(__efistub_stext_offset = stext - _text);
+
 #ifdef CONFIG_RELOCATABLE
 VERSION {
 	/* mark all symbols as local so they are not listed in .dynsym */
-- 
2.19.2


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RFT PATCH] arm64: add support for building the KASLR kernel with LLVM lld

[Nick Desaulniers]

Ard, thanks again for this patch set!  I'm impressed with your skills
with linker scripts (and assembly); something I don't get to work with
day to day.

On Sat, Dec 1, 2018 at 5:57 AM Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
>
> Work around some differences in the behavior of ld.lld as compared
> to lb.bfd:

s/lb/ld/

> - pass the -z notext and -z norelro options to convince the linker to
>   permit text relocations, and relro sections that are non-adjacent
>   (both of which are irrelevant for bare metal executables such as the
>   kernel)

-z notext is also an implicit default for ld.bfd as per:
https://bugs.llvm.org/show_bug.cgi?id=39810#c7
(so this addition is no functional change for bfd, just helps lld)

> - move the definition of some __efistub_ decorated section markers to
>   the linker script, which permits us to assign them as relative
>   quantities (since using an intermediate assignment loses the section
>   relative property when using ld.lld)

I've filed https://bugs.llvm.org/show_bug.cgi?id=39857 to follow up on
this lld bug.

> - handle .eh_frame and .gnu.hash sections to avoid them from being
>   emitted between .head.text and .text, screwing up the section layout.
>
> Cc: Nick Desaulniers <ndesaulniers@google.com>
> Cc: Peter Smith <peter.smith@linaro.org>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> ---
>
> With this patch and [0] applied on top of today's arm64/for-next/core [1],
> I can create a working KASLR kernel using ld.lld (using LLD 6 from
> Debian Buster)

I applied the following 3 patches to mainline:
http://lists.infradead.org/pipermail/linux-arm-kernel/2018-November/616512.html:
[PATCH] arm64: drop linker script hack to hide efistub symbols
http://lists.infradead.org/pipermail/linux-arm-kernel/2018-December/616754.html:
[RFT PATCH] arm64: relocatable: build the kernel as a proper shared
library
http://lists.infradead.org/pipermail/linux-arm-kernel/2018-December/616765.html:
[RFT PATCH] arm64: add support for building the KASLR kernel with LLVM
lld

(last one is this patch)
and was able to compile, link, and QEMU boot:
1. ld.bfd CONFIG_RANDOMIZE_BASE=n
2. ld.bfd CONFIG_RANDOMIZE_BASE=y
3. ld.lld CONFIG_RANDOMIZE_BASE=n
4. ld.lld CONFIG_RANDOMIZE_BASE=y

Tested-by: Nick Desaulniers <ndesaulniers@google.com>

I'll do further testing on metal next week, but I'm quite confident
with the QEMU boot tests.
Thanks again, Ard!

>
> [0] https://marc.info/?l=linux-arm-kernel&m=154366528003912
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git/log/?h=for-next/core
>
>  arch/arm64/Makefile             |  2 +-
>  arch/arm64/kernel/image.h       |  5 -----
>  arch/arm64/kernel/vmlinux.lds.S | 10 ++++++++++
>  3 files changed, 11 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
> index 7221494bcf60..8978f60779c4 100644
> --- a/arch/arm64/Makefile
> +++ b/arch/arm64/Makefile
> @@ -18,7 +18,7 @@ ifeq ($(CONFIG_RELOCATABLE), y)
>  # Pass --no-apply-dynamic-relocs to restore pre-binutils-2.27 behaviour
>  # for relative relocs, since this leads to better Image compression
>  # with the relocation offsets always being zero.
> -LDFLAGS_vmlinux                += -shared -Bsymbolic \
> +LDFLAGS_vmlinux                += -shared -Bsymbolic -z notext -z norelro \
>                         $(call ld-option, --no-apply-dynamic-relocs)
>  endif
>
> diff --git a/arch/arm64/kernel/image.h b/arch/arm64/kernel/image.h
> index 8da289dc843a..628c2fbf8939 100644
> --- a/arch/arm64/kernel/image.h
> +++ b/arch/arm64/kernel/image.h
> @@ -73,8 +73,6 @@
>
>  #ifdef CONFIG_EFI
>
> -__efistub_stext_offset = stext - _text;
> -
>  /*
>   * The EFI stub has its own symbol namespace prefixed by __efistub_, to
>   * isolate it from the kernel proper. The following symbols are legally
> @@ -102,9 +100,6 @@ __efistub___memmove         = __pi_memmove;
>  __efistub___memset             = __pi_memset;
>  #endif
>
> -__efistub__text                        = _text;
> -__efistub__end                 = _end;
> -__efistub__edata               = _edata;
>  __efistub_screen_info          = screen_info;
>
>  #endif
> diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
> index 9ba4016090b1..cf3ad76f339f 100644
> --- a/arch/arm64/kernel/vmlinux.lds.S
> +++ b/arch/arm64/kernel/vmlinux.lds.S
> @@ -98,12 +98,15 @@ SECTIONS
>                 EXIT_CALL
>                 *(.discard)
>                 *(.discard.*)
> +               *(.eh_frame)
>         }
>
>         . = KIMAGE_VADDR + TEXT_OFFSET;
>
>         .head.text : {
>                 _text = .;
> +               PROVIDE(__efistub__text = .);
> +
>                 HEAD_TEXT
>         }
>         .text : {                       /* Real text segment            */
> @@ -206,6 +209,9 @@ SECTIONS
>         .hash : {
>                 *(.hash)
>         }
> +       .gnu.hash : {
> +               *(.gnu.hash)
> +       }
>         __rela_offset   = ABSOLUTE(ADDR(.rela.dyn) - KIMAGE_VADDR);
>         __rela_size     = SIZEOF(.rela.dyn);
>  #endif
> @@ -239,6 +245,7 @@ SECTIONS
>         PECOFF_EDATA_PADDING
>         __pecoff_data_rawsize = ABSOLUTE(. - __initdata_begin);
>         _edata = .;
> +       PROVIDE(__efistub__edata = .);
>
>         BSS_SECTION(0, 0, 0)
>
> @@ -249,12 +256,15 @@ SECTIONS
>
>         __pecoff_data_size = ABSOLUTE(. - __initdata_begin);
>         _end = .;
> +       PROVIDE(__efistub__end = .);
>
>         STABS_DEBUG
>
>         HEAD_SYMBOLS
>  }
>
> +PROVIDE(__efistub_stext_offset = stext - _text);
> +
>  #ifdef CONFIG_RELOCATABLE
>  VERSION {
>         /* mark all symbols as local so they are not listed in .dynsym */
> --
> 2.19.2
>


-- 
Thanks,
~Nick Desaulniers

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RFT PATCH] arm64: add support for building the KASLR kernel with LLVM lld

[Will Deacon]

On Sat, Dec 01, 2018 at 02:57:34PM +0100, Ard Biesheuvel wrote:
> Work around some differences in the behavior of ld.lld as compared
> to lb.bfd:
> - pass the -z notext and -z norelro options to convince the linker to
>   permit text relocations, and relro sections that are non-adjacent
>   (both of which are irrelevant for bare metal executables such as the
>   kernel)
> - move the definition of some __efistub_ decorated section markers to
>   the linker script, which permits us to assign them as relative
>   quantities (since using an intermediate assignment loses the section
>   relative property when using ld.lld)

I'd prefer not to work around a linker bug in today's lld just for KASLR
if it's anything more than passing a funny set of linker options.

Can you respin without this part, please?

Will

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RFT PATCH] arm64: add support for building the KASLR kernel with LLVM lld

[Nick Desaulniers]

On Mon, Dec 3, 2018 at 8:58 AM Will Deacon <will.deacon@arm.com> wrote:
>
> On Sat, Dec 01, 2018 at 02:57:34PM +0100, Ard Biesheuvel wrote:
> > Work around some differences in the behavior of ld.lld as compared
> > to lb.bfd:
> > - pass the -z notext and -z norelro options to convince the linker to
> >   permit text relocations, and relro sections that are non-adjacent
> >   (both of which are irrelevant for bare metal executables such as the
> >   kernel)
> > - move the definition of some __efistub_ decorated section markers to
> >   the linker script, which permits us to assign them as relative
> >   quantities (since using an intermediate assignment loses the section
> >   relative property when using ld.lld)
>
> I'd prefer not to work around a linker bug in today's lld just for KASLR
> if it's anything more than passing a funny set of linker options.

Filed https://bugs.llvm.org/show_bug.cgi?id=39857 to follow up. Peter
has narrowed down a straightforward reproducer.

-- 
Thanks,
~Nick Desaulniers

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RFT PATCH] arm64: add support for building the KASLR kernel with LLVM lld

[Will Deacon]

On Mon, Dec 03, 2018 at 10:33:06AM -0800, Nick Desaulniers wrote:
> On Mon, Dec 3, 2018 at 8:58 AM Will Deacon <will.deacon@arm.com> wrote:
> >
> > On Sat, Dec 01, 2018 at 02:57:34PM +0100, Ard Biesheuvel wrote:
> > > Work around some differences in the behavior of ld.lld as compared
> > > to lb.bfd:
> > > - pass the -z notext and -z norelro options to convince the linker to
> > >   permit text relocations, and relro sections that are non-adjacent
> > >   (both of which are irrelevant for bare metal executables such as the
> > >   kernel)
> > > - move the definition of some __efistub_ decorated section markers to
> > >   the linker script, which permits us to assign them as relative
> > >   quantities (since using an intermediate assignment loses the section
> > >   relative property when using ld.lld)
> >
> > I'd prefer not to work around a linker bug in today's lld just for KASLR
> > if it's anything more than passing a funny set of linker options.
> 
> Filed https://bugs.llvm.org/show_bug.cgi?id=39857 to follow up. Peter
> has narrowed down a straightforward reproducer.

Thank you.

Will

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel