* [PATCH v2 01/29] net/core: work around section mismatch warning for ptp_classifier
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-03 12:07 ` [PATCH v2 02/29] asm-generic: add .data.rel.ro sections to __ro_after_init Ard Biesheuvel
` (28 subsequent siblings)
29 siblings, 0 replies; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
The routine ptp_classifier_init() uses an initializer for an
automatic struct type variable which refers to an __initdata
symbol. This is perfectly legal, but may trigger a section
mismatch warning when running the compiler in -fpic mode, due
to the fact that the initializer may be emitted into an anonymous
.data section thats lack the __init annotation. So work around it
by using assignments instead.
Cc: "David S. Miller" <davem@davemloft.net>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
net/core/ptp_classifier.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/net/core/ptp_classifier.c b/net/core/ptp_classifier.c
index 703cf76aa7c2..7109c168b5e0 100644
--- a/net/core/ptp_classifier.c
+++ b/net/core/ptp_classifier.c
@@ -185,9 +185,10 @@ void __init ptp_classifier_init(void)
{ 0x16, 0, 0, 0x00000000 },
{ 0x06, 0, 0, 0x00000000 },
};
- struct sock_fprog_kern ptp_prog = {
- .len = ARRAY_SIZE(ptp_filter), .filter = ptp_filter,
- };
+ struct sock_fprog_kern ptp_prog;
+
+ ptp_prog.len = ARRAY_SIZE(ptp_filter);
+ ptp_prog.filter = ptp_filter;
BUG_ON(bpf_prog_create(&ptp_insns, &ptp_prog));
}
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 02/29] asm-generic: add .data.rel.ro sections to __ro_after_init
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
2017-09-03 12:07 ` [PATCH v2 01/29] net/core: work around section mismatch warning for ptp_classifier Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-04 15:59 ` Nicolas Pitre
2017-09-04 17:09 ` Kees Cook
2017-09-03 12:07 ` [PATCH v2 03/29] ARM: assembler: introduce adr_l, ldr_l and str_l macros Ard Biesheuvel
` (27 subsequent siblings)
29 siblings, 2 replies; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
When running in PIC mode, the compiler will emit const structures
containing runtime relocatable quantities into .data.rel.ro.* sections,
so that the linker can be smart about placing them together in a segment
that is read-write initially, and is remapped read-only afterwards. This
is exactly what __ro_after_init aims to provide, so move these sections
together.
Acked-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
include/asm-generic/vmlinux.lds.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index da0be9a8d1de..d16537b0b102 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -261,7 +261,7 @@
#ifndef RO_AFTER_INIT_DATA
#define RO_AFTER_INIT_DATA \
VMLINUX_SYMBOL(__start_ro_after_init) = .; \
- *(.data..ro_after_init) \
+ *(.data..ro_after_init .data.rel.ro.*) \
VMLINUX_SYMBOL(__end_ro_after_init) = .;
#endif
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 02/29] asm-generic: add .data.rel.ro sections to __ro_after_init
2017-09-03 12:07 ` [PATCH v2 02/29] asm-generic: add .data.rel.ro sections to __ro_after_init Ard Biesheuvel
@ 2017-09-04 15:59 ` Nicolas Pitre
2017-09-04 17:09 ` Kees Cook
1 sibling, 0 replies; 83+ messages in thread
From: Nicolas Pitre @ 2017-09-04 15:59 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
> When running in PIC mode, the compiler will emit const structures
> containing runtime relocatable quantities into .data.rel.ro.* sections,
> so that the linker can be smart about placing them together in a segment
> that is read-write initially, and is remapped read-only afterwards. This
> is exactly what __ro_after_init aims to provide, so move these sections
> together.
>
> Acked-by: Arnd Bergmann <arnd@arndb.de>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
> ---
> include/asm-generic/vmlinux.lds.h | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
> index da0be9a8d1de..d16537b0b102 100644
> --- a/include/asm-generic/vmlinux.lds.h
> +++ b/include/asm-generic/vmlinux.lds.h
> @@ -261,7 +261,7 @@
> #ifndef RO_AFTER_INIT_DATA
> #define RO_AFTER_INIT_DATA \
> VMLINUX_SYMBOL(__start_ro_after_init) = .; \
> - *(.data..ro_after_init) \
> + *(.data..ro_after_init .data.rel.ro.*) \
> VMLINUX_SYMBOL(__end_ro_after_init) = .;
> #endif
>
> --
> 2.11.0
>
>
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 02/29] asm-generic: add .data.rel.ro sections to __ro_after_init
2017-09-03 12:07 ` [PATCH v2 02/29] asm-generic: add .data.rel.ro sections to __ro_after_init Ard Biesheuvel
2017-09-04 15:59 ` Nicolas Pitre
@ 2017-09-04 17:09 ` Kees Cook
1 sibling, 0 replies; 83+ messages in thread
From: Kees Cook @ 2017-09-04 17:09 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, Sep 3, 2017 at 5:07 AM, Ard Biesheuvel
<ard.biesheuvel@linaro.org> wrote:
> When running in PIC mode, the compiler will emit const structures
> containing runtime relocatable quantities into .data.rel.ro.* sections,
> so that the linker can be smart about placing them together in a segment
> that is read-write initially, and is remapped read-only afterwards. This
> is exactly what __ro_after_init aims to provide, so move these sections
> together.
>
> Acked-by: Arnd Bergmann <arnd@arndb.de>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Kees Cook <keescook@chromium.org>
-Kees
> ---
> include/asm-generic/vmlinux.lds.h | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
> index da0be9a8d1de..d16537b0b102 100644
> --- a/include/asm-generic/vmlinux.lds.h
> +++ b/include/asm-generic/vmlinux.lds.h
> @@ -261,7 +261,7 @@
> #ifndef RO_AFTER_INIT_DATA
> #define RO_AFTER_INIT_DATA \
> VMLINUX_SYMBOL(__start_ro_after_init) = .; \
> - *(.data..ro_after_init) \
> + *(.data..ro_after_init .data.rel.ro.*) \
> VMLINUX_SYMBOL(__end_ro_after_init) = .;
> #endif
>
> --
> 2.11.0
>
--
Kees Cook
Pixel Security
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 03/29] ARM: assembler: introduce adr_l, ldr_l and str_l macros
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
2017-09-03 12:07 ` [PATCH v2 01/29] net/core: work around section mismatch warning for ptp_classifier Ard Biesheuvel
2017-09-03 12:07 ` [PATCH v2 02/29] asm-generic: add .data.rel.ro sections to __ro_after_init Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-04 16:05 ` Nicolas Pitre
2017-09-03 12:07 ` [PATCH v2 04/29] ARM: head-common.S: use PC-relative insn sequence for __proc_info Ard Biesheuvel
` (26 subsequent siblings)
29 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
Like arm64, ARM supports position independent code sequences that
produce symbol references with a greater reach than the ordinary
adr/ldr instructions.
Currently, we use open coded instruction sequences involving literals
and arithmetic operations. Instead, we can use movw/movt pairs on v7
CPUs, circumventing the D-cache entirely. For older CPUs, we can emit
the literal into a subsection, allowing it to be emitted out of line
while retaining the ability to perform arithmetic on label offsets.
E.g., on pre-v7 CPUs, we can emit a PC-relative reference as follows:
ldr <reg>, 222f
111: add <reg>, <reg>, pc
.subsection 1
222: .long <sym> - (111b + 8)
.previous
This is allowed by the assembler because, unlike ordinary sections,
subsections are combined into a single section into the object file,
and so the label references are not true cross-section references that
are visible as relocations. Note that we could even do something like
add <reg>, pc, #(222f - 111f) & ~0xfff
ldr <reg>, [<reg>, #(222f - 111f) & 0xfff]
111: add <reg>, <reg>, pc
.subsection 1
222: .long <sym> - (111b + 8)
.previous
if it turns out that the 4 KB range of the ldr instruction is insufficient
to reach the literal in the subsection, although this is currently not a
problem (of the 98 objects built from .S files in a multi_v7_defconfig
build, only 11 have .text sections that are over 1 KB, and the largest one
[entry-armv.o] is 3308 bytes)
Subsections have been available in binutils since 2004 at least, so
they should not cause any issues with older toolchains.
So use the above to implement the macros mov_l, adr_l, ldr_l and str_l,
all of which will use movw/movt pairs on v7 and later CPUs, and use
PC-relative literals otherwise.
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/include/asm/assembler.h | 76 ++++++++++++++++++++
1 file changed, 76 insertions(+)
diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h
index ad301f107dd2..341e4ed1ef84 100644
--- a/arch/arm/include/asm/assembler.h
+++ b/arch/arm/include/asm/assembler.h
@@ -518,4 +518,80 @@ THUMB( orr \reg , \reg , #PSR_T_BIT )
#endif
.endm
+ .macro __adldst_l, op, reg, sym, tmp
+ .if __LINUX_ARM_ARCH__ < 7
+ ldr \tmp, 111f
+ .subsection 1
+ .align 2
+111: .long \sym - (222f + 8)
+ .previous
+ .else
+ /*
+ * In Thumb-2 builds, the PC bias depends on whether we are currently
+ * emitting into a .arm or a .thumb section. So emit a nop and take
+ * its size, so we can infer the execution mode and PC bias from it.
+ */
+ ARM( .set .Lnopsize, 4 )
+ THUMB( .pushsection ".discard.nop", "x", %note )
+ THUMB( 111: nop )
+ THUMB( .set .Lnopsize, . - 111b )
+ THUMB( .popsection )
+
+ movw \tmp, #:lower16:\sym - (222f + 2 * .Lnopsize)
+ movt \tmp, #:upper16:\sym - (222f + 2 * .Lnopsize)
+ .endif
+222:
+ .ifc \op, add
+ add \reg, \tmp, pc
+ .elseif .Lnopsize == 2 @ Thumb-2 mode
+ add \tmp, \tmp, pc
+ \op \reg, [\tmp]
+ .else
+ \op \reg, [pc, \tmp]
+ .endif
+ .endm
+
+ /*
+ * mov_l - move a constant value or [relocated] address into a register
+ */
+ .macro mov_l, dst:req, imm:req
+ .if __LINUX_ARM_ARCH__ < 7
+ ldr \dst, =\imm
+ .else
+ movw \dst, #:lower16:\imm
+ movt \dst, #:upper16:\imm
+ .endif
+ .endm
+
+ /*
+ * adr_l - adr pseudo-op with unlimited range
+ *
+ * @dst: destination register
+ * @sym: name of the symbol
+ */
+ .macro adr_l, dst:req, sym:req
+ __adldst_l add, \dst, \sym, \dst
+ .endm
+
+ /*
+ * ldr_l - ldr <literal> pseudo-op with unlimited range
+ *
+ * @dst: destination register
+ * @sym: name of the symbol
+ */
+ .macro ldr_l, dst:req, sym:req
+ __adldst_l ldr, \dst, \sym, \dst
+ .endm
+
+ /*
+ * str_l - str <literal> pseudo-op with unlimited range
+ *
+ * @src: source register
+ * @sym: name of the symbol
+ * @tmp: mandatory scratch register
+ */
+ .macro str_l, src:req, sym:req, tmp:req
+ __adldst_l str, \src, \sym, \tmp
+ .endm
+
#endif /* __ASM_ASSEMBLER_H__ */
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 03/29] ARM: assembler: introduce adr_l, ldr_l and str_l macros
2017-09-03 12:07 ` [PATCH v2 03/29] ARM: assembler: introduce adr_l, ldr_l and str_l macros Ard Biesheuvel
@ 2017-09-04 16:05 ` Nicolas Pitre
0 siblings, 0 replies; 83+ messages in thread
From: Nicolas Pitre @ 2017-09-04 16:05 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
> Like arm64, ARM supports position independent code sequences that
> produce symbol references with a greater reach than the ordinary
> adr/ldr instructions.
>
> Currently, we use open coded instruction sequences involving literals
> and arithmetic operations. Instead, we can use movw/movt pairs on v7
> CPUs, circumventing the D-cache entirely. For older CPUs, we can emit
> the literal into a subsection, allowing it to be emitted out of line
> while retaining the ability to perform arithmetic on label offsets.
>
> E.g., on pre-v7 CPUs, we can emit a PC-relative reference as follows:
>
> ldr <reg>, 222f
> 111: add <reg>, <reg>, pc
> .subsection 1
> 222: .long <sym> - (111b + 8)
> .previous
>
> This is allowed by the assembler because, unlike ordinary sections,
> subsections are combined into a single section into the object file,
> and so the label references are not true cross-section references that
> are visible as relocations. Note that we could even do something like
>
> add <reg>, pc, #(222f - 111f) & ~0xfff
> ldr <reg>, [<reg>, #(222f - 111f) & 0xfff]
> 111: add <reg>, <reg>, pc
> .subsection 1
> 222: .long <sym> - (111b + 8)
> .previous
>
> if it turns out that the 4 KB range of the ldr instruction is insufficient
> to reach the literal in the subsection, although this is currently not a
> problem (of the 98 objects built from .S files in a multi_v7_defconfig
> build, only 11 have .text sections that are over 1 KB, and the largest one
> [entry-armv.o] is 3308 bytes)
>
> Subsections have been available in binutils since 2004 at least, so
> they should not cause any issues with older toolchains.
>
> So use the above to implement the macros mov_l, adr_l, ldr_l and str_l,
> all of which will use movw/movt pairs on v7 and later CPUs, and use
> PC-relative literals otherwise.
>
> Cc: Russell King <linux@armlinux.org.uk>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
REviewed-by: Nicolas Pitre <nico@linaro.org>
> ---
> arch/arm/include/asm/assembler.h | 76 ++++++++++++++++++++
> 1 file changed, 76 insertions(+)
>
> diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h
> index ad301f107dd2..341e4ed1ef84 100644
> --- a/arch/arm/include/asm/assembler.h
> +++ b/arch/arm/include/asm/assembler.h
> @@ -518,4 +518,80 @@ THUMB( orr \reg , \reg , #PSR_T_BIT )
> #endif
> .endm
>
> + .macro __adldst_l, op, reg, sym, tmp
> + .if __LINUX_ARM_ARCH__ < 7
> + ldr \tmp, 111f
> + .subsection 1
> + .align 2
> +111: .long \sym - (222f + 8)
> + .previous
> + .else
> + /*
> + * In Thumb-2 builds, the PC bias depends on whether we are currently
> + * emitting into a .arm or a .thumb section. So emit a nop and take
> + * its size, so we can infer the execution mode and PC bias from it.
> + */
> + ARM( .set .Lnopsize, 4 )
> + THUMB( .pushsection ".discard.nop", "x", %note )
> + THUMB( 111: nop )
> + THUMB( .set .Lnopsize, . - 111b )
> + THUMB( .popsection )
> +
> + movw \tmp, #:lower16:\sym - (222f + 2 * .Lnopsize)
> + movt \tmp, #:upper16:\sym - (222f + 2 * .Lnopsize)
> + .endif
> +222:
> + .ifc \op, add
> + add \reg, \tmp, pc
> + .elseif .Lnopsize == 2 @ Thumb-2 mode
> + add \tmp, \tmp, pc
> + \op \reg, [\tmp]
> + .else
> + \op \reg, [pc, \tmp]
> + .endif
> + .endm
> +
> + /*
> + * mov_l - move a constant value or [relocated] address into a register
> + */
> + .macro mov_l, dst:req, imm:req
> + .if __LINUX_ARM_ARCH__ < 7
> + ldr \dst, =\imm
> + .else
> + movw \dst, #:lower16:\imm
> + movt \dst, #:upper16:\imm
> + .endif
> + .endm
> +
> + /*
> + * adr_l - adr pseudo-op with unlimited range
> + *
> + * @dst: destination register
> + * @sym: name of the symbol
> + */
> + .macro adr_l, dst:req, sym:req
> + __adldst_l add, \dst, \sym, \dst
> + .endm
> +
> + /*
> + * ldr_l - ldr <literal> pseudo-op with unlimited range
> + *
> + * @dst: destination register
> + * @sym: name of the symbol
> + */
> + .macro ldr_l, dst:req, sym:req
> + __adldst_l ldr, \dst, \sym, \dst
> + .endm
> +
> + /*
> + * str_l - str <literal> pseudo-op with unlimited range
> + *
> + * @src: source register
> + * @sym: name of the symbol
> + * @tmp: mandatory scratch register
> + */
> + .macro str_l, src:req, sym:req, tmp:req
> + __adldst_l str, \src, \sym, \tmp
> + .endm
> +
> #endif /* __ASM_ASSEMBLER_H__ */
> --
> 2.11.0
>
>
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 04/29] ARM: head-common.S: use PC-relative insn sequence for __proc_info
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (2 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 03/29] ARM: assembler: introduce adr_l, ldr_l and str_l macros Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-04 16:06 ` Nicolas Pitre
2017-09-03 12:07 ` [PATCH v2 05/29] ARM: head-common.S: use PC-relative insn sequence for idmap creation Ard Biesheuvel
` (25 subsequent siblings)
29 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
Replace the open coded PC relative offset calculations with a pair
of adr_l invocations. This ensures these quantities are invariant
under runtime relocation.
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/kernel/head-common.S | 22 ++++++--------------
1 file changed, 6 insertions(+), 16 deletions(-)
diff --git a/arch/arm/kernel/head-common.S b/arch/arm/kernel/head-common.S
index 8733012d231f..06035488130c 100644
--- a/arch/arm/kernel/head-common.S
+++ b/arch/arm/kernel/head-common.S
@@ -150,11 +150,12 @@ ENDPROC(lookup_processor_type)
* r9 = cpuid (preserved)
*/
__lookup_processor_type:
- adr r3, __lookup_processor_type_data
- ldmia r3, {r4 - r6}
- sub r3, r3, r4 @ get offset between virt&phys
- add r5, r5, r3 @ convert virt addresses to
- add r6, r6, r3 @ physical address space
+ /*
+ * Look in <asm/procinfo.h> for information about the __proc_info
+ * structure.
+ */
+ adr_l r5, __proc_info_begin
+ adr_l r6, __proc_info_end
1: ldmia r5, {r3, r4} @ value, mask
and r4, r4, r9 @ mask wanted bits
teq r3, r4
@@ -166,17 +167,6 @@ __lookup_processor_type:
2: ret lr
ENDPROC(__lookup_processor_type)
-/*
- * Look in <asm/procinfo.h> for information about the __proc_info structure.
- */
- .align 2
- .type __lookup_processor_type_data, %object
-__lookup_processor_type_data:
- .long .
- .long __proc_info_begin
- .long __proc_info_end
- .size __lookup_processor_type_data, . - __lookup_processor_type_data
-
__error_lpae:
#ifdef CONFIG_DEBUG_LL
adr r0, str_lpae
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 04/29] ARM: head-common.S: use PC-relative insn sequence for __proc_info
2017-09-03 12:07 ` [PATCH v2 04/29] ARM: head-common.S: use PC-relative insn sequence for __proc_info Ard Biesheuvel
@ 2017-09-04 16:06 ` Nicolas Pitre
0 siblings, 0 replies; 83+ messages in thread
From: Nicolas Pitre @ 2017-09-04 16:06 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
> Replace the open coded PC relative offset calculations with a pair
> of adr_l invocations. This ensures these quantities are invariant
> under runtime relocation.
>
> Cc: Russell King <linux@armlinux.org.uk>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
> ---
> arch/arm/kernel/head-common.S | 22 ++++++--------------
> 1 file changed, 6 insertions(+), 16 deletions(-)
>
> diff --git a/arch/arm/kernel/head-common.S b/arch/arm/kernel/head-common.S
> index 8733012d231f..06035488130c 100644
> --- a/arch/arm/kernel/head-common.S
> +++ b/arch/arm/kernel/head-common.S
> @@ -150,11 +150,12 @@ ENDPROC(lookup_processor_type)
> * r9 = cpuid (preserved)
> */
> __lookup_processor_type:
> - adr r3, __lookup_processor_type_data
> - ldmia r3, {r4 - r6}
> - sub r3, r3, r4 @ get offset between virt&phys
> - add r5, r5, r3 @ convert virt addresses to
> - add r6, r6, r3 @ physical address space
> + /*
> + * Look in <asm/procinfo.h> for information about the __proc_info
> + * structure.
> + */
> + adr_l r5, __proc_info_begin
> + adr_l r6, __proc_info_end
> 1: ldmia r5, {r3, r4} @ value, mask
> and r4, r4, r9 @ mask wanted bits
> teq r3, r4
> @@ -166,17 +167,6 @@ __lookup_processor_type:
> 2: ret lr
> ENDPROC(__lookup_processor_type)
>
> -/*
> - * Look in <asm/procinfo.h> for information about the __proc_info structure.
> - */
> - .align 2
> - .type __lookup_processor_type_data, %object
> -__lookup_processor_type_data:
> - .long .
> - .long __proc_info_begin
> - .long __proc_info_end
> - .size __lookup_processor_type_data, . - __lookup_processor_type_data
> -
> __error_lpae:
> #ifdef CONFIG_DEBUG_LL
> adr r0, str_lpae
> --
> 2.11.0
>
>
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 05/29] ARM: head-common.S: use PC-relative insn sequence for idmap creation
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (3 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 04/29] ARM: head-common.S: use PC-relative insn sequence for __proc_info Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-04 16:08 ` Nicolas Pitre
2017-09-03 12:07 ` [PATCH v2 06/29] ARM: head.S: use PC-relative insn sequence for secondary_data Ard Biesheuvel
` (24 subsequent siblings)
29 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
Replace the open coded PC relative offset calculations involving
__turn_mmu_on and __turn_mmu_on_end with a pair of adr_l invocations.
This ensures these quantities are invariant under runtime relocation.
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/kernel/head.S | 12 ++----------
1 file changed, 2 insertions(+), 10 deletions(-)
diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
index 04286fd9e09c..0a98aec0e39d 100644
--- a/arch/arm/kernel/head.S
+++ b/arch/arm/kernel/head.S
@@ -227,11 +227,8 @@ __create_page_tables:
* Create identity mapping to cater for __enable_mmu.
* This identity mapping will be removed by paging_init().
*/
- adr r0, __turn_mmu_on_loc
- ldmia r0, {r3, r5, r6}
- sub r0, r0, r3 @ virt->phys offset
- add r5, r5, r0 @ phys __turn_mmu_on
- add r6, r6, r0 @ phys __turn_mmu_on_end
+ adr_l r5, __turn_mmu_on @ _pa(__turn_mmu_on)
+ adr_l r6, __turn_mmu_on_end @ _pa(__turn_mmu_on_end)
mov r5, r5, lsr #SECTION_SHIFT
mov r6, r6, lsr #SECTION_SHIFT
@@ -354,11 +351,6 @@ __create_page_tables:
ret lr
ENDPROC(__create_page_tables)
.ltorg
- .align
-__turn_mmu_on_loc:
- .long .
- .long __turn_mmu_on
- .long __turn_mmu_on_end
#if defined(CONFIG_SMP)
.text
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 05/29] ARM: head-common.S: use PC-relative insn sequence for idmap creation
2017-09-03 12:07 ` [PATCH v2 05/29] ARM: head-common.S: use PC-relative insn sequence for idmap creation Ard Biesheuvel
@ 2017-09-04 16:08 ` Nicolas Pitre
0 siblings, 0 replies; 83+ messages in thread
From: Nicolas Pitre @ 2017-09-04 16:08 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
> Replace the open coded PC relative offset calculations involving
> __turn_mmu_on and __turn_mmu_on_end with a pair of adr_l invocations.
> This ensures these quantities are invariant under runtime relocation.
>
> Cc: Russell King <linux@armlinux.org.uk>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
> ---
> arch/arm/kernel/head.S | 12 ++----------
> 1 file changed, 2 insertions(+), 10 deletions(-)
>
> diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
> index 04286fd9e09c..0a98aec0e39d 100644
> --- a/arch/arm/kernel/head.S
> +++ b/arch/arm/kernel/head.S
> @@ -227,11 +227,8 @@ __create_page_tables:
> * Create identity mapping to cater for __enable_mmu.
> * This identity mapping will be removed by paging_init().
> */
> - adr r0, __turn_mmu_on_loc
> - ldmia r0, {r3, r5, r6}
> - sub r0, r0, r3 @ virt->phys offset
> - add r5, r5, r0 @ phys __turn_mmu_on
> - add r6, r6, r0 @ phys __turn_mmu_on_end
> + adr_l r5, __turn_mmu_on @ _pa(__turn_mmu_on)
> + adr_l r6, __turn_mmu_on_end @ _pa(__turn_mmu_on_end)
> mov r5, r5, lsr #SECTION_SHIFT
> mov r6, r6, lsr #SECTION_SHIFT
>
> @@ -354,11 +351,6 @@ __create_page_tables:
> ret lr
> ENDPROC(__create_page_tables)
> .ltorg
> - .align
> -__turn_mmu_on_loc:
> - .long .
> - .long __turn_mmu_on
> - .long __turn_mmu_on_end
>
> #if defined(CONFIG_SMP)
> .text
> --
> 2.11.0
>
>
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 06/29] ARM: head.S: use PC-relative insn sequence for secondary_data
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (4 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 05/29] ARM: head-common.S: use PC-relative insn sequence for idmap creation Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-04 16:09 ` Nicolas Pitre
2017-09-03 12:07 ` [PATCH v2 07/29] ARM: kernel: use relative references for UP/SMP alternatives Ard Biesheuvel
` (23 subsequent siblings)
29 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
Replace the open coded PC relative offset calculations with adr_l
and ldr_l invocations. This ensures these quantities are invariant
under runtime relocation.
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/kernel/head.S | 19 ++++---------------
1 file changed, 4 insertions(+), 15 deletions(-)
diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
index 0a98aec0e39d..6e9df3663a57 100644
--- a/arch/arm/kernel/head.S
+++ b/arch/arm/kernel/head.S
@@ -386,10 +386,8 @@ ENTRY(secondary_startup)
/*
* Use the page tables supplied from __cpu_up.
*/
- adr r4, __secondary_data
- ldmia r4, {r5, r7, r12} @ address to jump to after
- sub lr, r4, r5 @ mmu has been enabled
- add r3, r7, lr
+ adr_l r3, secondary_data
+ mov_l r12, __secondary_switched
ldrd r4, [r3, #0] @ get secondary_data.pgdir
ARM_BE8(eor r4, r4, r5) @ Swap r5 and r4 in BE:
ARM_BE8(eor r5, r4, r5) @ it can be done in 3 steps
@@ -404,22 +402,13 @@ ARM_BE8(eor r4, r4, r5) @ without using a temp reg.
ENDPROC(secondary_startup)
ENDPROC(secondary_startup_arm)
- /*
- * r6 = &secondary_data
- */
ENTRY(__secondary_switched)
- ldr sp, [r7, #12] @ get secondary_data.stack
+ ldr_l r7, secondary_data + 12 @ get secondary_data.stack
+ mov sp, r7
mov fp, #0
b secondary_start_kernel
ENDPROC(__secondary_switched)
- .align
-
- .type __secondary_data, %object
-__secondary_data:
- .long .
- .long secondary_data
- .long __secondary_switched
#endif /* defined(CONFIG_SMP) */
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 06/29] ARM: head.S: use PC-relative insn sequence for secondary_data
2017-09-03 12:07 ` [PATCH v2 06/29] ARM: head.S: use PC-relative insn sequence for secondary_data Ard Biesheuvel
@ 2017-09-04 16:09 ` Nicolas Pitre
0 siblings, 0 replies; 83+ messages in thread
From: Nicolas Pitre @ 2017-09-04 16:09 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
> Replace the open coded PC relative offset calculations with adr_l
> and ldr_l invocations. This ensures these quantities are invariant
> under runtime relocation.
>
> Cc: Russell King <linux@armlinux.org.uk>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
> ---
> arch/arm/kernel/head.S | 19 ++++---------------
> 1 file changed, 4 insertions(+), 15 deletions(-)
>
> diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
> index 0a98aec0e39d..6e9df3663a57 100644
> --- a/arch/arm/kernel/head.S
> +++ b/arch/arm/kernel/head.S
> @@ -386,10 +386,8 @@ ENTRY(secondary_startup)
> /*
> * Use the page tables supplied from __cpu_up.
> */
> - adr r4, __secondary_data
> - ldmia r4, {r5, r7, r12} @ address to jump to after
> - sub lr, r4, r5 @ mmu has been enabled
> - add r3, r7, lr
> + adr_l r3, secondary_data
> + mov_l r12, __secondary_switched
> ldrd r4, [r3, #0] @ get secondary_data.pgdir
> ARM_BE8(eor r4, r4, r5) @ Swap r5 and r4 in BE:
> ARM_BE8(eor r5, r4, r5) @ it can be done in 3 steps
> @@ -404,22 +402,13 @@ ARM_BE8(eor r4, r4, r5) @ without using a temp reg.
> ENDPROC(secondary_startup)
> ENDPROC(secondary_startup_arm)
>
> - /*
> - * r6 = &secondary_data
> - */
> ENTRY(__secondary_switched)
> - ldr sp, [r7, #12] @ get secondary_data.stack
> + ldr_l r7, secondary_data + 12 @ get secondary_data.stack
> + mov sp, r7
> mov fp, #0
> b secondary_start_kernel
> ENDPROC(__secondary_switched)
>
> - .align
> -
> - .type __secondary_data, %object
> -__secondary_data:
> - .long .
> - .long secondary_data
> - .long __secondary_switched
> #endif /* defined(CONFIG_SMP) */
>
>
> --
> 2.11.0
>
>
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 07/29] ARM: kernel: use relative references for UP/SMP alternatives
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (5 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 06/29] ARM: head.S: use PC-relative insn sequence for secondary_data Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-04 16:15 ` Nicolas Pitre
2017-09-03 12:07 ` [PATCH v2 08/29] ARM: head: use PC-relative insn sequence for __smp_alt Ard Biesheuvel
` (22 subsequent siblings)
29 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
To avoid absolute references that are subject to runtime relocation
when running a kernel built with CONFIG_RELOCATABLE=y, use relative
references in the smp.alt entries.
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/include/asm/assembler.h | 4 ++--
arch/arm/include/asm/processor.h | 2 +-
arch/arm/kernel/head.S | 10 +++++-----
3 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h
index 341e4ed1ef84..a0906c1fc65d 100644
--- a/arch/arm/include/asm/assembler.h
+++ b/arch/arm/include/asm/assembler.h
@@ -260,7 +260,7 @@
*/
#define ALT_UP(instr...) \
.pushsection ".alt.smp.init", "a" ;\
- .long 9998b ;\
+ .long 9998b - . ;\
9997: instr ;\
.if . - 9997b == 2 ;\
nop ;\
@@ -272,7 +272,7 @@
#define ALT_UP_B(label) \
.equ up_b_offset, label - 9998b ;\
.pushsection ".alt.smp.init", "a" ;\
- .long 9998b ;\
+ .long 9998b - . ;\
W(b) . + up_b_offset ;\
.popsection
#else
diff --git a/arch/arm/include/asm/processor.h b/arch/arm/include/asm/processor.h
index c3d5fc124a05..3667b395bb17 100644
--- a/arch/arm/include/asm/processor.h
+++ b/arch/arm/include/asm/processor.h
@@ -92,7 +92,7 @@ unsigned long get_wchan(struct task_struct *p);
#define __ALT_SMP_ASM(smp, up) \
"9998: " smp "\n" \
" .pushsection \".alt.smp.init\", \"a\"\n" \
- " .long 9998b\n" \
+ " .long 9998b - .\n" \
" " up "\n" \
" .popsection\n"
#else
diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
index 6e9df3663a57..ec22f42fd8bb 100644
--- a/arch/arm/kernel/head.S
+++ b/arch/arm/kernel/head.S
@@ -548,14 +548,15 @@ smp_on_up:
__do_fixup_smp_on_up:
cmp r4, r5
reths lr
- ldmia r4!, {r0, r6}
- ARM( str r6, [r0, r3] )
- THUMB( add r0, r0, r3 )
+ ldmia r4, {r0, r6}
+ ARM( str r6, [r0, r4] )
+ THUMB( add r0, r0, r4 )
+ add r4, r4, #8
#ifdef __ARMEB__
THUMB( mov r6, r6, ror #16 ) @ Convert word order for big-endian.
#endif
THUMB( strh r6, [r0], #2 ) @ For Thumb-2, store as two halfwords
- THUMB( mov r6, r6, lsr #16 ) @ to be robust against misaligned r3.
+ THUMB( mov r6, r6, lsr #16 ) @ to be robust against misaligned r0.
THUMB( strh r6, [r0] )
b __do_fixup_smp_on_up
ENDPROC(__do_fixup_smp_on_up)
@@ -564,7 +565,6 @@ ENTRY(fixup_smp)
stmfd sp!, {r4 - r6, lr}
mov r4, r0
add r5, r0, r1
- mov r3, #0
bl __do_fixup_smp_on_up
ldmfd sp!, {r4 - r6, pc}
ENDPROC(fixup_smp)
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 07/29] ARM: kernel: use relative references for UP/SMP alternatives
2017-09-03 12:07 ` [PATCH v2 07/29] ARM: kernel: use relative references for UP/SMP alternatives Ard Biesheuvel
@ 2017-09-04 16:15 ` Nicolas Pitre
0 siblings, 0 replies; 83+ messages in thread
From: Nicolas Pitre @ 2017-09-04 16:15 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
> To avoid absolute references that are subject to runtime relocation
> when running a kernel built with CONFIG_RELOCATABLE=y, use relative
> references in the smp.alt entries.
>
> Cc: Russell King <linux@armlinux.org.uk>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
> ---
> arch/arm/include/asm/assembler.h | 4 ++--
> arch/arm/include/asm/processor.h | 2 +-
> arch/arm/kernel/head.S | 10 +++++-----
> 3 files changed, 8 insertions(+), 8 deletions(-)
>
> diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h
> index 341e4ed1ef84..a0906c1fc65d 100644
> --- a/arch/arm/include/asm/assembler.h
> +++ b/arch/arm/include/asm/assembler.h
> @@ -260,7 +260,7 @@
> */
> #define ALT_UP(instr...) \
> .pushsection ".alt.smp.init", "a" ;\
> - .long 9998b ;\
> + .long 9998b - . ;\
> 9997: instr ;\
> .if . - 9997b == 2 ;\
> nop ;\
> @@ -272,7 +272,7 @@
> #define ALT_UP_B(label) \
> .equ up_b_offset, label - 9998b ;\
> .pushsection ".alt.smp.init", "a" ;\
> - .long 9998b ;\
> + .long 9998b - . ;\
> W(b) . + up_b_offset ;\
> .popsection
> #else
> diff --git a/arch/arm/include/asm/processor.h b/arch/arm/include/asm/processor.h
> index c3d5fc124a05..3667b395bb17 100644
> --- a/arch/arm/include/asm/processor.h
> +++ b/arch/arm/include/asm/processor.h
> @@ -92,7 +92,7 @@ unsigned long get_wchan(struct task_struct *p);
> #define __ALT_SMP_ASM(smp, up) \
> "9998: " smp "\n" \
> " .pushsection \".alt.smp.init\", \"a\"\n" \
> - " .long 9998b\n" \
> + " .long 9998b - .\n" \
> " " up "\n" \
> " .popsection\n"
> #else
> diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
> index 6e9df3663a57..ec22f42fd8bb 100644
> --- a/arch/arm/kernel/head.S
> +++ b/arch/arm/kernel/head.S
> @@ -548,14 +548,15 @@ smp_on_up:
> __do_fixup_smp_on_up:
> cmp r4, r5
> reths lr
> - ldmia r4!, {r0, r6}
> - ARM( str r6, [r0, r3] )
> - THUMB( add r0, r0, r3 )
> + ldmia r4, {r0, r6}
> + ARM( str r6, [r0, r4] )
> + THUMB( add r0, r0, r4 )
> + add r4, r4, #8
> #ifdef __ARMEB__
> THUMB( mov r6, r6, ror #16 ) @ Convert word order for big-endian.
> #endif
> THUMB( strh r6, [r0], #2 ) @ For Thumb-2, store as two halfwords
> - THUMB( mov r6, r6, lsr #16 ) @ to be robust against misaligned r3.
> + THUMB( mov r6, r6, lsr #16 ) @ to be robust against misaligned r0.
> THUMB( strh r6, [r0] )
> b __do_fixup_smp_on_up
> ENDPROC(__do_fixup_smp_on_up)
> @@ -564,7 +565,6 @@ ENTRY(fixup_smp)
> stmfd sp!, {r4 - r6, lr}
> mov r4, r0
> add r5, r0, r1
> - mov r3, #0
> bl __do_fixup_smp_on_up
> ldmfd sp!, {r4 - r6, pc}
> ENDPROC(fixup_smp)
> --
> 2.11.0
>
>
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 08/29] ARM: head: use PC-relative insn sequence for __smp_alt
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (6 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 07/29] ARM: kernel: use relative references for UP/SMP alternatives Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-04 16:19 ` Nicolas Pitre
2017-09-03 12:07 ` [PATCH v2 09/29] ARM: sleep.S: use PC-relative insn sequence for sleep_save_sp/mpidr_hash Ard Biesheuvel
` (21 subsequent siblings)
29 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
Replace the open coded PC relative offset calculations with a pair
of adr_l invocations. This ensures these quantities are invariant
under runtime relocation.
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/kernel/head.S | 12 ++----------
1 file changed, 2 insertions(+), 10 deletions(-)
diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
index ec22f42fd8bb..db6b823f20a4 100644
--- a/arch/arm/kernel/head.S
+++ b/arch/arm/kernel/head.S
@@ -523,19 +523,11 @@ ARM_BE8(rev r0, r0) @ byteswap if big endian
retne lr
__fixup_smp_on_up:
- adr r0, 1f
- ldmia r0, {r3 - r5}
- sub r3, r0, r3
- add r4, r4, r3
- add r5, r5, r3
+ adr_l r4, __smpalt_begin
+ adr_l r5, __smpalt_end
b __do_fixup_smp_on_up
ENDPROC(__fixup_smp)
- .align
-1: .word .
- .word __smpalt_begin
- .word __smpalt_end
-
.pushsection .data
.globl smp_on_up
smp_on_up:
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 08/29] ARM: head: use PC-relative insn sequence for __smp_alt
2017-09-03 12:07 ` [PATCH v2 08/29] ARM: head: use PC-relative insn sequence for __smp_alt Ard Biesheuvel
@ 2017-09-04 16:19 ` Nicolas Pitre
2017-09-04 16:20 ` Ard Biesheuvel
0 siblings, 1 reply; 83+ messages in thread
From: Nicolas Pitre @ 2017-09-04 16:19 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
> Replace the open coded PC relative offset calculations with a pair
> of adr_l invocations. This ensures these quantities are invariant
> under runtime relocation.
You probably should add that this patch must be applied on top of "ARM:
kernel: use relative references for UP/SMP alternatives" because of the
expected r3 value prior to that patch.
Acked-by: Nicolas Pitre <nico@linaro.org>
> Cc: Russell King <linux@armlinux.org.uk>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> ---
> arch/arm/kernel/head.S | 12 ++----------
> 1 file changed, 2 insertions(+), 10 deletions(-)
>
> diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
> index ec22f42fd8bb..db6b823f20a4 100644
> --- a/arch/arm/kernel/head.S
> +++ b/arch/arm/kernel/head.S
> @@ -523,19 +523,11 @@ ARM_BE8(rev r0, r0) @ byteswap if big endian
> retne lr
>
> __fixup_smp_on_up:
> - adr r0, 1f
> - ldmia r0, {r3 - r5}
> - sub r3, r0, r3
> - add r4, r4, r3
> - add r5, r5, r3
> + adr_l r4, __smpalt_begin
> + adr_l r5, __smpalt_end
> b __do_fixup_smp_on_up
> ENDPROC(__fixup_smp)
>
> - .align
> -1: .word .
> - .word __smpalt_begin
> - .word __smpalt_end
> -
> .pushsection .data
> .globl smp_on_up
> smp_on_up:
> --
> 2.11.0
>
>
^ permalink raw reply [flat|nested] 83+ messages in thread* [PATCH v2 08/29] ARM: head: use PC-relative insn sequence for __smp_alt
2017-09-04 16:19 ` Nicolas Pitre
@ 2017-09-04 16:20 ` Ard Biesheuvel
0 siblings, 0 replies; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-04 16:20 UTC (permalink / raw)
To: linux-arm-kernel
On 4 September 2017 at 17:19, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
>
>> Replace the open coded PC relative offset calculations with a pair
>> of adr_l invocations. This ensures these quantities are invariant
>> under runtime relocation.
>
> You probably should add that this patch must be applied on top of "ARM:
> kernel: use relative references for UP/SMP alternatives" because of the
> expected r3 value prior to that patch.
>
> Acked-by: Nicolas Pitre <nico@linaro.org>
>
Yes, that is why I needed to reorder them. I will add a mention to the
commit log.
Thanks,
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 09/29] ARM: sleep.S: use PC-relative insn sequence for sleep_save_sp/mpidr_hash
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (7 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 08/29] ARM: head: use PC-relative insn sequence for __smp_alt Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-04 16:20 ` Nicolas Pitre
2017-09-03 12:07 ` [PATCH v2 10/29] ARM: head.S: use PC-relative insn sequences for __fixup_pv_table Ard Biesheuvel
` (20 subsequent siblings)
29 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
Replace the open coded PC relative offset calculations with adr_l and
ldr_l invocations. Note that ALT_SMP() expects a single instruction
so move the macro invocation after it.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/kernel/sleep.S | 19 +++++--------------
1 file changed, 5 insertions(+), 14 deletions(-)
diff --git a/arch/arm/kernel/sleep.S b/arch/arm/kernel/sleep.S
index 0f6c1000582c..f4920b5d0fc4 100644
--- a/arch/arm/kernel/sleep.S
+++ b/arch/arm/kernel/sleep.S
@@ -71,8 +71,9 @@ ENTRY(__cpu_suspend)
ldr r3, =sleep_save_sp
stmfd sp!, {r0, r1} @ save suspend func arg and pointer
ldr r3, [r3, #SLEEP_SAVE_SP_VIRT]
- ALT_SMP(ldr r0, =mpidr_hash)
+ ALT_SMP(W(nop)) @ don't use adr_l inside ALT_SMP()
ALT_UP_B(1f)
+ adr_l r0, mpidr_hash
/* This ldmia relies on the memory layout of the mpidr_hash struct */
ldmia r0, {r1, r6-r8} @ r1 = mpidr mask (r6,r7,r8) = l[0,1,2] shifts
compute_mpidr_hash r0, r6, r7, r8, r2, r1
@@ -137,9 +138,8 @@ ARM_BE8(setend be) @ ensure we are in BE mode
mov r1, #0
ALT_SMP(mrc p15, 0, r0, c0, c0, 5)
ALT_UP_B(1f)
- adr r2, mpidr_hash_ptr
- ldr r3, [r2]
- add r2, r2, r3 @ r2 = struct mpidr_hash phys address
+ adr_l r2, mpidr_hash @ r2 = struct mpidr_hash phys address
+
/*
* This ldmia relies on the memory layout of the mpidr_hash
* struct mpidr_hash.
@@ -147,10 +147,7 @@ ARM_BE8(setend be) @ ensure we are in BE mode
ldmia r2, { r3-r6 } @ r3 = mpidr mask (r4,r5,r6) = l[0,1,2] shifts
compute_mpidr_hash r1, r4, r5, r6, r0, r3
1:
- adr r0, _sleep_save_sp
- ldr r2, [r0]
- add r0, r0, r2
- ldr r0, [r0, #SLEEP_SAVE_SP_PHYS]
+ ldr_l r0, sleep_save_sp + SLEEP_SAVE_SP_PHYS
ldr r0, [r0, r1, lsl #2]
@ load phys pgd, stack, resume fn
@@ -164,12 +161,6 @@ ENDPROC(cpu_resume)
ENDPROC(cpu_resume_arm)
#endif
- .align 2
-_sleep_save_sp:
- .long sleep_save_sp - .
-mpidr_hash_ptr:
- .long mpidr_hash - . @ mpidr_hash struct offset
-
.data
.type sleep_save_sp, #object
ENTRY(sleep_save_sp)
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 09/29] ARM: sleep.S: use PC-relative insn sequence for sleep_save_sp/mpidr_hash
2017-09-03 12:07 ` [PATCH v2 09/29] ARM: sleep.S: use PC-relative insn sequence for sleep_save_sp/mpidr_hash Ard Biesheuvel
@ 2017-09-04 16:20 ` Nicolas Pitre
0 siblings, 0 replies; 83+ messages in thread
From: Nicolas Pitre @ 2017-09-04 16:20 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
> Replace the open coded PC relative offset calculations with adr_l and
> ldr_l invocations. Note that ALT_SMP() expects a single instruction
> so move the macro invocation after it.
>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
> ---
> arch/arm/kernel/sleep.S | 19 +++++--------------
> 1 file changed, 5 insertions(+), 14 deletions(-)
>
> diff --git a/arch/arm/kernel/sleep.S b/arch/arm/kernel/sleep.S
> index 0f6c1000582c..f4920b5d0fc4 100644
> --- a/arch/arm/kernel/sleep.S
> +++ b/arch/arm/kernel/sleep.S
> @@ -71,8 +71,9 @@ ENTRY(__cpu_suspend)
> ldr r3, =sleep_save_sp
> stmfd sp!, {r0, r1} @ save suspend func arg and pointer
> ldr r3, [r3, #SLEEP_SAVE_SP_VIRT]
> - ALT_SMP(ldr r0, =mpidr_hash)
> + ALT_SMP(W(nop)) @ don't use adr_l inside ALT_SMP()
> ALT_UP_B(1f)
> + adr_l r0, mpidr_hash
> /* This ldmia relies on the memory layout of the mpidr_hash struct */
> ldmia r0, {r1, r6-r8} @ r1 = mpidr mask (r6,r7,r8) = l[0,1,2] shifts
> compute_mpidr_hash r0, r6, r7, r8, r2, r1
> @@ -137,9 +138,8 @@ ARM_BE8(setend be) @ ensure we are in BE mode
> mov r1, #0
> ALT_SMP(mrc p15, 0, r0, c0, c0, 5)
> ALT_UP_B(1f)
> - adr r2, mpidr_hash_ptr
> - ldr r3, [r2]
> - add r2, r2, r3 @ r2 = struct mpidr_hash phys address
> + adr_l r2, mpidr_hash @ r2 = struct mpidr_hash phys address
> +
> /*
> * This ldmia relies on the memory layout of the mpidr_hash
> * struct mpidr_hash.
> @@ -147,10 +147,7 @@ ARM_BE8(setend be) @ ensure we are in BE mode
> ldmia r2, { r3-r6 } @ r3 = mpidr mask (r4,r5,r6) = l[0,1,2] shifts
> compute_mpidr_hash r1, r4, r5, r6, r0, r3
> 1:
> - adr r0, _sleep_save_sp
> - ldr r2, [r0]
> - add r0, r0, r2
> - ldr r0, [r0, #SLEEP_SAVE_SP_PHYS]
> + ldr_l r0, sleep_save_sp + SLEEP_SAVE_SP_PHYS
> ldr r0, [r0, r1, lsl #2]
>
> @ load phys pgd, stack, resume fn
> @@ -164,12 +161,6 @@ ENDPROC(cpu_resume)
> ENDPROC(cpu_resume_arm)
> #endif
>
> - .align 2
> -_sleep_save_sp:
> - .long sleep_save_sp - .
> -mpidr_hash_ptr:
> - .long mpidr_hash - . @ mpidr_hash struct offset
> -
> .data
> .type sleep_save_sp, #object
> ENTRY(sleep_save_sp)
> --
> 2.11.0
>
>
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 10/29] ARM: head.S: use PC-relative insn sequences for __fixup_pv_table
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (8 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 09/29] ARM: sleep.S: use PC-relative insn sequence for sleep_save_sp/mpidr_hash Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-04 16:47 ` Nicolas Pitre
2017-09-03 12:07 ` [PATCH v2 11/29] ARM: head.S: use PC relative insn sequence to calculate PHYS_OFFSET Ard Biesheuvel
` (19 subsequent siblings)
29 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
Replace the open coded PC relative offset calculations with adr_l
and mov_l invocations. This ensures these quantities are invariant
under runtime relocation.
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/kernel/head.S | 27 ++++++--------------
1 file changed, 8 insertions(+), 19 deletions(-)
diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
index db6b823f20a4..f607e290ef4b 100644
--- a/arch/arm/kernel/head.S
+++ b/arch/arm/kernel/head.S
@@ -578,14 +578,11 @@ ENDPROC(fixup_smp)
*/
__HEAD
__fixup_pv_table:
- adr r0, 1f
- ldmia r0, {r3-r7}
+ adr_l r6, __pv_phys_pfn_offset
+ adr_l r7, __pv_offset @ __pa(__pv_offset)
+ mov_l r3, __pv_offset @ __va(__pv_offset)
mvn ip, #0
- subs r3, r0, r3 @ PHYS_OFFSET - PAGE_OFFSET
- add r4, r4, r3 @ adjust table start address
- add r5, r5, r3 @ adjust table end address
- add r6, r6, r3 @ adjust __pv_phys_pfn_offset address
- add r7, r7, r3 @ adjust __pv_offset address
+ subs r3, r7, r3 @ PHYS_OFFSET - PAGE_OFFSET
mov r0, r8, lsr #PAGE_SHIFT @ convert to PFN
str r0, [r6] @ save computed PHYS_OFFSET to __pv_phys_pfn_offset
strcc ip, [r7, #HIGH_OFFSET] @ save to __pv_offset high bits
@@ -594,20 +591,15 @@ __fixup_pv_table:
THUMB( it ne @ cross section branch )
bne __error
str r3, [r7, #LOW_OFFSET] @ save to __pv_offset low bits
+ adr_l r4, __pv_table_begin
+ adr_l r5, __pv_table_end
b __fixup_a_pv_table
ENDPROC(__fixup_pv_table)
-
- .align
-1: .long .
- .long __pv_table_begin
- .long __pv_table_end
-2: .long __pv_phys_pfn_offset
- .long __pv_offset
+ .ltorg
.text
__fixup_a_pv_table:
- adr r0, 3f
- ldr r6, [r0]
+ mov_l r6, __pv_offset
add r6, r6, r3
ldr r0, [r6, #HIGH_OFFSET] @ pv_offset high word
ldr r6, [r6, #LOW_OFFSET] @ pv_offset low word
@@ -676,9 +668,6 @@ ARM_BE8(rev16 ip, ip)
#endif
ENDPROC(__fixup_a_pv_table)
- .align
-3: .long __pv_offset
-
ENTRY(fixup_pv_table)
stmfd sp!, {r4 - r7, lr}
mov r3, #0 @ no offset
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 10/29] ARM: head.S: use PC-relative insn sequences for __fixup_pv_table
2017-09-03 12:07 ` [PATCH v2 10/29] ARM: head.S: use PC-relative insn sequences for __fixup_pv_table Ard Biesheuvel
@ 2017-09-04 16:47 ` Nicolas Pitre
0 siblings, 0 replies; 83+ messages in thread
From: Nicolas Pitre @ 2017-09-04 16:47 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
> Replace the open coded PC relative offset calculations with adr_l
> and mov_l invocations. This ensures these quantities are invariant
> under runtime relocation.
>
> Cc: Russell King <linux@armlinux.org.uk>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
> ---
> arch/arm/kernel/head.S | 27 ++++++--------------
> 1 file changed, 8 insertions(+), 19 deletions(-)
>
> diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
> index db6b823f20a4..f607e290ef4b 100644
> --- a/arch/arm/kernel/head.S
> +++ b/arch/arm/kernel/head.S
> @@ -578,14 +578,11 @@ ENDPROC(fixup_smp)
> */
> __HEAD
> __fixup_pv_table:
> - adr r0, 1f
> - ldmia r0, {r3-r7}
> + adr_l r6, __pv_phys_pfn_offset
> + adr_l r7, __pv_offset @ __pa(__pv_offset)
> + mov_l r3, __pv_offset @ __va(__pv_offset)
> mvn ip, #0
> - subs r3, r0, r3 @ PHYS_OFFSET - PAGE_OFFSET
> - add r4, r4, r3 @ adjust table start address
> - add r5, r5, r3 @ adjust table end address
> - add r6, r6, r3 @ adjust __pv_phys_pfn_offset address
> - add r7, r7, r3 @ adjust __pv_offset address
> + subs r3, r7, r3 @ PHYS_OFFSET - PAGE_OFFSET
> mov r0, r8, lsr #PAGE_SHIFT @ convert to PFN
> str r0, [r6] @ save computed PHYS_OFFSET to __pv_phys_pfn_offset
> strcc ip, [r7, #HIGH_OFFSET] @ save to __pv_offset high bits
> @@ -594,20 +591,15 @@ __fixup_pv_table:
> THUMB( it ne @ cross section branch )
> bne __error
> str r3, [r7, #LOW_OFFSET] @ save to __pv_offset low bits
> + adr_l r4, __pv_table_begin
> + adr_l r5, __pv_table_end
> b __fixup_a_pv_table
> ENDPROC(__fixup_pv_table)
> -
> - .align
> -1: .long .
> - .long __pv_table_begin
> - .long __pv_table_end
> -2: .long __pv_phys_pfn_offset
> - .long __pv_offset
> + .ltorg
>
> .text
> __fixup_a_pv_table:
> - adr r0, 3f
> - ldr r6, [r0]
> + mov_l r6, __pv_offset
> add r6, r6, r3
> ldr r0, [r6, #HIGH_OFFSET] @ pv_offset high word
> ldr r6, [r6, #LOW_OFFSET] @ pv_offset low word
> @@ -676,9 +668,6 @@ ARM_BE8(rev16 ip, ip)
> #endif
> ENDPROC(__fixup_a_pv_table)
>
> - .align
> -3: .long __pv_offset
> -
> ENTRY(fixup_pv_table)
> stmfd sp!, {r4 - r7, lr}
> mov r3, #0 @ no offset
> --
> 2.11.0
>
>
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 11/29] ARM: head.S: use PC relative insn sequence to calculate PHYS_OFFSET
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (9 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 10/29] ARM: head.S: use PC-relative insn sequences for __fixup_pv_table Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-04 16:50 ` Nicolas Pitre
2017-09-03 12:07 ` [PATCH v2 12/29] ARM: kvm: replace open coded VA->PA calculations with adr_l call Ard Biesheuvel
` (18 subsequent siblings)
29 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
Replace the open coded arithmetic with a simple adr_l/sub pair. This
ensures these quantities are invariant under runtime relocation.
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/kernel/head.S | 10 ++--------
1 file changed, 2 insertions(+), 8 deletions(-)
diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
index f607e290ef4b..62c961849035 100644
--- a/arch/arm/kernel/head.S
+++ b/arch/arm/kernel/head.S
@@ -106,10 +106,8 @@ ENTRY(stext)
#endif
#ifndef CONFIG_XIP_KERNEL
- adr r3, 2f
- ldmia r3, {r4, r8}
- sub r4, r3, r4 @ (PHYS_OFFSET - PAGE_OFFSET)
- add r8, r8, r4 @ PHYS_OFFSET
+ adr_l r8, _text @ __pa(_text)
+ sub r8, r8, #TEXT_OFFSET @ PHYS_OFFSET
#else
ldr r8, =PLAT_PHYS_OFFSET @ always constant in this case
#endif
@@ -161,10 +159,6 @@ ENTRY(stext)
1: b __enable_mmu
ENDPROC(stext)
.ltorg
-#ifndef CONFIG_XIP_KERNEL
-2: .long .
- .long PAGE_OFFSET
-#endif
/*
* Setup the initial page tables. We only setup the barest
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 11/29] ARM: head.S: use PC relative insn sequence to calculate PHYS_OFFSET
2017-09-03 12:07 ` [PATCH v2 11/29] ARM: head.S: use PC relative insn sequence to calculate PHYS_OFFSET Ard Biesheuvel
@ 2017-09-04 16:50 ` Nicolas Pitre
0 siblings, 0 replies; 83+ messages in thread
From: Nicolas Pitre @ 2017-09-04 16:50 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
> Replace the open coded arithmetic with a simple adr_l/sub pair. This
> ensures these quantities are invariant under runtime relocation.
>
> Cc: Russell King <linux@armlinux.org.uk>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Of course there won't be anything to relocate in a XIP kernel. But
consistency is good and this looks nicer.
Acked-by: Nicolas Pitre <nico@linaro.org>
> ---
> arch/arm/kernel/head.S | 10 ++--------
> 1 file changed, 2 insertions(+), 8 deletions(-)
>
> diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
> index f607e290ef4b..62c961849035 100644
> --- a/arch/arm/kernel/head.S
> +++ b/arch/arm/kernel/head.S
> @@ -106,10 +106,8 @@ ENTRY(stext)
> #endif
>
> #ifndef CONFIG_XIP_KERNEL
> - adr r3, 2f
> - ldmia r3, {r4, r8}
> - sub r4, r3, r4 @ (PHYS_OFFSET - PAGE_OFFSET)
> - add r8, r8, r4 @ PHYS_OFFSET
> + adr_l r8, _text @ __pa(_text)
> + sub r8, r8, #TEXT_OFFSET @ PHYS_OFFSET
> #else
> ldr r8, =PLAT_PHYS_OFFSET @ always constant in this case
> #endif
> @@ -161,10 +159,6 @@ ENTRY(stext)
> 1: b __enable_mmu
> ENDPROC(stext)
> .ltorg
> -#ifndef CONFIG_XIP_KERNEL
> -2: .long .
> - .long PAGE_OFFSET
> -#endif
>
> /*
> * Setup the initial page tables. We only setup the barest
> --
> 2.11.0
>
>
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 12/29] ARM: kvm: replace open coded VA->PA calculations with adr_l call
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (10 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 11/29] ARM: head.S: use PC relative insn sequence to calculate PHYS_OFFSET Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-04 16:57 ` Nicolas Pitre
2017-09-03 12:07 ` [PATCH v2 13/29] arm-soc: exynos: replace open coded VA->PA conversions Ard Biesheuvel
` (17 subsequent siblings)
29 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
Replace the open coded calculations of the actual physical address
of the KVM stub vector table with a single adr_l invocation.
Cc: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/boot/compressed/head.S | 15 ++-------
arch/arm/kernel/hyp-stub.S | 33 +++++++-------------
arch/arm/kvm/init.S | 8 +----
3 files changed, 15 insertions(+), 41 deletions(-)
diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
index 8a756870c238..5884e8151376 100644
--- a/arch/arm/boot/compressed/head.S
+++ b/arch/arm/boot/compressed/head.S
@@ -427,15 +427,10 @@ dtb_check_done:
/*
* Compute the address of the hyp vectors after relocation.
- * This requires some arithmetic since we cannot directly
- * reference __hyp_stub_vectors in a PC-relative way.
* Call __hyp_set_vectors with the new address so that we
* can HVC again after the copy.
*/
-0: adr r0, 0b
- movw r1, #:lower16:__hyp_stub_vectors - 0b
- movt r1, #:upper16:__hyp_stub_vectors - 0b
- add r0, r0, r1
+ adr_l r0, __hyp_stub_vectors
sub r0, r0, r5
add r0, r0, r10
bl __hyp_set_vectors
@@ -568,17 +563,11 @@ not_relocated: mov r0, #0
cmp r0, #HYP_MODE @ if not booted in HYP mode...
bne __enter_kernel @ boot kernel directly
- adr r12, .L__hyp_reentry_vectors_offset
- ldr r0, [r12]
- add r0, r0, r12
-
+ adr_l r0, __hyp_reentry_vectors
bl __hyp_set_vectors
__HVC(0) @ otherwise bounce to hyp mode
b . @ should never be reached
-
- .align 2
-.L__hyp_reentry_vectors_offset: .long __hyp_reentry_vectors - .
#else
b __enter_kernel
#endif
diff --git a/arch/arm/kernel/hyp-stub.S b/arch/arm/kernel/hyp-stub.S
index ec7e7377d423..3c2d1738d3f4 100644
--- a/arch/arm/kernel/hyp-stub.S
+++ b/arch/arm/kernel/hyp-stub.S
@@ -36,41 +36,38 @@ ENTRY(__boot_cpu_mode)
.text
/*
- * Save the primary CPU boot mode. Requires 3 scratch registers.
+ * Save the primary CPU boot mode. Requires 2 scratch registers.
*/
- .macro store_primary_cpu_mode reg1, reg2, reg3
+ .macro store_primary_cpu_mode reg1, reg2
mrs \reg1, cpsr
and \reg1, \reg1, #MODE_MASK
- adr \reg2, .L__boot_cpu_mode_offset
- ldr \reg3, [\reg2]
- str \reg1, [\reg2, \reg3]
+ str_l \reg1, __boot_cpu_mode, \reg2
.endm
/*
* Compare the current mode with the one saved on the primary CPU.
* If they don't match, record that fact. The Z bit indicates
* if there's a match or not.
- * Requires 3 additionnal scratch registers.
+ * Requires 2 additional scratch registers.
*/
- .macro compare_cpu_mode_with_primary mode, reg1, reg2, reg3
- adr \reg2, .L__boot_cpu_mode_offset
- ldr \reg3, [\reg2]
- ldr \reg1, [\reg2, \reg3]
+ .macro compare_cpu_mode_with_primary mode, reg1, reg2
+ adr_l \reg2, __boot_cpu_mode
+ ldr \reg1, [\reg2]
cmp \mode, \reg1 @ matches primary CPU boot mode?
orrne \reg1, \reg1, #BOOT_CPU_MODE_MISMATCH
- strne \reg1, [\reg2, \reg3] @ record what happened and give up
+ strne \reg1, [\reg2] @ record what happened and give up
.endm
#else /* ZIMAGE */
- .macro store_primary_cpu_mode reg1:req, reg2:req, reg3:req
+ .macro store_primary_cpu_mode reg1:req, reg2:req
.endm
/*
* The zImage loader only runs on one CPU, so we don't bother with mult-CPU
* consistency checking:
*/
- .macro compare_cpu_mode_with_primary mode, reg1, reg2, reg3
+ .macro compare_cpu_mode_with_primary mode, reg1, reg2
cmp \mode, \mode
.endm
@@ -85,7 +82,7 @@ ENTRY(__boot_cpu_mode)
*/
@ Call this from the primary CPU
ENTRY(__hyp_stub_install)
- store_primary_cpu_mode r4, r5, r6
+ store_primary_cpu_mode r4, r5
ENDPROC(__hyp_stub_install)
@ fall through...
@@ -99,7 +96,7 @@ ENTRY(__hyp_stub_install_secondary)
* If the secondary has booted with a different mode, give up
* immediately.
*/
- compare_cpu_mode_with_primary r4, r5, r6, r7
+ compare_cpu_mode_with_primary r4, r5, r6
retne lr
/*
@@ -264,12 +261,6 @@ ENTRY(__hyp_reset_vectors)
ret lr
ENDPROC(__hyp_reset_vectors)
-#ifndef ZIMAGE
-.align 2
-.L__boot_cpu_mode_offset:
- .long __boot_cpu_mode - .
-#endif
-
.align 5
ENTRY(__hyp_stub_vectors)
__hyp_stub_reset: W(b) .
diff --git a/arch/arm/kvm/init.S b/arch/arm/kvm/init.S
index 5386528665b5..d777c6fbd869 100644
--- a/arch/arm/kvm/init.S
+++ b/arch/arm/kvm/init.S
@@ -143,13 +143,7 @@ reset:
bic r1, r1, r0
mcr p15, 4, r1, c1, c0, 0 @ HSCTLR
- /*
- * Install stub vectors, using ardb's VA->PA trick.
- */
-0: adr r0, 0b @ PA(0)
- movw r1, #:lower16:__hyp_stub_vectors - 0b @ VA(stub) - VA(0)
- movt r1, #:upper16:__hyp_stub_vectors - 0b
- add r1, r1, r0 @ PA(stub)
+ adr_l r1, __hyp_stub_vectors @ PA(stub)
mcr p15, 4, r1, c12, c0, 0 @ HVBAR
b exit
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 12/29] ARM: kvm: replace open coded VA->PA calculations with adr_l call
2017-09-03 12:07 ` [PATCH v2 12/29] ARM: kvm: replace open coded VA->PA calculations with adr_l call Ard Biesheuvel
@ 2017-09-04 16:57 ` Nicolas Pitre
0 siblings, 0 replies; 83+ messages in thread
From: Nicolas Pitre @ 2017-09-04 16:57 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
> Replace the open coded calculations of the actual physical address
> of the KVM stub vector table with a single adr_l invocation.
>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
> ---
> arch/arm/boot/compressed/head.S | 15 ++-------
> arch/arm/kernel/hyp-stub.S | 33 +++++++-------------
> arch/arm/kvm/init.S | 8 +----
> 3 files changed, 15 insertions(+), 41 deletions(-)
>
> diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
> index 8a756870c238..5884e8151376 100644
> --- a/arch/arm/boot/compressed/head.S
> +++ b/arch/arm/boot/compressed/head.S
> @@ -427,15 +427,10 @@ dtb_check_done:
>
> /*
> * Compute the address of the hyp vectors after relocation.
> - * This requires some arithmetic since we cannot directly
> - * reference __hyp_stub_vectors in a PC-relative way.
> * Call __hyp_set_vectors with the new address so that we
> * can HVC again after the copy.
> */
> -0: adr r0, 0b
> - movw r1, #:lower16:__hyp_stub_vectors - 0b
> - movt r1, #:upper16:__hyp_stub_vectors - 0b
> - add r0, r0, r1
> + adr_l r0, __hyp_stub_vectors
> sub r0, r0, r5
> add r0, r0, r10
> bl __hyp_set_vectors
> @@ -568,17 +563,11 @@ not_relocated: mov r0, #0
> cmp r0, #HYP_MODE @ if not booted in HYP mode...
> bne __enter_kernel @ boot kernel directly
>
> - adr r12, .L__hyp_reentry_vectors_offset
> - ldr r0, [r12]
> - add r0, r0, r12
> -
> + adr_l r0, __hyp_reentry_vectors
> bl __hyp_set_vectors
> __HVC(0) @ otherwise bounce to hyp mode
>
> b . @ should never be reached
> -
> - .align 2
> -.L__hyp_reentry_vectors_offset: .long __hyp_reentry_vectors - .
> #else
> b __enter_kernel
> #endif
> diff --git a/arch/arm/kernel/hyp-stub.S b/arch/arm/kernel/hyp-stub.S
> index ec7e7377d423..3c2d1738d3f4 100644
> --- a/arch/arm/kernel/hyp-stub.S
> +++ b/arch/arm/kernel/hyp-stub.S
> @@ -36,41 +36,38 @@ ENTRY(__boot_cpu_mode)
> .text
>
> /*
> - * Save the primary CPU boot mode. Requires 3 scratch registers.
> + * Save the primary CPU boot mode. Requires 2 scratch registers.
> */
> - .macro store_primary_cpu_mode reg1, reg2, reg3
> + .macro store_primary_cpu_mode reg1, reg2
> mrs \reg1, cpsr
> and \reg1, \reg1, #MODE_MASK
> - adr \reg2, .L__boot_cpu_mode_offset
> - ldr \reg3, [\reg2]
> - str \reg1, [\reg2, \reg3]
> + str_l \reg1, __boot_cpu_mode, \reg2
> .endm
>
> /*
> * Compare the current mode with the one saved on the primary CPU.
> * If they don't match, record that fact. The Z bit indicates
> * if there's a match or not.
> - * Requires 3 additionnal scratch registers.
> + * Requires 2 additional scratch registers.
> */
> - .macro compare_cpu_mode_with_primary mode, reg1, reg2, reg3
> - adr \reg2, .L__boot_cpu_mode_offset
> - ldr \reg3, [\reg2]
> - ldr \reg1, [\reg2, \reg3]
> + .macro compare_cpu_mode_with_primary mode, reg1, reg2
> + adr_l \reg2, __boot_cpu_mode
> + ldr \reg1, [\reg2]
> cmp \mode, \reg1 @ matches primary CPU boot mode?
> orrne \reg1, \reg1, #BOOT_CPU_MODE_MISMATCH
> - strne \reg1, [\reg2, \reg3] @ record what happened and give up
> + strne \reg1, [\reg2] @ record what happened and give up
> .endm
>
> #else /* ZIMAGE */
>
> - .macro store_primary_cpu_mode reg1:req, reg2:req, reg3:req
> + .macro store_primary_cpu_mode reg1:req, reg2:req
> .endm
>
> /*
> * The zImage loader only runs on one CPU, so we don't bother with mult-CPU
> * consistency checking:
> */
> - .macro compare_cpu_mode_with_primary mode, reg1, reg2, reg3
> + .macro compare_cpu_mode_with_primary mode, reg1, reg2
> cmp \mode, \mode
> .endm
>
> @@ -85,7 +82,7 @@ ENTRY(__boot_cpu_mode)
> */
> @ Call this from the primary CPU
> ENTRY(__hyp_stub_install)
> - store_primary_cpu_mode r4, r5, r6
> + store_primary_cpu_mode r4, r5
> ENDPROC(__hyp_stub_install)
>
> @ fall through...
> @@ -99,7 +96,7 @@ ENTRY(__hyp_stub_install_secondary)
> * If the secondary has booted with a different mode, give up
> * immediately.
> */
> - compare_cpu_mode_with_primary r4, r5, r6, r7
> + compare_cpu_mode_with_primary r4, r5, r6
> retne lr
>
> /*
> @@ -264,12 +261,6 @@ ENTRY(__hyp_reset_vectors)
> ret lr
> ENDPROC(__hyp_reset_vectors)
>
> -#ifndef ZIMAGE
> -.align 2
> -.L__boot_cpu_mode_offset:
> - .long __boot_cpu_mode - .
> -#endif
> -
> .align 5
> ENTRY(__hyp_stub_vectors)
> __hyp_stub_reset: W(b) .
> diff --git a/arch/arm/kvm/init.S b/arch/arm/kvm/init.S
> index 5386528665b5..d777c6fbd869 100644
> --- a/arch/arm/kvm/init.S
> +++ b/arch/arm/kvm/init.S
> @@ -143,13 +143,7 @@ reset:
> bic r1, r1, r0
> mcr p15, 4, r1, c1, c0, 0 @ HSCTLR
>
> - /*
> - * Install stub vectors, using ardb's VA->PA trick.
> - */
> -0: adr r0, 0b @ PA(0)
> - movw r1, #:lower16:__hyp_stub_vectors - 0b @ VA(stub) - VA(0)
> - movt r1, #:upper16:__hyp_stub_vectors - 0b
> - add r1, r1, r0 @ PA(stub)
> + adr_l r1, __hyp_stub_vectors @ PA(stub)
> mcr p15, 4, r1, c12, c0, 0 @ HVBAR
> b exit
>
> --
> 2.11.0
>
>
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 13/29] arm-soc: exynos: replace open coded VA->PA conversions
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (11 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 12/29] ARM: kvm: replace open coded VA->PA calculations with adr_l call Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-04 16:59 ` Nicolas Pitre
2017-09-03 12:07 ` [PATCH v2 14/29] arm-soc: mvebu: replace open coded VA->PA conversion Ard Biesheuvel
` (16 subsequent siblings)
29 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
This replaces a couple of open coded calculations to obtain the
physical address of a far symbol with calls to the new adr_l etc
macros.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/mach-exynos/headsmp.S | 9 +------
arch/arm/mach-exynos/sleep.S | 26 ++++----------------
2 files changed, 6 insertions(+), 29 deletions(-)
diff --git a/arch/arm/mach-exynos/headsmp.S b/arch/arm/mach-exynos/headsmp.S
index d3d24ab351ae..25de5aff1c41 100644
--- a/arch/arm/mach-exynos/headsmp.S
+++ b/arch/arm/mach-exynos/headsmp.S
@@ -23,10 +23,7 @@ ENTRY(exynos4_secondary_startup)
ARM_BE8(setend be)
mrc p15, 0, r0, c0, c0, 5
and r0, r0, #15
- adr r4, 1f
- ldmia r4, {r5, r6}
- sub r4, r4, r5
- add r6, r6, r4
+ adr_l r6, pen_release
pen: ldr r7, [r6]
cmp r7, r0
bne pen
@@ -37,7 +34,3 @@ pen: ldr r7, [r6]
*/
b secondary_startup
ENDPROC(exynos4_secondary_startup)
-
- .align 2
-1: .long .
- .long pen_release
diff --git a/arch/arm/mach-exynos/sleep.S b/arch/arm/mach-exynos/sleep.S
index cf950790fbdc..003465699263 100644
--- a/arch/arm/mach-exynos/sleep.S
+++ b/arch/arm/mach-exynos/sleep.S
@@ -17,6 +17,7 @@
#include <linux/linkage.h>
#include <asm/asm-offsets.h>
+#include <asm/assembler.h>
#include <asm/hardware/cache-l2x0.h>
#include "smc.h"
@@ -62,19 +63,13 @@ ENTRY(exynos_cpu_resume_ns)
cmp r0, r1
bne skip_cp15
- adr r0, _cp15_save_power
- ldr r1, [r0]
- ldr r1, [r0, r1]
- adr r0, _cp15_save_diag
- ldr r2, [r0]
- ldr r2, [r0, r2]
+ ldr_l r1, cp15_save_power
+ ldr_l r2, cp15_save_diag
mov r0, #SMC_CMD_C15RESUME
dsb
smc #0
#ifdef CONFIG_CACHE_L2X0
- adr r0, 1f
- ldr r2, [r0]
- add r0, r2, r0
+ adr_l r0, l2x0_saved_regs
/* Check that the address has been initialised. */
ldr r1, [r0, #L2X0_R_PHY_BASE]
@@ -93,9 +88,7 @@ ENTRY(exynos_cpu_resume_ns)
smc #0
/* Reload saved regs pointer because smc corrupts registers. */
- adr r0, 1f
- ldr r2, [r0]
- add r0, r2, r0
+ adr_l r0, l2x0_saved_regs
ldr r1, [r0, #L2X0_R_PWR_CTRL]
ldr r2, [r0, #L2X0_R_AUX_CTRL]
@@ -114,15 +107,6 @@ skip_cp15:
b cpu_resume
ENDPROC(exynos_cpu_resume_ns)
- .align
-_cp15_save_power:
- .long cp15_save_power - .
-_cp15_save_diag:
- .long cp15_save_diag - .
-#ifdef CONFIG_CACHE_L2X0
-1: .long l2x0_saved_regs - .
-#endif /* CONFIG_CACHE_L2X0 */
-
.data
.globl cp15_save_diag
cp15_save_diag:
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 13/29] arm-soc: exynos: replace open coded VA->PA conversions
2017-09-03 12:07 ` [PATCH v2 13/29] arm-soc: exynos: replace open coded VA->PA conversions Ard Biesheuvel
@ 2017-09-04 16:59 ` Nicolas Pitre
0 siblings, 0 replies; 83+ messages in thread
From: Nicolas Pitre @ 2017-09-04 16:59 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
> This replaces a couple of open coded calculations to obtain the
> physical address of a far symbol with calls to the new adr_l etc
> macros.
>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
> ---
> arch/arm/mach-exynos/headsmp.S | 9 +------
> arch/arm/mach-exynos/sleep.S | 26 ++++----------------
> 2 files changed, 6 insertions(+), 29 deletions(-)
>
> diff --git a/arch/arm/mach-exynos/headsmp.S b/arch/arm/mach-exynos/headsmp.S
> index d3d24ab351ae..25de5aff1c41 100644
> --- a/arch/arm/mach-exynos/headsmp.S
> +++ b/arch/arm/mach-exynos/headsmp.S
> @@ -23,10 +23,7 @@ ENTRY(exynos4_secondary_startup)
> ARM_BE8(setend be)
> mrc p15, 0, r0, c0, c0, 5
> and r0, r0, #15
> - adr r4, 1f
> - ldmia r4, {r5, r6}
> - sub r4, r4, r5
> - add r6, r6, r4
> + adr_l r6, pen_release
> pen: ldr r7, [r6]
> cmp r7, r0
> bne pen
> @@ -37,7 +34,3 @@ pen: ldr r7, [r6]
> */
> b secondary_startup
> ENDPROC(exynos4_secondary_startup)
> -
> - .align 2
> -1: .long .
> - .long pen_release
> diff --git a/arch/arm/mach-exynos/sleep.S b/arch/arm/mach-exynos/sleep.S
> index cf950790fbdc..003465699263 100644
> --- a/arch/arm/mach-exynos/sleep.S
> +++ b/arch/arm/mach-exynos/sleep.S
> @@ -17,6 +17,7 @@
>
> #include <linux/linkage.h>
> #include <asm/asm-offsets.h>
> +#include <asm/assembler.h>
> #include <asm/hardware/cache-l2x0.h>
> #include "smc.h"
>
> @@ -62,19 +63,13 @@ ENTRY(exynos_cpu_resume_ns)
> cmp r0, r1
> bne skip_cp15
>
> - adr r0, _cp15_save_power
> - ldr r1, [r0]
> - ldr r1, [r0, r1]
> - adr r0, _cp15_save_diag
> - ldr r2, [r0]
> - ldr r2, [r0, r2]
> + ldr_l r1, cp15_save_power
> + ldr_l r2, cp15_save_diag
> mov r0, #SMC_CMD_C15RESUME
> dsb
> smc #0
> #ifdef CONFIG_CACHE_L2X0
> - adr r0, 1f
> - ldr r2, [r0]
> - add r0, r2, r0
> + adr_l r0, l2x0_saved_regs
>
> /* Check that the address has been initialised. */
> ldr r1, [r0, #L2X0_R_PHY_BASE]
> @@ -93,9 +88,7 @@ ENTRY(exynos_cpu_resume_ns)
> smc #0
>
> /* Reload saved regs pointer because smc corrupts registers. */
> - adr r0, 1f
> - ldr r2, [r0]
> - add r0, r2, r0
> + adr_l r0, l2x0_saved_regs
>
> ldr r1, [r0, #L2X0_R_PWR_CTRL]
> ldr r2, [r0, #L2X0_R_AUX_CTRL]
> @@ -114,15 +107,6 @@ skip_cp15:
> b cpu_resume
> ENDPROC(exynos_cpu_resume_ns)
>
> - .align
> -_cp15_save_power:
> - .long cp15_save_power - .
> -_cp15_save_diag:
> - .long cp15_save_diag - .
> -#ifdef CONFIG_CACHE_L2X0
> -1: .long l2x0_saved_regs - .
> -#endif /* CONFIG_CACHE_L2X0 */
> -
> .data
> .globl cp15_save_diag
> cp15_save_diag:
> --
> 2.11.0
>
>
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 14/29] arm-soc: mvebu: replace open coded VA->PA conversion
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (12 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 13/29] arm-soc: exynos: replace open coded VA->PA conversions Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-04 17:00 ` Nicolas Pitre
2017-09-03 12:07 ` [PATCH v2 15/29] arm-soc: various: replace open coded VA->PA calculation of pen_release Ard Biesheuvel
` (15 subsequent siblings)
29 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
This replaces an open coded calculation to obtain the physical
address of a far symbol with a call to the new ldr_l etc macro.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/mach-mvebu/coherency_ll.S | 8 +-------
1 file changed, 1 insertion(+), 7 deletions(-)
diff --git a/arch/arm/mach-mvebu/coherency_ll.S b/arch/arm/mach-mvebu/coherency_ll.S
index 8b2fbc8b6bc6..df0ccb5cef1d 100644
--- a/arch/arm/mach-mvebu/coherency_ll.S
+++ b/arch/arm/mach-mvebu/coherency_ll.S
@@ -42,9 +42,7 @@ ENTRY(ll_get_coherency_base)
ldr r1, =coherency_base
cmp r1, #0
beq 2f
- adr r1, 3f
- ldr r3, [r1]
- ldr r1, [r1, r3]
+ ldr_l r1, coherency_phys_base
b 2f
1:
/*
@@ -160,7 +158,3 @@ ENTRY(ll_disable_coherency)
dsb
ret lr
ENDPROC(ll_disable_coherency)
-
- .align 2
-3:
- .long coherency_phys_base - .
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 14/29] arm-soc: mvebu: replace open coded VA->PA conversion
2017-09-03 12:07 ` [PATCH v2 14/29] arm-soc: mvebu: replace open coded VA->PA conversion Ard Biesheuvel
@ 2017-09-04 17:00 ` Nicolas Pitre
0 siblings, 0 replies; 83+ messages in thread
From: Nicolas Pitre @ 2017-09-04 17:00 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
> This replaces an open coded calculation to obtain the physical
> address of a far symbol with a call to the new ldr_l etc macro.
>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
> ---
> arch/arm/mach-mvebu/coherency_ll.S | 8 +-------
> 1 file changed, 1 insertion(+), 7 deletions(-)
>
> diff --git a/arch/arm/mach-mvebu/coherency_ll.S b/arch/arm/mach-mvebu/coherency_ll.S
> index 8b2fbc8b6bc6..df0ccb5cef1d 100644
> --- a/arch/arm/mach-mvebu/coherency_ll.S
> +++ b/arch/arm/mach-mvebu/coherency_ll.S
> @@ -42,9 +42,7 @@ ENTRY(ll_get_coherency_base)
> ldr r1, =coherency_base
> cmp r1, #0
> beq 2f
> - adr r1, 3f
> - ldr r3, [r1]
> - ldr r1, [r1, r3]
> + ldr_l r1, coherency_phys_base
> b 2f
> 1:
> /*
> @@ -160,7 +158,3 @@ ENTRY(ll_disable_coherency)
> dsb
> ret lr
> ENDPROC(ll_disable_coherency)
> -
> - .align 2
> -3:
> - .long coherency_phys_base - .
> --
> 2.11.0
>
>
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 15/29] arm-soc: various: replace open coded VA->PA calculation of pen_release
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (13 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 14/29] arm-soc: mvebu: replace open coded VA->PA conversion Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-04 17:01 ` Nicolas Pitre
2017-09-03 12:07 ` [PATCH v2 16/29] ARM: kernel: switch to relative exception tables Ard Biesheuvel
` (14 subsequent siblings)
29 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
This replaces a few copies of the open coded calculations of the
physical address of 'pen_release' in the secondary startup code
of a couple of platforms. This ensures these quantities are invariant
under runtime relocation.
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/mach-prima2/headsmp.S | 11 +++--------
arch/arm/mach-spear/headsmp.S | 11 +++--------
arch/arm/mach-sti/headsmp.S | 10 +++-------
arch/arm/plat-versatile/headsmp.S | 9 +--------
4 files changed, 10 insertions(+), 31 deletions(-)
diff --git a/arch/arm/mach-prima2/headsmp.S b/arch/arm/mach-prima2/headsmp.S
index 209d9fc5c16c..070df700bb38 100644
--- a/arch/arm/mach-prima2/headsmp.S
+++ b/arch/arm/mach-prima2/headsmp.S
@@ -9,6 +9,8 @@
#include <linux/linkage.h>
#include <linux/init.h>
+#include <asm/assembler.h>
+
/*
* SIRFSOC specific entry point for secondary CPUs. This provides
* a "holding pen" into which all secondary cores are held until we're
@@ -17,10 +19,7 @@
ENTRY(sirfsoc_secondary_startup)
mrc p15, 0, r0, c0, c0, 5
and r0, r0, #15
- adr r4, 1f
- ldmia r4, {r5, r6}
- sub r4, r4, r5
- add r6, r6, r4
+ adr_l r6, pen_release
pen: ldr r7, [r6]
cmp r7, r0
bne pen
@@ -31,7 +30,3 @@ pen: ldr r7, [r6]
*/
b secondary_startup
ENDPROC(sirfsoc_secondary_startup)
-
- .align
-1: .long .
- .long pen_release
diff --git a/arch/arm/mach-spear/headsmp.S b/arch/arm/mach-spear/headsmp.S
index c52192dc3d9f..4da01b103f33 100644
--- a/arch/arm/mach-spear/headsmp.S
+++ b/arch/arm/mach-spear/headsmp.S
@@ -13,6 +13,8 @@
#include <linux/linkage.h>
#include <linux/init.h>
+#include <asm/assembler.h>
+
__INIT
/*
@@ -23,10 +25,7 @@
ENTRY(spear13xx_secondary_startup)
mrc p15, 0, r0, c0, c0, 5
and r0, r0, #15
- adr r4, 1f
- ldmia r4, {r5, r6}
- sub r4, r4, r5
- add r6, r6, r4
+ adr_l r6, pen_release
pen: ldr r7, [r6]
cmp r7, r0
bne pen
@@ -40,8 +39,4 @@ pen: ldr r7, [r6]
* should now contain the SVC stack for this core
*/
b secondary_startup
-
- .align
-1: .long .
- .long pen_release
ENDPROC(spear13xx_secondary_startup)
diff --git a/arch/arm/mach-sti/headsmp.S b/arch/arm/mach-sti/headsmp.S
index e0ad451700d5..cdf3442f397b 100644
--- a/arch/arm/mach-sti/headsmp.S
+++ b/arch/arm/mach-sti/headsmp.S
@@ -16,6 +16,8 @@
#include <linux/linkage.h>
#include <linux/init.h>
+#include <asm/assembler.h>
+
/*
* ST specific entry point for secondary CPUs. This provides
* a "holding pen" into which all secondary cores are held until we're
@@ -24,10 +26,7 @@
ENTRY(sti_secondary_startup)
mrc p15, 0, r0, c0, c0, 5
and r0, r0, #15
- adr r4, 1f
- ldmia r4, {r5, r6}
- sub r4, r4, r5
- add r6, r6, r4
+ adr_l r6, pen_release
pen: ldr r7, [r6]
cmp r7, r0
bne pen
@@ -38,6 +37,3 @@ pen: ldr r7, [r6]
*/
b secondary_startup
ENDPROC(sti_secondary_startup)
-
-1: .long .
- .long pen_release
diff --git a/arch/arm/plat-versatile/headsmp.S b/arch/arm/plat-versatile/headsmp.S
index 40f27e52de75..0f2a5eddac5a 100644
--- a/arch/arm/plat-versatile/headsmp.S
+++ b/arch/arm/plat-versatile/headsmp.S
@@ -21,10 +21,7 @@ ENTRY(versatile_secondary_startup)
ARM_BE8(setend be)
mrc p15, 0, r0, c0, c0, 5
bic r0, #0xff000000
- adr r4, 1f
- ldmia r4, {r5, r6}
- sub r4, r4, r5
- add r6, r6, r4
+ adr_l r6, pen_release
pen: ldr r7, [r6]
cmp r7, r0
bne pen
@@ -34,8 +31,4 @@ pen: ldr r7, [r6]
* should now contain the SVC stack for this core
*/
b secondary_startup
-
- .align
-1: .long .
- .long pen_release
ENDPROC(versatile_secondary_startup)
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 15/29] arm-soc: various: replace open coded VA->PA calculation of pen_release
2017-09-03 12:07 ` [PATCH v2 15/29] arm-soc: various: replace open coded VA->PA calculation of pen_release Ard Biesheuvel
@ 2017-09-04 17:01 ` Nicolas Pitre
0 siblings, 0 replies; 83+ messages in thread
From: Nicolas Pitre @ 2017-09-04 17:01 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
> This replaces a few copies of the open coded calculations of the
> physical address of 'pen_release' in the secondary startup code
> of a couple of platforms. This ensures these quantities are invariant
> under runtime relocation.
>
> Cc: Russell King <linux@armlinux.org.uk>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
> ---
> arch/arm/mach-prima2/headsmp.S | 11 +++--------
> arch/arm/mach-spear/headsmp.S | 11 +++--------
> arch/arm/mach-sti/headsmp.S | 10 +++-------
> arch/arm/plat-versatile/headsmp.S | 9 +--------
> 4 files changed, 10 insertions(+), 31 deletions(-)
>
> diff --git a/arch/arm/mach-prima2/headsmp.S b/arch/arm/mach-prima2/headsmp.S
> index 209d9fc5c16c..070df700bb38 100644
> --- a/arch/arm/mach-prima2/headsmp.S
> +++ b/arch/arm/mach-prima2/headsmp.S
> @@ -9,6 +9,8 @@
> #include <linux/linkage.h>
> #include <linux/init.h>
>
> +#include <asm/assembler.h>
> +
> /*
> * SIRFSOC specific entry point for secondary CPUs. This provides
> * a "holding pen" into which all secondary cores are held until we're
> @@ -17,10 +19,7 @@
> ENTRY(sirfsoc_secondary_startup)
> mrc p15, 0, r0, c0, c0, 5
> and r0, r0, #15
> - adr r4, 1f
> - ldmia r4, {r5, r6}
> - sub r4, r4, r5
> - add r6, r6, r4
> + adr_l r6, pen_release
> pen: ldr r7, [r6]
> cmp r7, r0
> bne pen
> @@ -31,7 +30,3 @@ pen: ldr r7, [r6]
> */
> b secondary_startup
> ENDPROC(sirfsoc_secondary_startup)
> -
> - .align
> -1: .long .
> - .long pen_release
> diff --git a/arch/arm/mach-spear/headsmp.S b/arch/arm/mach-spear/headsmp.S
> index c52192dc3d9f..4da01b103f33 100644
> --- a/arch/arm/mach-spear/headsmp.S
> +++ b/arch/arm/mach-spear/headsmp.S
> @@ -13,6 +13,8 @@
> #include <linux/linkage.h>
> #include <linux/init.h>
>
> +#include <asm/assembler.h>
> +
> __INIT
>
> /*
> @@ -23,10 +25,7 @@
> ENTRY(spear13xx_secondary_startup)
> mrc p15, 0, r0, c0, c0, 5
> and r0, r0, #15
> - adr r4, 1f
> - ldmia r4, {r5, r6}
> - sub r4, r4, r5
> - add r6, r6, r4
> + adr_l r6, pen_release
> pen: ldr r7, [r6]
> cmp r7, r0
> bne pen
> @@ -40,8 +39,4 @@ pen: ldr r7, [r6]
> * should now contain the SVC stack for this core
> */
> b secondary_startup
> -
> - .align
> -1: .long .
> - .long pen_release
> ENDPROC(spear13xx_secondary_startup)
> diff --git a/arch/arm/mach-sti/headsmp.S b/arch/arm/mach-sti/headsmp.S
> index e0ad451700d5..cdf3442f397b 100644
> --- a/arch/arm/mach-sti/headsmp.S
> +++ b/arch/arm/mach-sti/headsmp.S
> @@ -16,6 +16,8 @@
> #include <linux/linkage.h>
> #include <linux/init.h>
>
> +#include <asm/assembler.h>
> +
> /*
> * ST specific entry point for secondary CPUs. This provides
> * a "holding pen" into which all secondary cores are held until we're
> @@ -24,10 +26,7 @@
> ENTRY(sti_secondary_startup)
> mrc p15, 0, r0, c0, c0, 5
> and r0, r0, #15
> - adr r4, 1f
> - ldmia r4, {r5, r6}
> - sub r4, r4, r5
> - add r6, r6, r4
> + adr_l r6, pen_release
> pen: ldr r7, [r6]
> cmp r7, r0
> bne pen
> @@ -38,6 +37,3 @@ pen: ldr r7, [r6]
> */
> b secondary_startup
> ENDPROC(sti_secondary_startup)
> -
> -1: .long .
> - .long pen_release
> diff --git a/arch/arm/plat-versatile/headsmp.S b/arch/arm/plat-versatile/headsmp.S
> index 40f27e52de75..0f2a5eddac5a 100644
> --- a/arch/arm/plat-versatile/headsmp.S
> +++ b/arch/arm/plat-versatile/headsmp.S
> @@ -21,10 +21,7 @@ ENTRY(versatile_secondary_startup)
> ARM_BE8(setend be)
> mrc p15, 0, r0, c0, c0, 5
> bic r0, #0xff000000
> - adr r4, 1f
> - ldmia r4, {r5, r6}
> - sub r4, r4, r5
> - add r6, r6, r4
> + adr_l r6, pen_release
> pen: ldr r7, [r6]
> cmp r7, r0
> bne pen
> @@ -34,8 +31,4 @@ pen: ldr r7, [r6]
> * should now contain the SVC stack for this core
> */
> b secondary_startup
> -
> - .align
> -1: .long .
> - .long pen_release
> ENDPROC(versatile_secondary_startup)
> --
> 2.11.0
>
>
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 16/29] ARM: kernel: switch to relative exception tables
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (14 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 15/29] arm-soc: various: replace open coded VA->PA calculation of pen_release Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-04 17:17 ` Nicolas Pitre
2017-09-03 12:07 ` [PATCH v2 17/29] ARM: kernel: use relative phys-to-virt patch tables Ard Biesheuvel
` (13 subsequent siblings)
29 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
To avoid having to relocate the contents of extable entries at
runtime when running with KASLR enabled, wire up the existing
support for emitting them as relative references. This ensures
these quantities are invariant under runtime relocation.
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/include/asm/Kbuild | 1 -
arch/arm/include/asm/assembler.h | 6 +++---
arch/arm/include/asm/extable.h | 19 +++++++++++++++++
arch/arm/include/asm/futex.h | 2 +-
arch/arm/include/asm/uaccess.h | 8 +++----
arch/arm/include/asm/word-at-a-time.h | 2 +-
arch/arm/kernel/entry-armv.S | 6 +++---
arch/arm/kernel/swp_emulate.c | 4 ++--
arch/arm/lib/backtrace.S | 8 +++----
arch/arm/lib/getuser.S | 22 ++++++++++----------
arch/arm/lib/putuser.S | 12 +++++------
arch/arm/mm/alignment.c | 14 ++++++-------
arch/arm/mm/extable.c | 2 +-
arch/arm/nwfpe/entry.S | 2 +-
scripts/sortextable.c | 2 +-
15 files changed, 64 insertions(+), 46 deletions(-)
diff --git a/arch/arm/include/asm/Kbuild b/arch/arm/include/asm/Kbuild
index 721ab5ecfb9b..f2d67fa34eb3 100644
--- a/arch/arm/include/asm/Kbuild
+++ b/arch/arm/include/asm/Kbuild
@@ -3,7 +3,6 @@ generic-y += current.h
generic-y += early_ioremap.h
generic-y += emergency-restart.h
generic-y += exec.h
-generic-y += extable.h
generic-y += irq_regs.h
generic-y += kdebug.h
generic-y += local.h
diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h
index a0906c1fc65d..ae43859a9000 100644
--- a/arch/arm/include/asm/assembler.h
+++ b/arch/arm/include/asm/assembler.h
@@ -247,7 +247,7 @@
9999: x; \
.pushsection __ex_table,"a"; \
.align 3; \
- .long 9999b,9001f; \
+ .long 9999b - ., 9001f - .; \
.popsection
#ifdef CONFIG_SMP
@@ -383,7 +383,7 @@ THUMB( orr \reg , \reg , #PSR_T_BIT )
.pushsection __ex_table,"a"
.align 3
- .long 9999b, \abort
+ .long 9999b - ., \abort - .
.popsection
.endm
@@ -424,7 +424,7 @@ THUMB( orr \reg , \reg , #PSR_T_BIT )
.pushsection __ex_table,"a"
.align 3
- .long 9999b, \abort
+ .long 9999b - ., \abort - .
.popsection
.endr
.endm
diff --git a/arch/arm/include/asm/extable.h b/arch/arm/include/asm/extable.h
new file mode 100644
index 000000000000..c85d1f4404b7
--- /dev/null
+++ b/arch/arm/include/asm/extable.h
@@ -0,0 +1,19 @@
+#ifndef __ASM_EXTABLE_H
+#define __ASM_EXTABLE_H
+
+/*
+ * The exception table consists of pairs of relative offsets: the first
+ * is the relative offset to an instruction that is allowed to fault,
+ * and the second is the relative offset at which the program should
+ * continue. No registers are modified, so it is entirely up to the
+ * continuation code to figure out what to do.
+ */
+
+struct exception_table_entry {
+ int insn, fixup;
+};
+
+#define ARCH_HAS_RELATIVE_EXTABLE
+
+extern int fixup_exception(struct pt_regs *regs);
+#endif
diff --git a/arch/arm/include/asm/futex.h b/arch/arm/include/asm/futex.h
index 6795368ad023..31760d0ab76b 100644
--- a/arch/arm/include/asm/futex.h
+++ b/arch/arm/include/asm/futex.h
@@ -11,7 +11,7 @@
"3:\n" \
" .pushsection __ex_table,\"a\"\n" \
" .align 3\n" \
- " .long 1b, 4f, 2b, 4f\n" \
+ " .long 1b - ., 4f - ., 2b - ., 4f - .\n" \
" .popsection\n" \
" .pushsection .text.fixup,\"ax\"\n" \
" .align 2\n" \
diff --git a/arch/arm/include/asm/uaccess.h b/arch/arm/include/asm/uaccess.h
index 0bf2347495f1..62b9bb7d991a 100644
--- a/arch/arm/include/asm/uaccess.h
+++ b/arch/arm/include/asm/uaccess.h
@@ -295,7 +295,7 @@ do { \
" .popsection\n" \
" .pushsection __ex_table,\"a\"\n" \
" .align 3\n" \
- " .long 1b, 3b\n" \
+ " .long 1b - ., 3b - .\n" \
" .popsection" \
: "+r" (err), "=&r" (x) \
: "r" (addr), "i" (-EFAULT) \
@@ -385,7 +385,7 @@ do { \
" .popsection\n" \
" .pushsection __ex_table,\"a\"\n" \
" .align 3\n" \
- " .long 1b, 3b\n" \
+ " .long 1b - ., 3b - .\n" \
" .popsection" \
: "+r" (err) \
: "r" (x), "r" (__pu_addr), "i" (-EFAULT) \
@@ -435,8 +435,8 @@ do { \
" .popsection\n" \
" .pushsection __ex_table,\"a\"\n" \
" .align 3\n" \
- " .long 1b, 4b\n" \
- " .long 2b, 4b\n" \
+ " .long 1b - ., 4b - .\n" \
+ " .long 2b - ., 4b - .\n" \
" .popsection" \
: "+r" (err), "+r" (__pu_addr) \
: "r" (x), "i" (-EFAULT) \
diff --git a/arch/arm/include/asm/word-at-a-time.h b/arch/arm/include/asm/word-at-a-time.h
index 5831dce4b51c..d433c686d9ca 100644
--- a/arch/arm/include/asm/word-at-a-time.h
+++ b/arch/arm/include/asm/word-at-a-time.h
@@ -86,7 +86,7 @@ static inline unsigned long load_unaligned_zeropad(const void *addr)
" .popsection\n"
" .pushsection __ex_table,\"a\"\n"
" .align 3\n"
- " .long 1b, 3b\n"
+ " .long 1b - ., 3b - .\n"
" .popsection"
: "=&r" (ret), "=&r" (offset)
: "r" (addr), "Qo" (*(unsigned long *)addr));
diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S
index c731f0d2b2af..1c7c12123b18 100644
--- a/arch/arm/kernel/entry-armv.S
+++ b/arch/arm/kernel/entry-armv.S
@@ -570,10 +570,10 @@ ENDPROC(__und_usr)
ret r9
.popsection
.pushsection __ex_table,"a"
- .long 1b, 4b
+ .long 1b - ., 4b - .
#if CONFIG_ARM_THUMB && __LINUX_ARM_ARCH__ >= 6 && CONFIG_CPU_V7
- .long 2b, 4b
- .long 3b, 4b
+ .long 2b - ., 4b - .
+ .long 3b - ., 4b - .
#endif
.popsection
diff --git a/arch/arm/kernel/swp_emulate.c b/arch/arm/kernel/swp_emulate.c
index 3bda08bee674..a783e9c0b266 100644
--- a/arch/arm/kernel/swp_emulate.c
+++ b/arch/arm/kernel/swp_emulate.c
@@ -50,8 +50,8 @@
" .previous\n" \
" .section __ex_table,\"a\"\n" \
" .align 3\n" \
- " .long 0b, 3b\n" \
- " .long 1b, 3b\n" \
+ " .long 0b - ., 3b - .\n" \
+ " .long 1b - ., 3b - .\n" \
" .previous" \
: "=&r" (res), "+r" (data), "=&r" (temp) \
: "r" (addr), "i" (-EAGAIN), "i" (-EFAULT) \
diff --git a/arch/arm/lib/backtrace.S b/arch/arm/lib/backtrace.S
index 7d7952e5a3b1..84a8df7aa63c 100644
--- a/arch/arm/lib/backtrace.S
+++ b/arch/arm/lib/backtrace.S
@@ -107,10 +107,10 @@ ENDPROC(c_backtrace)
.pushsection __ex_table,"a"
.align 3
- .long 1001b, 1006b
- .long 1002b, 1006b
- .long 1003b, 1006b
- .long 1004b, 1006b
+ .long 1001b - ., 1006b - .
+ .long 1002b - ., 1006b - .
+ .long 1003b - ., 1006b - .
+ .long 1004b - ., 1006b - .
.popsection
.Lbad: .asciz "Backtrace aborted due to bad frame pointer <%p>\n"
diff --git a/arch/arm/lib/getuser.S b/arch/arm/lib/getuser.S
index df73914e81c8..d47b4776d308 100644
--- a/arch/arm/lib/getuser.S
+++ b/arch/arm/lib/getuser.S
@@ -133,17 +133,17 @@ ENDPROC(__get_user_bad)
ENDPROC(__get_user_bad8)
.pushsection __ex_table, "a"
- .long 1b, __get_user_bad
- .long 2b, __get_user_bad
- .long 3b, __get_user_bad
- .long 4b, __get_user_bad
- .long 5b, __get_user_bad8
- .long 6b, __get_user_bad8
+ .long 1b - ., __get_user_bad - .
+ .long 2b - ., __get_user_bad - .
+ .long 3b - ., __get_user_bad - .
+ .long 4b - ., __get_user_bad - .
+ .long 5b - ., __get_user_bad8 - .
+ .long 6b - ., __get_user_bad8 - .
#ifdef __ARMEB__
- .long 7b, __get_user_bad
- .long 8b, __get_user_bad8
- .long 9b, __get_user_bad8
- .long 10b, __get_user_bad8
- .long 11b, __get_user_bad8
+ .long 7b - ., __get_user_bad - .
+ .long 8b - ., __get_user_bad8 - .
+ .long 9b - ., __get_user_bad8 - .
+ .long 10b - ., __get_user_bad8 - .
+ .long 11b - ., __get_user_bad8 - .
#endif
.popsection
diff --git a/arch/arm/lib/putuser.S b/arch/arm/lib/putuser.S
index 38d660d3705f..6b854197ff48 100644
--- a/arch/arm/lib/putuser.S
+++ b/arch/arm/lib/putuser.S
@@ -89,10 +89,10 @@ __put_user_bad:
ENDPROC(__put_user_bad)
.pushsection __ex_table, "a"
- .long 1b, __put_user_bad
- .long 2b, __put_user_bad
- .long 3b, __put_user_bad
- .long 4b, __put_user_bad
- .long 5b, __put_user_bad
- .long 6b, __put_user_bad
+ .long 1b - ., __put_user_bad - .
+ .long 2b - ., __put_user_bad - .
+ .long 3b - ., __put_user_bad - .
+ .long 4b - ., __put_user_bad - .
+ .long 5b - ., __put_user_bad - .
+ .long 6b - ., __put_user_bad - .
.popsection
diff --git a/arch/arm/mm/alignment.c b/arch/arm/mm/alignment.c
index 2c96190e018b..2c74cb826b6a 100644
--- a/arch/arm/mm/alignment.c
+++ b/arch/arm/mm/alignment.c
@@ -209,7 +209,7 @@ union offset_union {
" .popsection\n" \
" .pushsection __ex_table,\"a\"\n" \
" .align 3\n" \
- " .long 1b, 3b\n" \
+ " .long 1b - ., 3b - .\n" \
" .popsection\n" \
: "=r" (err), "=&r" (val), "=r" (addr) \
: "0" (err), "2" (addr))
@@ -269,8 +269,8 @@ union offset_union {
" .popsection\n" \
" .pushsection __ex_table,\"a\"\n" \
" .align 3\n" \
- " .long 1b, 4b\n" \
- " .long 2b, 4b\n" \
+ " .long 1b - ., 4b - .\n" \
+ " .long 2b - ., 4b - .\n" \
" .popsection\n" \
: "=r" (err), "=&r" (v), "=&r" (a) \
: "0" (err), "1" (v), "2" (a)); \
@@ -309,10 +309,10 @@ union offset_union {
" .popsection\n" \
" .pushsection __ex_table,\"a\"\n" \
" .align 3\n" \
- " .long 1b, 6b\n" \
- " .long 2b, 6b\n" \
- " .long 3b, 6b\n" \
- " .long 4b, 6b\n" \
+ " .long 1b - ., 6b - .\n" \
+ " .long 2b - ., 6b - .\n" \
+ " .long 3b - ., 6b - .\n" \
+ " .long 4b - ., 6b - .\n" \
" .popsection\n" \
: "=r" (err), "=&r" (v), "=&r" (a) \
: "0" (err), "1" (v), "2" (a)); \
diff --git a/arch/arm/mm/extable.c b/arch/arm/mm/extable.c
index f436f7439e46..6a8b85bf0cac 100644
--- a/arch/arm/mm/extable.c
+++ b/arch/arm/mm/extable.c
@@ -10,7 +10,7 @@ int fixup_exception(struct pt_regs *regs)
fixup = search_exception_tables(instruction_pointer(regs));
if (fixup) {
- regs->ARM_pc = fixup->fixup;
+ regs->ARM_pc = (unsigned long)&fixup->fixup + fixup->fixup;
#ifdef CONFIG_THUMB2_KERNEL
/* Clear the IT state to avoid nasty surprises in the fixup */
regs->ARM_cpsr &= ~PSR_IT_MASK;
diff --git a/arch/arm/nwfpe/entry.S b/arch/arm/nwfpe/entry.S
index 39c20afad7ed..c4fe13b31705 100644
--- a/arch/arm/nwfpe/entry.S
+++ b/arch/arm/nwfpe/entry.S
@@ -121,5 +121,5 @@ next:
.pushsection __ex_table,"a"
.align 3
- .long .Lx1, .Lfix
+ .long .Lx1 - ., .Lfix - .
.popsection
diff --git a/scripts/sortextable.c b/scripts/sortextable.c
index 365a907f98b3..56a4c6714da7 100644
--- a/scripts/sortextable.c
+++ b/scripts/sortextable.c
@@ -314,6 +314,7 @@ do_file(char const *const fname)
break;
case EM_S390:
+ case EM_ARM:
case EM_AARCH64:
case EM_PARISC:
case EM_PPC:
@@ -322,7 +323,6 @@ do_file(char const *const fname)
break;
case EM_ARCOMPACT:
case EM_ARCV2:
- case EM_ARM:
case EM_MICROBLAZE:
case EM_MIPS:
case EM_XTENSA:
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 16/29] ARM: kernel: switch to relative exception tables
2017-09-03 12:07 ` [PATCH v2 16/29] ARM: kernel: switch to relative exception tables Ard Biesheuvel
@ 2017-09-04 17:17 ` Nicolas Pitre
2017-09-04 17:30 ` Ard Biesheuvel
0 siblings, 1 reply; 83+ messages in thread
From: Nicolas Pitre @ 2017-09-04 17:17 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
> To avoid having to relocate the contents of extable entries at
> runtime when running with KASLR enabled, wire up the existing
> support for emitting them as relative references. This ensures
> these quantities are invariant under runtime relocation.
>
> Cc: Russell King <linux@armlinux.org.uk>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Could it be better to have a macro that encapsulates exception entries
e.g.:
.macro ex_entry insn fixup
.long \insn - ., \fixup - .
.endif
so that if ever this changes again or for whatever reason someone needs
absolute entries then this won't have to be done everywhere. The section
switch could be included in such a macro too. And it is so easy to
forget to do the "- ." when adding new entries.
> ---
> arch/arm/include/asm/Kbuild | 1 -
> arch/arm/include/asm/assembler.h | 6 +++---
> arch/arm/include/asm/extable.h | 19 +++++++++++++++++
> arch/arm/include/asm/futex.h | 2 +-
> arch/arm/include/asm/uaccess.h | 8 +++----
> arch/arm/include/asm/word-at-a-time.h | 2 +-
> arch/arm/kernel/entry-armv.S | 6 +++---
> arch/arm/kernel/swp_emulate.c | 4 ++--
> arch/arm/lib/backtrace.S | 8 +++----
> arch/arm/lib/getuser.S | 22 ++++++++++----------
> arch/arm/lib/putuser.S | 12 +++++------
> arch/arm/mm/alignment.c | 14 ++++++-------
> arch/arm/mm/extable.c | 2 +-
> arch/arm/nwfpe/entry.S | 2 +-
> scripts/sortextable.c | 2 +-
> 15 files changed, 64 insertions(+), 46 deletions(-)
>
> diff --git a/arch/arm/include/asm/Kbuild b/arch/arm/include/asm/Kbuild
> index 721ab5ecfb9b..f2d67fa34eb3 100644
> --- a/arch/arm/include/asm/Kbuild
> +++ b/arch/arm/include/asm/Kbuild
> @@ -3,7 +3,6 @@ generic-y += current.h
> generic-y += early_ioremap.h
> generic-y += emergency-restart.h
> generic-y += exec.h
> -generic-y += extable.h
> generic-y += irq_regs.h
> generic-y += kdebug.h
> generic-y += local.h
> diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h
> index a0906c1fc65d..ae43859a9000 100644
> --- a/arch/arm/include/asm/assembler.h
> +++ b/arch/arm/include/asm/assembler.h
> @@ -247,7 +247,7 @@
> 9999: x; \
> .pushsection __ex_table,"a"; \
> .align 3; \
> - .long 9999b,9001f; \
> + .long 9999b - ., 9001f - .; \
> .popsection
>
> #ifdef CONFIG_SMP
> @@ -383,7 +383,7 @@ THUMB( orr \reg , \reg , #PSR_T_BIT )
>
> .pushsection __ex_table,"a"
> .align 3
> - .long 9999b, \abort
> + .long 9999b - ., \abort - .
> .popsection
> .endm
>
> @@ -424,7 +424,7 @@ THUMB( orr \reg , \reg , #PSR_T_BIT )
>
> .pushsection __ex_table,"a"
> .align 3
> - .long 9999b, \abort
> + .long 9999b - ., \abort - .
> .popsection
> .endr
> .endm
> diff --git a/arch/arm/include/asm/extable.h b/arch/arm/include/asm/extable.h
> new file mode 100644
> index 000000000000..c85d1f4404b7
> --- /dev/null
> +++ b/arch/arm/include/asm/extable.h
> @@ -0,0 +1,19 @@
> +#ifndef __ASM_EXTABLE_H
> +#define __ASM_EXTABLE_H
> +
> +/*
> + * The exception table consists of pairs of relative offsets: the first
> + * is the relative offset to an instruction that is allowed to fault,
> + * and the second is the relative offset at which the program should
> + * continue. No registers are modified, so it is entirely up to the
> + * continuation code to figure out what to do.
> + */
> +
> +struct exception_table_entry {
> + int insn, fixup;
> +};
> +
> +#define ARCH_HAS_RELATIVE_EXTABLE
> +
> +extern int fixup_exception(struct pt_regs *regs);
> +#endif
> diff --git a/arch/arm/include/asm/futex.h b/arch/arm/include/asm/futex.h
> index 6795368ad023..31760d0ab76b 100644
> --- a/arch/arm/include/asm/futex.h
> +++ b/arch/arm/include/asm/futex.h
> @@ -11,7 +11,7 @@
> "3:\n" \
> " .pushsection __ex_table,\"a\"\n" \
> " .align 3\n" \
> - " .long 1b, 4f, 2b, 4f\n" \
> + " .long 1b - ., 4f - ., 2b - ., 4f - .\n" \
> " .popsection\n" \
> " .pushsection .text.fixup,\"ax\"\n" \
> " .align 2\n" \
> diff --git a/arch/arm/include/asm/uaccess.h b/arch/arm/include/asm/uaccess.h
> index 0bf2347495f1..62b9bb7d991a 100644
> --- a/arch/arm/include/asm/uaccess.h
> +++ b/arch/arm/include/asm/uaccess.h
> @@ -295,7 +295,7 @@ do { \
> " .popsection\n" \
> " .pushsection __ex_table,\"a\"\n" \
> " .align 3\n" \
> - " .long 1b, 3b\n" \
> + " .long 1b - ., 3b - .\n" \
> " .popsection" \
> : "+r" (err), "=&r" (x) \
> : "r" (addr), "i" (-EFAULT) \
> @@ -385,7 +385,7 @@ do { \
> " .popsection\n" \
> " .pushsection __ex_table,\"a\"\n" \
> " .align 3\n" \
> - " .long 1b, 3b\n" \
> + " .long 1b - ., 3b - .\n" \
> " .popsection" \
> : "+r" (err) \
> : "r" (x), "r" (__pu_addr), "i" (-EFAULT) \
> @@ -435,8 +435,8 @@ do { \
> " .popsection\n" \
> " .pushsection __ex_table,\"a\"\n" \
> " .align 3\n" \
> - " .long 1b, 4b\n" \
> - " .long 2b, 4b\n" \
> + " .long 1b - ., 4b - .\n" \
> + " .long 2b - ., 4b - .\n" \
> " .popsection" \
> : "+r" (err), "+r" (__pu_addr) \
> : "r" (x), "i" (-EFAULT) \
> diff --git a/arch/arm/include/asm/word-at-a-time.h b/arch/arm/include/asm/word-at-a-time.h
> index 5831dce4b51c..d433c686d9ca 100644
> --- a/arch/arm/include/asm/word-at-a-time.h
> +++ b/arch/arm/include/asm/word-at-a-time.h
> @@ -86,7 +86,7 @@ static inline unsigned long load_unaligned_zeropad(const void *addr)
> " .popsection\n"
> " .pushsection __ex_table,\"a\"\n"
> " .align 3\n"
> - " .long 1b, 3b\n"
> + " .long 1b - ., 3b - .\n"
> " .popsection"
> : "=&r" (ret), "=&r" (offset)
> : "r" (addr), "Qo" (*(unsigned long *)addr));
> diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S
> index c731f0d2b2af..1c7c12123b18 100644
> --- a/arch/arm/kernel/entry-armv.S
> +++ b/arch/arm/kernel/entry-armv.S
> @@ -570,10 +570,10 @@ ENDPROC(__und_usr)
> ret r9
> .popsection
> .pushsection __ex_table,"a"
> - .long 1b, 4b
> + .long 1b - ., 4b - .
> #if CONFIG_ARM_THUMB && __LINUX_ARM_ARCH__ >= 6 && CONFIG_CPU_V7
> - .long 2b, 4b
> - .long 3b, 4b
> + .long 2b - ., 4b - .
> + .long 3b - ., 4b - .
> #endif
> .popsection
>
> diff --git a/arch/arm/kernel/swp_emulate.c b/arch/arm/kernel/swp_emulate.c
> index 3bda08bee674..a783e9c0b266 100644
> --- a/arch/arm/kernel/swp_emulate.c
> +++ b/arch/arm/kernel/swp_emulate.c
> @@ -50,8 +50,8 @@
> " .previous\n" \
> " .section __ex_table,\"a\"\n" \
> " .align 3\n" \
> - " .long 0b, 3b\n" \
> - " .long 1b, 3b\n" \
> + " .long 0b - ., 3b - .\n" \
> + " .long 1b - ., 3b - .\n" \
> " .previous" \
> : "=&r" (res), "+r" (data), "=&r" (temp) \
> : "r" (addr), "i" (-EAGAIN), "i" (-EFAULT) \
> diff --git a/arch/arm/lib/backtrace.S b/arch/arm/lib/backtrace.S
> index 7d7952e5a3b1..84a8df7aa63c 100644
> --- a/arch/arm/lib/backtrace.S
> +++ b/arch/arm/lib/backtrace.S
> @@ -107,10 +107,10 @@ ENDPROC(c_backtrace)
>
> .pushsection __ex_table,"a"
> .align 3
> - .long 1001b, 1006b
> - .long 1002b, 1006b
> - .long 1003b, 1006b
> - .long 1004b, 1006b
> + .long 1001b - ., 1006b - .
> + .long 1002b - ., 1006b - .
> + .long 1003b - ., 1006b - .
> + .long 1004b - ., 1006b - .
> .popsection
>
> .Lbad: .asciz "Backtrace aborted due to bad frame pointer <%p>\n"
> diff --git a/arch/arm/lib/getuser.S b/arch/arm/lib/getuser.S
> index df73914e81c8..d47b4776d308 100644
> --- a/arch/arm/lib/getuser.S
> +++ b/arch/arm/lib/getuser.S
> @@ -133,17 +133,17 @@ ENDPROC(__get_user_bad)
> ENDPROC(__get_user_bad8)
>
> .pushsection __ex_table, "a"
> - .long 1b, __get_user_bad
> - .long 2b, __get_user_bad
> - .long 3b, __get_user_bad
> - .long 4b, __get_user_bad
> - .long 5b, __get_user_bad8
> - .long 6b, __get_user_bad8
> + .long 1b - ., __get_user_bad - .
> + .long 2b - ., __get_user_bad - .
> + .long 3b - ., __get_user_bad - .
> + .long 4b - ., __get_user_bad - .
> + .long 5b - ., __get_user_bad8 - .
> + .long 6b - ., __get_user_bad8 - .
> #ifdef __ARMEB__
> - .long 7b, __get_user_bad
> - .long 8b, __get_user_bad8
> - .long 9b, __get_user_bad8
> - .long 10b, __get_user_bad8
> - .long 11b, __get_user_bad8
> + .long 7b - ., __get_user_bad - .
> + .long 8b - ., __get_user_bad8 - .
> + .long 9b - ., __get_user_bad8 - .
> + .long 10b - ., __get_user_bad8 - .
> + .long 11b - ., __get_user_bad8 - .
> #endif
> .popsection
> diff --git a/arch/arm/lib/putuser.S b/arch/arm/lib/putuser.S
> index 38d660d3705f..6b854197ff48 100644
> --- a/arch/arm/lib/putuser.S
> +++ b/arch/arm/lib/putuser.S
> @@ -89,10 +89,10 @@ __put_user_bad:
> ENDPROC(__put_user_bad)
>
> .pushsection __ex_table, "a"
> - .long 1b, __put_user_bad
> - .long 2b, __put_user_bad
> - .long 3b, __put_user_bad
> - .long 4b, __put_user_bad
> - .long 5b, __put_user_bad
> - .long 6b, __put_user_bad
> + .long 1b - ., __put_user_bad - .
> + .long 2b - ., __put_user_bad - .
> + .long 3b - ., __put_user_bad - .
> + .long 4b - ., __put_user_bad - .
> + .long 5b - ., __put_user_bad - .
> + .long 6b - ., __put_user_bad - .
> .popsection
> diff --git a/arch/arm/mm/alignment.c b/arch/arm/mm/alignment.c
> index 2c96190e018b..2c74cb826b6a 100644
> --- a/arch/arm/mm/alignment.c
> +++ b/arch/arm/mm/alignment.c
> @@ -209,7 +209,7 @@ union offset_union {
> " .popsection\n" \
> " .pushsection __ex_table,\"a\"\n" \
> " .align 3\n" \
> - " .long 1b, 3b\n" \
> + " .long 1b - ., 3b - .\n" \
> " .popsection\n" \
> : "=r" (err), "=&r" (val), "=r" (addr) \
> : "0" (err), "2" (addr))
> @@ -269,8 +269,8 @@ union offset_union {
> " .popsection\n" \
> " .pushsection __ex_table,\"a\"\n" \
> " .align 3\n" \
> - " .long 1b, 4b\n" \
> - " .long 2b, 4b\n" \
> + " .long 1b - ., 4b - .\n" \
> + " .long 2b - ., 4b - .\n" \
> " .popsection\n" \
> : "=r" (err), "=&r" (v), "=&r" (a) \
> : "0" (err), "1" (v), "2" (a)); \
> @@ -309,10 +309,10 @@ union offset_union {
> " .popsection\n" \
> " .pushsection __ex_table,\"a\"\n" \
> " .align 3\n" \
> - " .long 1b, 6b\n" \
> - " .long 2b, 6b\n" \
> - " .long 3b, 6b\n" \
> - " .long 4b, 6b\n" \
> + " .long 1b - ., 6b - .\n" \
> + " .long 2b - ., 6b - .\n" \
> + " .long 3b - ., 6b - .\n" \
> + " .long 4b - ., 6b - .\n" \
> " .popsection\n" \
> : "=r" (err), "=&r" (v), "=&r" (a) \
> : "0" (err), "1" (v), "2" (a)); \
> diff --git a/arch/arm/mm/extable.c b/arch/arm/mm/extable.c
> index f436f7439e46..6a8b85bf0cac 100644
> --- a/arch/arm/mm/extable.c
> +++ b/arch/arm/mm/extable.c
> @@ -10,7 +10,7 @@ int fixup_exception(struct pt_regs *regs)
>
> fixup = search_exception_tables(instruction_pointer(regs));
> if (fixup) {
> - regs->ARM_pc = fixup->fixup;
> + regs->ARM_pc = (unsigned long)&fixup->fixup + fixup->fixup;
> #ifdef CONFIG_THUMB2_KERNEL
> /* Clear the IT state to avoid nasty surprises in the fixup */
> regs->ARM_cpsr &= ~PSR_IT_MASK;
> diff --git a/arch/arm/nwfpe/entry.S b/arch/arm/nwfpe/entry.S
> index 39c20afad7ed..c4fe13b31705 100644
> --- a/arch/arm/nwfpe/entry.S
> +++ b/arch/arm/nwfpe/entry.S
> @@ -121,5 +121,5 @@ next:
>
> .pushsection __ex_table,"a"
> .align 3
> - .long .Lx1, .Lfix
> + .long .Lx1 - ., .Lfix - .
> .popsection
> diff --git a/scripts/sortextable.c b/scripts/sortextable.c
> index 365a907f98b3..56a4c6714da7 100644
> --- a/scripts/sortextable.c
> +++ b/scripts/sortextable.c
> @@ -314,6 +314,7 @@ do_file(char const *const fname)
> break;
>
> case EM_S390:
> + case EM_ARM:
> case EM_AARCH64:
> case EM_PARISC:
> case EM_PPC:
> @@ -322,7 +323,6 @@ do_file(char const *const fname)
> break;
> case EM_ARCOMPACT:
> case EM_ARCV2:
> - case EM_ARM:
> case EM_MICROBLAZE:
> case EM_MIPS:
> case EM_XTENSA:
> --
> 2.11.0
>
>
^ permalink raw reply [flat|nested] 83+ messages in thread* [PATCH v2 16/29] ARM: kernel: switch to relative exception tables
2017-09-04 17:17 ` Nicolas Pitre
@ 2017-09-04 17:30 ` Ard Biesheuvel
0 siblings, 0 replies; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-04 17:30 UTC (permalink / raw)
To: linux-arm-kernel
On 4 September 2017 at 18:17, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
>
>> To avoid having to relocate the contents of extable entries at
>> runtime when running with KASLR enabled, wire up the existing
>> support for emitting them as relative references. This ensures
>> these quantities are invariant under runtime relocation.
>>
>> Cc: Russell King <linux@armlinux.org.uk>
>> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>
> Could it be better to have a macro that encapsulates exception entries
> e.g.:
>
> .macro ex_entry insn fixup
> .long \insn - ., \fixup - .
> .endif
>
> so that if ever this changes again or for whatever reason someone needs
> absolute entries then this won't have to be done everywhere. The section
> switch could be included in such a macro too. And it is so easy to
> forget to do the "- ." when adding new entries.
>
Good point. I will change that.
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 17/29] ARM: kernel: use relative phys-to-virt patch tables
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (15 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 16/29] ARM: kernel: switch to relative exception tables Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-04 18:03 ` Nicolas Pitre
2017-09-03 12:07 ` [PATCH v2 18/29] arm-soc: tegra: make sleep asm code runtime relocatable Ard Biesheuvel
` (12 subsequent siblings)
29 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
Replace the contents of the __pv_table entries with relative references
so that we don't have to relocate them at runtime when running the KASLR
kernel. This ensures these quantities are invariant under runtime
relocation, which makes any cache maintenance after runtime relocation
unnecessary.
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/include/asm/memory.h | 6 +++---
arch/arm/kernel/head.S | 21 ++++++++++----------
2 files changed, 13 insertions(+), 14 deletions(-)
diff --git a/arch/arm/include/asm/memory.h b/arch/arm/include/asm/memory.h
index 1f54e4e98c1e..47a984e3a244 100644
--- a/arch/arm/include/asm/memory.h
+++ b/arch/arm/include/asm/memory.h
@@ -195,7 +195,7 @@ extern const void *__pv_table_begin, *__pv_table_end;
__asm__("@ __pv_stub\n" \
"1: " instr " %0, %1, %2\n" \
" .pushsection .pv_table,\"a\"\n" \
- " .long 1b\n" \
+ " .long 1b - .\n" \
" .popsection\n" \
: "=r" (to) \
: "r" (from), "I" (type))
@@ -204,7 +204,7 @@ extern const void *__pv_table_begin, *__pv_table_end;
__asm__ volatile("@ __pv_stub_mov\n" \
"1: mov %R0, %1\n" \
" .pushsection .pv_table,\"a\"\n" \
- " .long 1b\n" \
+ " .long 1b - .\n" \
" .popsection\n" \
: "=r" (t) \
: "I" (__PV_BITS_7_0))
@@ -214,7 +214,7 @@ extern const void *__pv_table_begin, *__pv_table_end;
"1: adds %Q0, %1, %2\n" \
" adc %R0, %R0, #0\n" \
" .pushsection .pv_table,\"a\"\n" \
- " .long 1b\n" \
+ " .long 1b - .\n" \
" .popsection\n" \
: "+r" (y) \
: "r" (x), "I" (__PV_BITS_31_24) \
diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
index 62c961849035..5d685e86148c 100644
--- a/arch/arm/kernel/head.S
+++ b/arch/arm/kernel/head.S
@@ -593,8 +593,7 @@ ENDPROC(__fixup_pv_table)
.text
__fixup_a_pv_table:
- mov_l r6, __pv_offset
- add r6, r6, r3
+ adr_l r6, __pv_offset
ldr r0, [r6, #HIGH_OFFSET] @ pv_offset high word
ldr r6, [r6, #LOW_OFFSET] @ pv_offset low word
mov r6, r6, lsr #24
@@ -612,22 +611,22 @@ __fixup_a_pv_table:
orr r6, r6, r7, lsl #12
orr r6, #0x4000
b 2f
-1: add r7, r3
- ldrh ip, [r7, #2]
+1: add r7, r4
+ ldrh ip, [r7, #-2]
ARM_BE8(rev16 ip, ip)
tst ip, #0x4000
and ip, #0x8f00
orrne ip, r6 @ mask in offset bits 31-24
orreq ip, r0 @ mask in offset bits 7-0
ARM_BE8(rev16 ip, ip)
- strh ip, [r7, #2]
+ strh ip, [r7, #-2]
bne 2f
- ldrh ip, [r7]
+ ldrh ip, [r7, #-4]
ARM_BE8(rev16 ip, ip)
bic ip, #0x20
orr ip, ip, r0, lsr #16
ARM_BE8(rev16 ip, ip)
- strh ip, [r7]
+ strh ip, [r7, #-4]
2: cmp r4, r5
ldrcc r7, [r4], #4 @ use branch for delay slot
bcc 1b
@@ -639,7 +638,8 @@ ARM_BE8(rev16 ip, ip)
moveq r0, #0x400000 @ set bit 22, mov to mvn instruction
#endif
b 2f
-1: ldr ip, [r7, r3]
+1: ldr ip, [r7, r4]!
+ add r4, r4, #4
#ifdef CONFIG_CPU_ENDIAN_BE8
@ in BE8, we load data in BE, but instructions still in LE
bic ip, ip, #0xff000000
@@ -654,9 +654,9 @@ ARM_BE8(rev16 ip, ip)
biceq ip, ip, #0x400000 @ clear bit 22
orreq ip, ip, r0 @ mask in offset bits 7-0
#endif
- str ip, [r7, r3]
+ str ip, [r7]
2: cmp r4, r5
- ldrcc r7, [r4], #4 @ use branch for delay slot
+ ldrcc r7, [r4] @ use branch for delay slot
bcc 1b
ret lr
#endif
@@ -664,7 +664,6 @@ ENDPROC(__fixup_a_pv_table)
ENTRY(fixup_pv_table)
stmfd sp!, {r4 - r7, lr}
- mov r3, #0 @ no offset
mov r4, r0 @ r0 = table start
add r5, r0, r1 @ r1 = table size
bl __fixup_a_pv_table
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 17/29] ARM: kernel: use relative phys-to-virt patch tables
2017-09-03 12:07 ` [PATCH v2 17/29] ARM: kernel: use relative phys-to-virt patch tables Ard Biesheuvel
@ 2017-09-04 18:03 ` Nicolas Pitre
2017-09-04 19:09 ` Ard Biesheuvel
0 siblings, 1 reply; 83+ messages in thread
From: Nicolas Pitre @ 2017-09-04 18:03 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
> Replace the contents of the __pv_table entries with relative references
> so that we don't have to relocate them at runtime when running the KASLR
> kernel. This ensures these quantities are invariant under runtime
> relocation, which makes any cache maintenance after runtime relocation
> unnecessary.
>
> Cc: Russell King <linux@armlinux.org.uk>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> ---
> arch/arm/include/asm/memory.h | 6 +++---
> arch/arm/kernel/head.S | 21 ++++++++++----------
> 2 files changed, 13 insertions(+), 14 deletions(-)
>
> diff --git a/arch/arm/include/asm/memory.h b/arch/arm/include/asm/memory.h
> index 1f54e4e98c1e..47a984e3a244 100644
> --- a/arch/arm/include/asm/memory.h
> +++ b/arch/arm/include/asm/memory.h
> @@ -195,7 +195,7 @@ extern const void *__pv_table_begin, *__pv_table_end;
> __asm__("@ __pv_stub\n" \
> "1: " instr " %0, %1, %2\n" \
> " .pushsection .pv_table,\"a\"\n" \
> - " .long 1b\n" \
> + " .long 1b - .\n" \
> " .popsection\n" \
[...]
> --- a/arch/arm/kernel/head.S
> +++ b/arch/arm/kernel/head.S
> @@ -612,22 +611,22 @@ __fixup_a_pv_table:
> orr r6, r6, r7, lsl #12
> orr r6, #0x4000
> b 2f
> -1: add r7, r3
> - ldrh ip, [r7, #2]
> +1: add r7, r4
> + ldrh ip, [r7, #-2]
Please use "ldrh ip, [r7, #(-4 + 2)]" so the -4 offset is clearly
spelled out. The #-2 gave me a pause of confusion.
> ARM_BE8(rev16 ip, ip)
> tst ip, #0x4000
> and ip, #0x8f00
> orrne ip, r6 @ mask in offset bits 31-24
> orreq ip, r0 @ mask in offset bits 7-0
> ARM_BE8(rev16 ip, ip)
> - strh ip, [r7, #2]
> + strh ip, [r7, #-2]
Ditto here.
> bne 2f
> - ldrh ip, [r7]
> + ldrh ip, [r7, #-4]
And to be clear, here I'd use "#(-4 + 0)" to be explicit.
> ARM_BE8(rev16 ip, ip)
> bic ip, #0x20
> orr ip, ip, r0, lsr #16
> ARM_BE8(rev16 ip, ip)
> - strh ip, [r7]
> + strh ip, [r7, #-4]
And so here.
With that:
Reviewed-by: Nicolas Pitre <nico@linaro.org>
> 2: cmp r4, r5
> ldrcc r7, [r4], #4 @ use branch for delay slot
> bcc 1b
> @@ -639,7 +638,8 @@ ARM_BE8(rev16 ip, ip)
> moveq r0, #0x400000 @ set bit 22, mov to mvn instruction
> #endif
> b 2f
> -1: ldr ip, [r7, r3]
> +1: ldr ip, [r7, r4]!
> + add r4, r4, #4
> #ifdef CONFIG_CPU_ENDIAN_BE8
> @ in BE8, we load data in BE, but instructions still in LE
> bic ip, ip, #0xff000000
> @@ -654,9 +654,9 @@ ARM_BE8(rev16 ip, ip)
> biceq ip, ip, #0x400000 @ clear bit 22
> orreq ip, ip, r0 @ mask in offset bits 7-0
> #endif
> - str ip, [r7, r3]
> + str ip, [r7]
> 2: cmp r4, r5
> - ldrcc r7, [r4], #4 @ use branch for delay slot
> + ldrcc r7, [r4] @ use branch for delay slot
> bcc 1b
> ret lr
> #endif
> @@ -664,7 +664,6 @@ ENDPROC(__fixup_a_pv_table)
>
> ENTRY(fixup_pv_table)
> stmfd sp!, {r4 - r7, lr}
> - mov r3, #0 @ no offset
> mov r4, r0 @ r0 = table start
> add r5, r0, r1 @ r1 = table size
> bl __fixup_a_pv_table
> --
> 2.11.0
>
>
^ permalink raw reply [flat|nested] 83+ messages in thread* [PATCH v2 17/29] ARM: kernel: use relative phys-to-virt patch tables
2017-09-04 18:03 ` Nicolas Pitre
@ 2017-09-04 19:09 ` Ard Biesheuvel
0 siblings, 0 replies; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-04 19:09 UTC (permalink / raw)
To: linux-arm-kernel
On 4 September 2017 at 19:03, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
>
>> Replace the contents of the __pv_table entries with relative references
>> so that we don't have to relocate them at runtime when running the KASLR
>> kernel. This ensures these quantities are invariant under runtime
>> relocation, which makes any cache maintenance after runtime relocation
>> unnecessary.
>>
>> Cc: Russell King <linux@armlinux.org.uk>
>> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>> ---
>> arch/arm/include/asm/memory.h | 6 +++---
>> arch/arm/kernel/head.S | 21 ++++++++++----------
>> 2 files changed, 13 insertions(+), 14 deletions(-)
>>
>> diff --git a/arch/arm/include/asm/memory.h b/arch/arm/include/asm/memory.h
>> index 1f54e4e98c1e..47a984e3a244 100644
>> --- a/arch/arm/include/asm/memory.h
>> +++ b/arch/arm/include/asm/memory.h
>> @@ -195,7 +195,7 @@ extern const void *__pv_table_begin, *__pv_table_end;
>> __asm__("@ __pv_stub\n" \
>> "1: " instr " %0, %1, %2\n" \
>> " .pushsection .pv_table,\"a\"\n" \
>> - " .long 1b\n" \
>> + " .long 1b - .\n" \
>> " .popsection\n" \
>
> [...]
>
>> --- a/arch/arm/kernel/head.S
>> +++ b/arch/arm/kernel/head.S
>> @@ -612,22 +611,22 @@ __fixup_a_pv_table:
>> orr r6, r6, r7, lsl #12
>> orr r6, #0x4000
>> b 2f
>> -1: add r7, r3
>> - ldrh ip, [r7, #2]
>> +1: add r7, r4
>> + ldrh ip, [r7, #-2]
>
> Please use "ldrh ip, [r7, #(-4 + 2)]" so the -4 offset is clearly
> spelled out. The #-2 gave me a pause of confusion.
>
>> ARM_BE8(rev16 ip, ip)
>> tst ip, #0x4000
>> and ip, #0x8f00
>> orrne ip, r6 @ mask in offset bits 31-24
>> orreq ip, r0 @ mask in offset bits 7-0
>> ARM_BE8(rev16 ip, ip)
>> - strh ip, [r7, #2]
>> + strh ip, [r7, #-2]
>
> Ditto here.
>
>> bne 2f
>> - ldrh ip, [r7]
>> + ldrh ip, [r7, #-4]
>
> And to be clear, here I'd use "#(-4 + 0)" to be explicit.
>
>> ARM_BE8(rev16 ip, ip)
>> bic ip, #0x20
>> orr ip, ip, r0, lsr #16
>> ARM_BE8(rev16 ip, ip)
>> - strh ip, [r7]
>> + strh ip, [r7, #-4]
>
> And so here.
>
OK
>
> With that:
>
> Reviewed-by: Nicolas Pitre <nico@linaro.org>
Thanks.
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 18/29] arm-soc: tegra: make sleep asm code runtime relocatable
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (16 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 17/29] ARM: kernel: use relative phys-to-virt patch tables Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-03 12:07 ` [PATCH v2 19/29] ARM: kernel: make vmlinux buildable as a PIE executable Ard Biesheuvel
` (11 subsequent siblings)
29 siblings, 0 replies; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
The PIE kernel build does not allow absolute references encoded in
movw/movt instruction pairs, so use our mov_l macro instead (which
will still use such a pair unless CONFIG_RELOCATABLE is defined)
Also, avoid 32-bit absolute literals to refer to absolute symbols.
Instead, use a 16 bit reference so that PIE linker cannot get
confused whether the symbol reference is subject to relocation at
runtime.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/mach-tegra/sleep-tegra20.S | 22 ++++++++++++--------
arch/arm/mach-tegra/sleep-tegra30.S | 6 +++---
arch/arm/mach-tegra/sleep.S | 4 ++--
3 files changed, 18 insertions(+), 14 deletions(-)
diff --git a/arch/arm/mach-tegra/sleep-tegra20.S b/arch/arm/mach-tegra/sleep-tegra20.S
index 5c8e638ee51a..cab95de5c8f1 100644
--- a/arch/arm/mach-tegra/sleep-tegra20.S
+++ b/arch/arm/mach-tegra/sleep-tegra20.S
@@ -99,7 +99,7 @@ ENTRY(tegra20_cpu_shutdown)
cmp r0, #0
reteq lr @ must not be called for CPU 0
mov32 r1, TEGRA_IRAM_RESET_BASE_VIRT
- ldr r2, =__tegra20_cpu1_resettable_status_offset
+ ldrh r2, 0f
mov r12, #CPU_RESETTABLE
strb r12, [r1, r2]
@@ -121,6 +121,7 @@ ENTRY(tegra20_cpu_shutdown)
beq .
ret lr
ENDPROC(tegra20_cpu_shutdown)
+0: .short __tegra20_cpu1_resettable_status_offset
#endif
#ifdef CONFIG_PM_SLEEP
@@ -181,6 +182,9 @@ ENTRY(tegra_pen_unlock)
ret lr
ENDPROC(tegra_pen_unlock)
+.L__tegra20_cpu1_resettable_status_offset:
+ .short __tegra20_cpu1_resettable_status_offset
+
/*
* tegra20_cpu_clear_resettable(void)
*
@@ -189,7 +193,7 @@ ENDPROC(tegra_pen_unlock)
*/
ENTRY(tegra20_cpu_clear_resettable)
mov32 r1, TEGRA_IRAM_RESET_BASE_VIRT
- ldr r2, =__tegra20_cpu1_resettable_status_offset
+ ldrh r2, .L__tegra20_cpu1_resettable_status_offset
mov r12, #CPU_NOT_RESETTABLE
strb r12, [r1, r2]
ret lr
@@ -203,7 +207,7 @@ ENDPROC(tegra20_cpu_clear_resettable)
*/
ENTRY(tegra20_cpu_set_resettable_soon)
mov32 r1, TEGRA_IRAM_RESET_BASE_VIRT
- ldr r2, =__tegra20_cpu1_resettable_status_offset
+ ldrh r2, .L__tegra20_cpu1_resettable_status_offset
mov r12, #CPU_RESETTABLE_SOON
strb r12, [r1, r2]
ret lr
@@ -217,7 +221,7 @@ ENDPROC(tegra20_cpu_set_resettable_soon)
*/
ENTRY(tegra20_cpu_is_resettable_soon)
mov32 r1, TEGRA_IRAM_RESET_BASE_VIRT
- ldr r2, =__tegra20_cpu1_resettable_status_offset
+ ldrh r2, .L__tegra20_cpu1_resettable_status_offset
ldrb r12, [r1, r2]
cmp r12, #CPU_RESETTABLE_SOON
moveq r0, #1
@@ -238,11 +242,11 @@ ENTRY(tegra20_sleep_core_finish)
bl tegra_disable_clean_inv_dcache
mov r0, r4
- mov32 r3, tegra_shut_off_mmu
+ mov_l r3, tegra_shut_off_mmu
add r3, r3, r0
- mov32 r0, tegra20_tear_down_core
- mov32 r1, tegra20_iram_start
+ mov_l r0, tegra20_tear_down_core
+ mov_l r1, tegra20_iram_start
sub r0, r0, r1
mov32 r1, TEGRA_IRAM_LPx_RESUME_AREA
add r0, r0, r1
@@ -265,7 +269,7 @@ ENTRY(tegra20_sleep_cpu_secondary_finish)
bl tegra_disable_clean_inv_dcache
mov32 r0, TEGRA_IRAM_RESET_BASE_VIRT
- ldr r4, =__tegra20_cpu1_resettable_status_offset
+ ldrh r4, .L__tegra20_cpu1_resettable_status_offset
mov r3, #CPU_RESETTABLE
strb r3, [r0, r4]
@@ -284,7 +288,7 @@ ENTRY(tegra20_sleep_cpu_secondary_finish)
bl tegra_pen_lock
mov32 r0, TEGRA_IRAM_RESET_BASE_VIRT
- ldr r4, =__tegra20_cpu1_resettable_status_offset
+ ldrh r4, .L__tegra20_cpu1_resettable_status_offset
mov r3, #CPU_NOT_RESETTABLE
strb r3, [r0, r4]
diff --git a/arch/arm/mach-tegra/sleep-tegra30.S b/arch/arm/mach-tegra/sleep-tegra30.S
index dd4a67dabd91..478b2ca3ef6e 100644
--- a/arch/arm/mach-tegra/sleep-tegra30.S
+++ b/arch/arm/mach-tegra/sleep-tegra30.S
@@ -261,11 +261,11 @@ ENTRY(tegra30_sleep_core_finish)
mov32 r6, TEGRA_FLOW_CTRL_BASE
mov32 r7, TEGRA_TMRUS_BASE
- mov32 r3, tegra_shut_off_mmu
+ mov_l r3, tegra_shut_off_mmu
add r3, r3, r0
- mov32 r0, tegra30_tear_down_core
- mov32 r1, tegra30_iram_start
+ mov_l r0, tegra30_tear_down_core
+ mov_l r1, tegra30_iram_start
sub r0, r0, r1
mov32 r1, TEGRA_IRAM_LPx_RESUME_AREA
add r0, r0, r1
diff --git a/arch/arm/mach-tegra/sleep.S b/arch/arm/mach-tegra/sleep.S
index 5e3496753df1..785df3edc767 100644
--- a/arch/arm/mach-tegra/sleep.S
+++ b/arch/arm/mach-tegra/sleep.S
@@ -101,11 +101,11 @@ ENTRY(tegra_sleep_cpu_finish)
bl tegra_disable_clean_inv_dcache
mov r0, r4
- mov32 r6, tegra_tear_down_cpu
+ mov_l r6, tegra_tear_down_cpu
ldr r1, [r6]
add r1, r1, r0
- mov32 r3, tegra_shut_off_mmu
+ mov_l r3, tegra_shut_off_mmu
add r3, r3, r0
mov r0, r1
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 19/29] ARM: kernel: make vmlinux buildable as a PIE executable
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (17 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 18/29] arm-soc: tegra: make sleep asm code runtime relocatable Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-04 18:11 ` Nicolas Pitre
2017-09-03 12:07 ` [PATCH v2 20/29] ARM: kernel: use PC-relative symbol references in MMU switch code Ard Biesheuvel
` (10 subsequent siblings)
29 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
Update the build flags and linker script to allow vmlinux to be built
as a PIE binary, which retains relocation information about absolute
symbol references so that they can be fixed up at runtime. This will
be used for implementing KASLR,
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/Kconfig | 4 ++++
arch/arm/Makefile | 5 +++++
arch/arm/include/asm/assembler.h | 2 +-
arch/arm/kernel/vmlinux.lds.S | 9 +++++++++
include/linux/hidden.h | 20 ++++++++++++++++++++
scripts/module-common.lds | 1 +
6 files changed, 40 insertions(+), 1 deletion(-)
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 61a0cb15067e..300add3b8023 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -2085,6 +2085,10 @@ config DMI
firmware need to be enabled. This would require the DMI subsystem
to be enabled much earlier than we do on ARM, which is non-trivial.
+config RELOCATABLE
+ bool
+ select HAVE_ARCH_PREL32_RELOCATIONS
+
endmenu
menu "CPU Power Management"
diff --git a/arch/arm/Makefile b/arch/arm/Makefile
index 47d3a1ab08d2..8544c8148b9f 100644
--- a/arch/arm/Makefile
+++ b/arch/arm/Makefile
@@ -52,6 +52,11 @@ AS += -EL
LD += -EL
endif
+ifeq ($(CONFIG_RELOCATABLE),y)
+KBUILD_CFLAGS += -fpic -include $(srctree)/include/linux/hidden.h
+LDFLAGS_vmlinux += -pie -shared -Bsymbolic
+endif
+
#
# The Scalar Replacement of Aggregates (SRA) optimization pass in GCC 4.9 and
# later may result in code being generated that handles signed short and signed
diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h
index ae43859a9000..0586e1c77a80 100644
--- a/arch/arm/include/asm/assembler.h
+++ b/arch/arm/include/asm/assembler.h
@@ -555,7 +555,7 @@ THUMB( orr \reg , \reg , #PSR_T_BIT )
* mov_l - move a constant value or [relocated] address into a register
*/
.macro mov_l, dst:req, imm:req
- .if __LINUX_ARM_ARCH__ < 7
+ .if CONFIG_RELOCATABLE == 1 || __LINUX_ARM_ARCH__ < 7
ldr \dst, =\imm
.else
movw \dst, #:lower16:\imm
diff --git a/arch/arm/kernel/vmlinux.lds.S b/arch/arm/kernel/vmlinux.lds.S
index c83a7ba737d6..5853d4be2067 100644
--- a/arch/arm/kernel/vmlinux.lds.S
+++ b/arch/arm/kernel/vmlinux.lds.S
@@ -89,6 +89,9 @@ SECTIONS
#endif
*(.discard)
*(.discard.*)
+ *(.ARM.exidx.discard.text)
+ *(.interp .dynamic)
+ *(.dynsym .dynstr .hash)
}
. = PAGE_OFFSET + TEXT_OFFSET;
@@ -209,6 +212,12 @@ SECTIONS
__smpalt_end = .;
}
#endif
+ .rel.dyn : ALIGN(8) {
+ __rel_begin = .;
+ *(.rel .rel.* .rel.dyn)
+ }
+ __rel_end = ADDR(.rel.dyn) + SIZEOF(.rel.dyn);
+
.init.pv_table : {
__pv_table_begin = .;
*(.pv_table)
diff --git a/include/linux/hidden.h b/include/linux/hidden.h
new file mode 100644
index 000000000000..781fb6f0c742
--- /dev/null
+++ b/include/linux/hidden.h
@@ -0,0 +1,20 @@
+/*
+ * GCC assumes that we are building shared libraries or hosted binaries
+ * when the -fpic or -fpie switches are used. This results in all references
+ * to symbols with external linkage to be redirected via entries in the global
+ * offset table (GOT), which keeps .text pages clean and reduces the footprint
+ * of CoWed dirty pages to the GOT itself. It also allows symbol preemption,
+ * which is mandatory under ELF rules for shared libraries.
+ *
+ * For the kernel, we use PIC so that we can relocate the executable image at
+ * runtime. This does not involve CoW or symbol preemption, and we'd rather
+ * have relative references instead of absolute ones whenever possible.
+ * So set the default visibility to hidden: this informs the compiler that
+ * none of our symbols will ever be exported from a shared library, allowing
+ * it to use relative references where possible.
+ *
+ * Note that simply passing -fvisibility=hidden is not sufficient to achieve
+ * this: In that case, definitions will be marked hidden, but declarations
+ * will not, and we still end up with GOT entries unnecessarily.
+ */
+#pragma GCC visibility push(hidden)
diff --git a/scripts/module-common.lds b/scripts/module-common.lds
index d61b9e8678e8..55bb41f32c8a 100644
--- a/scripts/module-common.lds
+++ b/scripts/module-common.lds
@@ -7,6 +7,7 @@ SECTIONS {
/DISCARD/ : {
*(.discard)
*(.discard.*)
+ *(*.discard.*)
}
__ksymtab 0 : { *(SORT(___ksymtab+*)) }
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 19/29] ARM: kernel: make vmlinux buildable as a PIE executable
2017-09-03 12:07 ` [PATCH v2 19/29] ARM: kernel: make vmlinux buildable as a PIE executable Ard Biesheuvel
@ 2017-09-04 18:11 ` Nicolas Pitre
2017-09-04 19:10 ` Ard Biesheuvel
0 siblings, 1 reply; 83+ messages in thread
From: Nicolas Pitre @ 2017-09-04 18:11 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
> Update the build flags and linker script to allow vmlinux to be built
> as a PIE binary, which retains relocation information about absolute
> symbol references so that they can be fixed up at runtime. This will
> be used for implementing KASLR,
>
> Cc: Russell King <linux@armlinux.org.uk>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> ---
> arch/arm/Kconfig | 4 ++++
> arch/arm/Makefile | 5 +++++
> arch/arm/include/asm/assembler.h | 2 +-
> arch/arm/kernel/vmlinux.lds.S | 9 +++++++++
> include/linux/hidden.h | 20 ++++++++++++++++++++
> scripts/module-common.lds | 1 +
> 6 files changed, 40 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
> index 61a0cb15067e..300add3b8023 100644
> --- a/arch/arm/Kconfig
> +++ b/arch/arm/Kconfig
> @@ -2085,6 +2085,10 @@ config DMI
> firmware need to be enabled. This would require the DMI subsystem
> to be enabled much earlier than we do on ARM, which is non-trivial.
>
> +config RELOCATABLE
> + bool
> + select HAVE_ARCH_PREL32_RELOCATIONS
> +
You should probably make it depend on !XIP_KERNEL.
Other than that
Acked-by: Nicolas Pitre <nico@linaro.org>
>
> menu "CPU Power Management"
> diff --git a/arch/arm/Makefile b/arch/arm/Makefile
> index 47d3a1ab08d2..8544c8148b9f 100644
> --- a/arch/arm/Makefile
> +++ b/arch/arm/Makefile
> @@ -52,6 +52,11 @@ AS += -EL
> LD += -EL
> endif
>
> +ifeq ($(CONFIG_RELOCATABLE),y)
> +KBUILD_CFLAGS += -fpic -include $(srctree)/include/linux/hidden.h
> +LDFLAGS_vmlinux += -pie -shared -Bsymbolic
> +endif
> +
> #
> # The Scalar Replacement of Aggregates (SRA) optimization pass in GCC 4.9 and
> # later may result in code being generated that handles signed short and signed
> diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h
> index ae43859a9000..0586e1c77a80 100644
> --- a/arch/arm/include/asm/assembler.h
> +++ b/arch/arm/include/asm/assembler.h
> @@ -555,7 +555,7 @@ THUMB( orr \reg , \reg , #PSR_T_BIT )
> * mov_l - move a constant value or [relocated] address into a register
> */
> .macro mov_l, dst:req, imm:req
> - .if __LINUX_ARM_ARCH__ < 7
> + .if CONFIG_RELOCATABLE == 1 || __LINUX_ARM_ARCH__ < 7
> ldr \dst, =\imm
> .else
> movw \dst, #:lower16:\imm
> diff --git a/arch/arm/kernel/vmlinux.lds.S b/arch/arm/kernel/vmlinux.lds.S
> index c83a7ba737d6..5853d4be2067 100644
> --- a/arch/arm/kernel/vmlinux.lds.S
> +++ b/arch/arm/kernel/vmlinux.lds.S
> @@ -89,6 +89,9 @@ SECTIONS
> #endif
> *(.discard)
> *(.discard.*)
> + *(.ARM.exidx.discard.text)
> + *(.interp .dynamic)
> + *(.dynsym .dynstr .hash)
> }
>
> . = PAGE_OFFSET + TEXT_OFFSET;
> @@ -209,6 +212,12 @@ SECTIONS
> __smpalt_end = .;
> }
> #endif
> + .rel.dyn : ALIGN(8) {
> + __rel_begin = .;
> + *(.rel .rel.* .rel.dyn)
> + }
> + __rel_end = ADDR(.rel.dyn) + SIZEOF(.rel.dyn);
> +
> .init.pv_table : {
> __pv_table_begin = .;
> *(.pv_table)
> diff --git a/include/linux/hidden.h b/include/linux/hidden.h
> new file mode 100644
> index 000000000000..781fb6f0c742
> --- /dev/null
> +++ b/include/linux/hidden.h
> @@ -0,0 +1,20 @@
> +/*
> + * GCC assumes that we are building shared libraries or hosted binaries
> + * when the -fpic or -fpie switches are used. This results in all references
> + * to symbols with external linkage to be redirected via entries in the global
> + * offset table (GOT), which keeps .text pages clean and reduces the footprint
> + * of CoWed dirty pages to the GOT itself. It also allows symbol preemption,
> + * which is mandatory under ELF rules for shared libraries.
> + *
> + * For the kernel, we use PIC so that we can relocate the executable image at
> + * runtime. This does not involve CoW or symbol preemption, and we'd rather
> + * have relative references instead of absolute ones whenever possible.
> + * So set the default visibility to hidden: this informs the compiler that
> + * none of our symbols will ever be exported from a shared library, allowing
> + * it to use relative references where possible.
> + *
> + * Note that simply passing -fvisibility=hidden is not sufficient to achieve
> + * this: In that case, definitions will be marked hidden, but declarations
> + * will not, and we still end up with GOT entries unnecessarily.
> + */
> +#pragma GCC visibility push(hidden)
> diff --git a/scripts/module-common.lds b/scripts/module-common.lds
> index d61b9e8678e8..55bb41f32c8a 100644
> --- a/scripts/module-common.lds
> +++ b/scripts/module-common.lds
> @@ -7,6 +7,7 @@ SECTIONS {
> /DISCARD/ : {
> *(.discard)
> *(.discard.*)
> + *(*.discard.*)
> }
>
> __ksymtab 0 : { *(SORT(___ksymtab+*)) }
> --
> 2.11.0
>
>
^ permalink raw reply [flat|nested] 83+ messages in thread* [PATCH v2 19/29] ARM: kernel: make vmlinux buildable as a PIE executable
2017-09-04 18:11 ` Nicolas Pitre
@ 2017-09-04 19:10 ` Ard Biesheuvel
0 siblings, 0 replies; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-04 19:10 UTC (permalink / raw)
To: linux-arm-kernel
On 4 September 2017 at 19:11, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
>
>> Update the build flags and linker script to allow vmlinux to be built
>> as a PIE binary, which retains relocation information about absolute
>> symbol references so that they can be fixed up at runtime. This will
>> be used for implementing KASLR,
>>
>> Cc: Russell King <linux@armlinux.org.uk>
>> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>> ---
>> arch/arm/Kconfig | 4 ++++
>> arch/arm/Makefile | 5 +++++
>> arch/arm/include/asm/assembler.h | 2 +-
>> arch/arm/kernel/vmlinux.lds.S | 9 +++++++++
>> include/linux/hidden.h | 20 ++++++++++++++++++++
>> scripts/module-common.lds | 1 +
>> 6 files changed, 40 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
>> index 61a0cb15067e..300add3b8023 100644
>> --- a/arch/arm/Kconfig
>> +++ b/arch/arm/Kconfig
>> @@ -2085,6 +2085,10 @@ config DMI
>> firmware need to be enabled. This would require the DMI subsystem
>> to be enabled much earlier than we do on ARM, which is non-trivial.
>>
>> +config RELOCATABLE
>> + bool
>> + select HAVE_ARCH_PREL32_RELOCATIONS
>> +
>
> You should probably make it depend on !XIP_KERNEL.
>
Indeed.
> Other than that
>
> Acked-by: Nicolas Pitre <nico@linaro.org>
>
Thanks.
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 20/29] ARM: kernel: use PC-relative symbol references in MMU switch code
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (18 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 19/29] ARM: kernel: make vmlinux buildable as a PIE executable Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-04 18:15 ` Nicolas Pitre
2017-09-03 12:07 ` [PATCH v2 21/29] ARM: kernel: use PC relative symbol references in suspend/resume code Ard Biesheuvel
` (9 subsequent siblings)
29 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
To prepare for adding support for KASLR, which relocates all absolute
symbol references at runtime after the caches have been enabled,
update the MMU switch code to avoid using absolute symbol references
where possible. This ensures these quantities are invariant under
runtime relocation.
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/kernel/head-common.S | 39 ++++++++------------
1 file changed, 15 insertions(+), 24 deletions(-)
diff --git a/arch/arm/kernel/head-common.S b/arch/arm/kernel/head-common.S
index 06035488130c..b74477507a12 100644
--- a/arch/arm/kernel/head-common.S
+++ b/arch/arm/kernel/head-common.S
@@ -79,9 +79,10 @@ ENDPROC(__vet_atags)
*/
__INIT
__mmap_switched:
- adr r3, __mmap_switched_data
-
- ldmia r3!, {r4, r5, r6, r7}
+ adr_l r4, __data_loc
+ adr_l r5, _sdata
+ adr_l r6, __bss_start
+ adr_l r7, _end
cmp r4, r5 @ Copy data segment if needed
1: cmpne r5, r6
ldrne fp, [r4], #4
@@ -93,9 +94,17 @@ __mmap_switched:
strcc fp, [r6],#4
bcc 1b
- ARM( ldmia r3, {r4, r5, r6, r7, sp})
- THUMB( ldmia r3, {r4, r5, r6, r7} )
- THUMB( ldr sp, [r3, #16] )
+ adr_l r3, init_thread_union + THREAD_START_SP
+ mov sp, r3
+ adr_l r4, processor_id
+ adr_l r5, __machine_arch_type
+ adr_l r6, __atags_pointer
+#ifdef CONFIG_CPU_CP15
+ adr_l r7, cr_alignment
+#else
+ mov r7, #0
+#endif
+
str r9, [r4] @ Save processor ID
str r1, [r5] @ Save machine type
str r2, [r6] @ Save atags pointer
@@ -104,24 +113,6 @@ __mmap_switched:
b start_kernel
ENDPROC(__mmap_switched)
- .align 2
- .type __mmap_switched_data, %object
-__mmap_switched_data:
- .long __data_loc @ r4
- .long _sdata @ r5
- .long __bss_start @ r6
- .long _end @ r7
- .long processor_id @ r4
- .long __machine_arch_type @ r5
- .long __atags_pointer @ r6
-#ifdef CONFIG_CPU_CP15
- .long cr_alignment @ r7
-#else
- .long 0 @ r7
-#endif
- .long init_thread_union + THREAD_START_SP @ sp
- .size __mmap_switched_data, . - __mmap_switched_data
-
/*
* This provides a C-API version of __lookup_processor_type
*/
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 20/29] ARM: kernel: use PC-relative symbol references in MMU switch code
2017-09-03 12:07 ` [PATCH v2 20/29] ARM: kernel: use PC-relative symbol references in MMU switch code Ard Biesheuvel
@ 2017-09-04 18:15 ` Nicolas Pitre
2017-09-04 19:14 ` Ard Biesheuvel
0 siblings, 1 reply; 83+ messages in thread
From: Nicolas Pitre @ 2017-09-04 18:15 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
> To prepare for adding support for KASLR, which relocates all absolute
> symbol references at runtime after the caches have been enabled,
> update the MMU switch code to avoid using absolute symbol references
> where possible. This ensures these quantities are invariant under
> runtime relocation.
>
> Cc: Russell King <linux@armlinux.org.uk>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> ---
> arch/arm/kernel/head-common.S | 39 ++++++++------------
> 1 file changed, 15 insertions(+), 24 deletions(-)
>
> diff --git a/arch/arm/kernel/head-common.S b/arch/arm/kernel/head-common.S
> index 06035488130c..b74477507a12 100644
> --- a/arch/arm/kernel/head-common.S
> +++ b/arch/arm/kernel/head-common.S
> @@ -79,9 +79,10 @@ ENDPROC(__vet_atags)
> */
> __INIT
> __mmap_switched:
> - adr r3, __mmap_switched_data
> -
> - ldmia r3!, {r4, r5, r6, r7}
> + adr_l r4, __data_loc
> + adr_l r5, _sdata
> + adr_l r6, __bss_start
> + adr_l r7, _end
> cmp r4, r5 @ Copy data segment if needed
> 1: cmpne r5, r6
> ldrne fp, [r4], #4
> @@ -93,9 +94,17 @@ __mmap_switched:
> strcc fp, [r6],#4
> bcc 1b
>
> - ARM( ldmia r3, {r4, r5, r6, r7, sp})
> - THUMB( ldmia r3, {r4, r5, r6, r7} )
> - THUMB( ldr sp, [r3, #16] )
> + adr_l r3, init_thread_union + THREAD_START_SP
> + mov sp, r3
> + adr_l r4, processor_id
> + adr_l r5, __machine_arch_type
> + adr_l r6, __atags_pointer
> +#ifdef CONFIG_CPU_CP15
> + adr_l r7, cr_alignment
> +#else
> + mov r7, #0
> +#endif
The code that follows is testing for a non-zero r7 value to store r0 so
you could make that code conditional rather than loading 0 here.
> +
> str r9, [r4] @ Save processor ID
> str r1, [r5] @ Save machine type
> str r2, [r6] @ Save atags pointer
> @@ -104,24 +113,6 @@ __mmap_switched:
> b start_kernel
> ENDPROC(__mmap_switched)
>
> - .align 2
> - .type __mmap_switched_data, %object
> -__mmap_switched_data:
> - .long __data_loc @ r4
> - .long _sdata @ r5
> - .long __bss_start @ r6
> - .long _end @ r7
> - .long processor_id @ r4
> - .long __machine_arch_type @ r5
> - .long __atags_pointer @ r6
> -#ifdef CONFIG_CPU_CP15
> - .long cr_alignment @ r7
> -#else
> - .long 0 @ r7
> -#endif
> - .long init_thread_union + THREAD_START_SP @ sp
> - .size __mmap_switched_data, . - __mmap_switched_data
> -
> /*
> * This provides a C-API version of __lookup_processor_type
> */
> --
> 2.11.0
>
>
^ permalink raw reply [flat|nested] 83+ messages in thread* [PATCH v2 20/29] ARM: kernel: use PC-relative symbol references in MMU switch code
2017-09-04 18:15 ` Nicolas Pitre
@ 2017-09-04 19:14 ` Ard Biesheuvel
0 siblings, 0 replies; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-04 19:14 UTC (permalink / raw)
To: linux-arm-kernel
On 4 September 2017 at 19:15, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
>
>> To prepare for adding support for KASLR, which relocates all absolute
>> symbol references at runtime after the caches have been enabled,
>> update the MMU switch code to avoid using absolute symbol references
>> where possible. This ensures these quantities are invariant under
>> runtime relocation.
>>
>> Cc: Russell King <linux@armlinux.org.uk>
>> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>> ---
>> arch/arm/kernel/head-common.S | 39 ++++++++------------
>> 1 file changed, 15 insertions(+), 24 deletions(-)
>>
>> diff --git a/arch/arm/kernel/head-common.S b/arch/arm/kernel/head-common.S
>> index 06035488130c..b74477507a12 100644
>> --- a/arch/arm/kernel/head-common.S
>> +++ b/arch/arm/kernel/head-common.S
>> @@ -79,9 +79,10 @@ ENDPROC(__vet_atags)
>> */
>> __INIT
>> __mmap_switched:
>> - adr r3, __mmap_switched_data
>> -
>> - ldmia r3!, {r4, r5, r6, r7}
>> + adr_l r4, __data_loc
>> + adr_l r5, _sdata
>> + adr_l r6, __bss_start
>> + adr_l r7, _end
>> cmp r4, r5 @ Copy data segment if needed
>> 1: cmpne r5, r6
>> ldrne fp, [r4], #4
>> @@ -93,9 +94,17 @@ __mmap_switched:
>> strcc fp, [r6],#4
>> bcc 1b
>>
>> - ARM( ldmia r3, {r4, r5, r6, r7, sp})
>> - THUMB( ldmia r3, {r4, r5, r6, r7} )
>> - THUMB( ldr sp, [r3, #16] )
>> + adr_l r3, init_thread_union + THREAD_START_SP
>> + mov sp, r3
>> + adr_l r4, processor_id
>> + adr_l r5, __machine_arch_type
>> + adr_l r6, __atags_pointer
>> +#ifdef CONFIG_CPU_CP15
>> + adr_l r7, cr_alignment
>> +#else
>> + mov r7, #0
>> +#endif
>
> The code that follows is testing for a non-zero r7 value to store r0 so
> you could make that code conditional rather than loading 0 here.
>
OK, I will look into that.
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 21/29] ARM: kernel: use PC relative symbol references in suspend/resume code
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (19 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 20/29] ARM: kernel: use PC-relative symbol references in MMU switch code Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-04 18:24 ` Nicolas Pitre
2017-09-03 12:07 ` [PATCH v2 22/29] ARM: mm: export default vmalloc base address Ard Biesheuvel
` (8 subsequent siblings)
29 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
Replace some unnecessary absolute references with relative ones. Also,
to prepare for runtime relocation, which occurs with the caches on,
defer taking the absolute address of cpu_resume_after_mmu() until after
the MMU is enabled.
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/kernel/sleep.S | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/arch/arm/kernel/sleep.S b/arch/arm/kernel/sleep.S
index f4920b5d0fc4..5b02744f2f12 100644
--- a/arch/arm/kernel/sleep.S
+++ b/arch/arm/kernel/sleep.S
@@ -60,15 +60,14 @@
ENTRY(__cpu_suspend)
stmfd sp!, {r4 - r11, lr}
#ifdef MULTI_CPU
- ldr r10, =processor
- ldr r4, [r10, #CPU_SLEEP_SIZE] @ size of CPU sleep state
+ ldr_l r4, processor + CPU_SLEEP_SIZE @ size of CPU sleep state
#else
- ldr r4, =cpu_suspend_size
+ adr_l r4, cpu_suspend_size
#endif
mov r5, sp @ current virtual SP
add r4, r4, #12 @ Space for pgd, virt sp, phys resume fn
sub sp, sp, r4 @ allocate CPU state on stack
- ldr r3, =sleep_save_sp
+ adr_l r3, sleep_save_sp
stmfd sp!, {r0, r1} @ save suspend func arg and pointer
ldr r3, [r3, #SLEEP_SAVE_SP_VIRT]
ALT_SMP(W(nop)) @ don't use adr_l inside ALT_SMP()
@@ -101,13 +100,13 @@ ENDPROC(cpu_suspend_abort)
.align 5
.pushsection .idmap.text,"ax"
ENTRY(cpu_resume_mmu)
- ldr r3, =cpu_resume_after_mmu
instr_sync
mcr p15, 0, r0, c1, c0, 0 @ turn on MMU, I-cache, etc
mrc p15, 0, r0, c0, c0, 0 @ read id reg
instr_sync
mov r0, r0
mov r0, r0
+ ldr r3, =cpu_resume_after_mmu
ret r3 @ jump to virtual address
ENDPROC(cpu_resume_mmu)
.popsection
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 21/29] ARM: kernel: use PC relative symbol references in suspend/resume code
2017-09-03 12:07 ` [PATCH v2 21/29] ARM: kernel: use PC relative symbol references in suspend/resume code Ard Biesheuvel
@ 2017-09-04 18:24 ` Nicolas Pitre
2017-09-04 19:17 ` Ard Biesheuvel
0 siblings, 1 reply; 83+ messages in thread
From: Nicolas Pitre @ 2017-09-04 18:24 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
> Replace some unnecessary absolute references with relative ones. Also,
> to prepare for runtime relocation, which occurs with the caches on,
> defer taking the absolute address of cpu_resume_after_mmu() until after
> the MMU is enabled.
>
> Cc: Russell King <linux@armlinux.org.uk>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
What's the point of that last hunk? I understand you load the address
after the mMU is on. But you should be coming back from a sleep and
caches ought to be clean at that point. Not that it is a bad thing to do
but I don't understand your reason for it.
> ---
> arch/arm/kernel/sleep.S | 9 ++++-----
> 1 file changed, 4 insertions(+), 5 deletions(-)
>
> diff --git a/arch/arm/kernel/sleep.S b/arch/arm/kernel/sleep.S
> index f4920b5d0fc4..5b02744f2f12 100644
> --- a/arch/arm/kernel/sleep.S
> +++ b/arch/arm/kernel/sleep.S
> @@ -60,15 +60,14 @@
> ENTRY(__cpu_suspend)
> stmfd sp!, {r4 - r11, lr}
> #ifdef MULTI_CPU
> - ldr r10, =processor
> - ldr r4, [r10, #CPU_SLEEP_SIZE] @ size of CPU sleep state
> + ldr_l r4, processor + CPU_SLEEP_SIZE @ size of CPU sleep state
> #else
> - ldr r4, =cpu_suspend_size
> + adr_l r4, cpu_suspend_size
> #endif
> mov r5, sp @ current virtual SP
> add r4, r4, #12 @ Space for pgd, virt sp, phys resume fn
> sub sp, sp, r4 @ allocate CPU state on stack
> - ldr r3, =sleep_save_sp
> + adr_l r3, sleep_save_sp
> stmfd sp!, {r0, r1} @ save suspend func arg and pointer
> ldr r3, [r3, #SLEEP_SAVE_SP_VIRT]
> ALT_SMP(W(nop)) @ don't use adr_l inside ALT_SMP()
> @@ -101,13 +100,13 @@ ENDPROC(cpu_suspend_abort)
> .align 5
> .pushsection .idmap.text,"ax"
> ENTRY(cpu_resume_mmu)
> - ldr r3, =cpu_resume_after_mmu
> instr_sync
> mcr p15, 0, r0, c1, c0, 0 @ turn on MMU, I-cache, etc
> mrc p15, 0, r0, c0, c0, 0 @ read id reg
> instr_sync
> mov r0, r0
> mov r0, r0
> + ldr r3, =cpu_resume_after_mmu
> ret r3 @ jump to virtual address
> ENDPROC(cpu_resume_mmu)
> .popsection
> --
> 2.11.0
>
>
^ permalink raw reply [flat|nested] 83+ messages in thread* [PATCH v2 21/29] ARM: kernel: use PC relative symbol references in suspend/resume code
2017-09-04 18:24 ` Nicolas Pitre
@ 2017-09-04 19:17 ` Ard Biesheuvel
0 siblings, 0 replies; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-04 19:17 UTC (permalink / raw)
To: linux-arm-kernel
On 4 September 2017 at 19:24, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
>
>> Replace some unnecessary absolute references with relative ones. Also,
>> to prepare for runtime relocation, which occurs with the caches on,
>> defer taking the absolute address of cpu_resume_after_mmu() until after
>> the MMU is enabled.
>>
>> Cc: Russell King <linux@armlinux.org.uk>
>> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>
> What's the point of that last hunk? I understand you load the address
> after the mMU is on. But you should be coming back from a sleep and
> caches ought to be clean at that point. Not that it is a bad thing to do
> but I don't understand your reason for it.
>
I simply attempted to eliminate all uses of relocated quantities with
the caches off, but you are correct that this is only necessary for
boot and not for suspend/resume. This is actually somewhat of a
relief, since there is so much SOC specific suspend/resume code under
arch/arm, and the boot code is mostly shared.
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 22/29] ARM: mm: export default vmalloc base address
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (20 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 21/29] ARM: kernel: use PC relative symbol references in suspend/resume code Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-04 18:25 ` Nicolas Pitre
2017-09-03 12:07 ` [PATCH v2 23/29] ARM: kernel: refer to swapper_pg_dir via its symbol Ard Biesheuvel
` (7 subsequent siblings)
29 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
In order for the EFI stub to be able to decide over what range to
randomize the load address of the kernel, expose the definition of
the default vmalloc base address as VMALLOC_DEFAULT_BASE.
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/include/asm/pgtable.h | 1 +
arch/arm/mm/mmu.c | 3 +--
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/arm/include/asm/pgtable.h b/arch/arm/include/asm/pgtable.h
index 1c462381c225..a0d4a63a6516 100644
--- a/arch/arm/include/asm/pgtable.h
+++ b/arch/arm/include/asm/pgtable.h
@@ -45,6 +45,7 @@
#define VMALLOC_OFFSET (8*1024*1024)
#define VMALLOC_START (((unsigned long)high_memory + VMALLOC_OFFSET) & ~(VMALLOC_OFFSET-1))
#define VMALLOC_END 0xff800000UL
+#define VMALLOC_DEFAULT_BASE (VMALLOC_END - (240 << 20) - VMALLOC_OFFSET)
#define LIBRARY_TEXT_START 0x0c000000
diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c
index e46a6a446cdd..566f7e657931 100644
--- a/arch/arm/mm/mmu.c
+++ b/arch/arm/mm/mmu.c
@@ -1128,8 +1128,7 @@ void __init debug_ll_io_init(void)
}
#endif
-static void * __initdata vmalloc_min =
- (void *)(VMALLOC_END - (240 << 20) - VMALLOC_OFFSET);
+static void * __initdata vmalloc_min = (void *)VMALLOC_DEFAULT_BASE;
/*
* vmalloc=size forces the vmalloc area to be exactly 'size'
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 22/29] ARM: mm: export default vmalloc base address
2017-09-03 12:07 ` [PATCH v2 22/29] ARM: mm: export default vmalloc base address Ard Biesheuvel
@ 2017-09-04 18:25 ` Nicolas Pitre
0 siblings, 0 replies; 83+ messages in thread
From: Nicolas Pitre @ 2017-09-04 18:25 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
> In order for the EFI stub to be able to decide over what range to
> randomize the load address of the kernel, expose the definition of
> the default vmalloc base address as VMALLOC_DEFAULT_BASE.
>
> Cc: Russell King <linux@armlinux.org.uk>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
> ---
> arch/arm/include/asm/pgtable.h | 1 +
> arch/arm/mm/mmu.c | 3 +--
> 2 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm/include/asm/pgtable.h b/arch/arm/include/asm/pgtable.h
> index 1c462381c225..a0d4a63a6516 100644
> --- a/arch/arm/include/asm/pgtable.h
> +++ b/arch/arm/include/asm/pgtable.h
> @@ -45,6 +45,7 @@
> #define VMALLOC_OFFSET (8*1024*1024)
> #define VMALLOC_START (((unsigned long)high_memory + VMALLOC_OFFSET) & ~(VMALLOC_OFFSET-1))
> #define VMALLOC_END 0xff800000UL
> +#define VMALLOC_DEFAULT_BASE (VMALLOC_END - (240 << 20) - VMALLOC_OFFSET)
>
> #define LIBRARY_TEXT_START 0x0c000000
>
> diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c
> index e46a6a446cdd..566f7e657931 100644
> --- a/arch/arm/mm/mmu.c
> +++ b/arch/arm/mm/mmu.c
> @@ -1128,8 +1128,7 @@ void __init debug_ll_io_init(void)
> }
> #endif
>
> -static void * __initdata vmalloc_min =
> - (void *)(VMALLOC_END - (240 << 20) - VMALLOC_OFFSET);
> +static void * __initdata vmalloc_min = (void *)VMALLOC_DEFAULT_BASE;
>
> /*
> * vmalloc=size forces the vmalloc area to be exactly 'size'
> --
> 2.11.0
>
>
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 23/29] ARM: kernel: refer to swapper_pg_dir via its symbol
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (21 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 22/29] ARM: mm: export default vmalloc base address Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-04 18:30 ` Nicolas Pitre
2017-09-03 12:07 ` [PATCH v2 24/29] ARM: kernel: implement randomization of the kernel load address Ard Biesheuvel
` (6 subsequent siblings)
29 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
The location of swapper_pg_dir is relative to the kernel, not to
PAGE_OFFSET or PHYS_OFFSET. So define the symbol relative to the
start of the kernel image, and refer to it via its name.
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/kernel/head.S | 13 ++++---------
1 file changed, 4 insertions(+), 9 deletions(-)
diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
index 5d685e86148c..71bc0d037bc9 100644
--- a/arch/arm/kernel/head.S
+++ b/arch/arm/kernel/head.S
@@ -48,14 +48,6 @@
#define PMD_ORDER 2
#endif
- .globl swapper_pg_dir
- .equ swapper_pg_dir, KERNEL_RAM_VADDR - PG_DIR_SIZE
-
- .macro pgtbl, rd, phys
- add \rd, \phys, #TEXT_OFFSET
- sub \rd, \rd, #PG_DIR_SIZE
- .endm
-
/*
* Kernel startup entry point.
* ---------------------------
@@ -77,6 +69,9 @@
.arm
__HEAD
+ .globl swapper_pg_dir
+ .equ swapper_pg_dir, . - PG_DIR_SIZE
+
ENTRY(stext)
ARM_BE8(setend be ) @ ensure we are in BE8 mode
@@ -172,7 +167,7 @@ ENDPROC(stext)
* r4 = physical page table address
*/
__create_page_tables:
- pgtbl r4, r8 @ page table address
+ adr_l r4, swapper_pg_dir @ page table address
/*
* Clear the swapper page table
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 23/29] ARM: kernel: refer to swapper_pg_dir via its symbol
2017-09-03 12:07 ` [PATCH v2 23/29] ARM: kernel: refer to swapper_pg_dir via its symbol Ard Biesheuvel
@ 2017-09-04 18:30 ` Nicolas Pitre
2017-09-04 19:26 ` Ard Biesheuvel
0 siblings, 1 reply; 83+ messages in thread
From: Nicolas Pitre @ 2017-09-04 18:30 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
> The location of swapper_pg_dir is relative to the kernel, not to
> PAGE_OFFSET or PHYS_OFFSET. So define the symbol relative to the
> start of the kernel image, and refer to it via its name.
Could it cause a section access mismatch if defined in the __HEAD
section?
Otherwise
Acked-by: Nicolas Pitre <nico@linaro.org>
>
> Cc: Russell King <linux@armlinux.org.uk>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> ---
> arch/arm/kernel/head.S | 13 ++++---------
> 1 file changed, 4 insertions(+), 9 deletions(-)
>
> diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
> index 5d685e86148c..71bc0d037bc9 100644
> --- a/arch/arm/kernel/head.S
> +++ b/arch/arm/kernel/head.S
> @@ -48,14 +48,6 @@
> #define PMD_ORDER 2
> #endif
>
> - .globl swapper_pg_dir
> - .equ swapper_pg_dir, KERNEL_RAM_VADDR - PG_DIR_SIZE
> -
> - .macro pgtbl, rd, phys
> - add \rd, \phys, #TEXT_OFFSET
> - sub \rd, \rd, #PG_DIR_SIZE
> - .endm
> -
> /*
> * Kernel startup entry point.
> * ---------------------------
> @@ -77,6 +69,9 @@
> .arm
>
> __HEAD
> + .globl swapper_pg_dir
> + .equ swapper_pg_dir, . - PG_DIR_SIZE
> +
> ENTRY(stext)
> ARM_BE8(setend be ) @ ensure we are in BE8 mode
>
> @@ -172,7 +167,7 @@ ENDPROC(stext)
> * r4 = physical page table address
> */
> __create_page_tables:
> - pgtbl r4, r8 @ page table address
> + adr_l r4, swapper_pg_dir @ page table address
>
> /*
> * Clear the swapper page table
> --
> 2.11.0
>
>
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 23/29] ARM: kernel: refer to swapper_pg_dir via its symbol
2017-09-04 18:30 ` Nicolas Pitre
@ 2017-09-04 19:26 ` Ard Biesheuvel
0 siblings, 0 replies; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-04 19:26 UTC (permalink / raw)
To: linux-arm-kernel
On 4 September 2017 at 19:30, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
>
>> The location of swapper_pg_dir is relative to the kernel, not to
>> PAGE_OFFSET or PHYS_OFFSET. So define the symbol relative to the
>> start of the kernel image, and refer to it via its name.
>
> Could it cause a section access mismatch if defined in the __HEAD
> section?
>
No, I never saw any mismatch warnings building this code, and I don't
think .head.text is treated as an __init section.
> Otherwise
>
> Acked-by: Nicolas Pitre <nico@linaro.org>
>
Thanks
>>
>> Cc: Russell King <linux@armlinux.org.uk>
>> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>> ---
>> arch/arm/kernel/head.S | 13 ++++---------
>> 1 file changed, 4 insertions(+), 9 deletions(-)
>>
>> diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
>> index 5d685e86148c..71bc0d037bc9 100644
>> --- a/arch/arm/kernel/head.S
>> +++ b/arch/arm/kernel/head.S
>> @@ -48,14 +48,6 @@
>> #define PMD_ORDER 2
>> #endif
>>
>> - .globl swapper_pg_dir
>> - .equ swapper_pg_dir, KERNEL_RAM_VADDR - PG_DIR_SIZE
>> -
>> - .macro pgtbl, rd, phys
>> - add \rd, \phys, #TEXT_OFFSET
>> - sub \rd, \rd, #PG_DIR_SIZE
>> - .endm
>> -
>> /*
>> * Kernel startup entry point.
>> * ---------------------------
>> @@ -77,6 +69,9 @@
>> .arm
>>
>> __HEAD
>> + .globl swapper_pg_dir
>> + .equ swapper_pg_dir, . - PG_DIR_SIZE
>> +
>> ENTRY(stext)
>> ARM_BE8(setend be ) @ ensure we are in BE8 mode
>>
>> @@ -172,7 +167,7 @@ ENDPROC(stext)
>> * r4 = physical page table address
>> */
>> __create_page_tables:
>> - pgtbl r4, r8 @ page table address
>> + adr_l r4, swapper_pg_dir @ page table address
>>
>> /*
>> * Clear the swapper page table
>> --
>> 2.11.0
>>
>>
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 24/29] ARM: kernel: implement randomization of the kernel load address
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (22 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 23/29] ARM: kernel: refer to swapper_pg_dir via its symbol Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-04 18:44 ` Nicolas Pitre
2017-09-03 12:07 ` [PATCH v2 25/29] ARM: decompressor: explicitly map decompressor binary cacheable Ard Biesheuvel
` (5 subsequent siblings)
29 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
This implements randomization of the placement of the kernel image
inside the lowmem region. It is intended to work together with the
decompressor to place the kernel at an offset in physical memory
that is a multiple of 2 MB, and to take the same offset into account
when creating the virtual mapping.
This uses runtime relocation of the kernel built as a PIE binary, to
fix up all absolute symbol references to refer to their runtime virtual
address. The physical-to-virtual mapping remains unchanged.
In order to allow the decompressor to hand over to the core kernel
without making assumptions that are not guaranteed to hold when
invoking the core kernel directly using bootloaders that are not
KASLR aware, the KASLR offset is expected to be placed in r3 when
entering the kernel 4 bytes past the entry point, skipping the first
instruction.
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/Kconfig | 15 +++
arch/arm/kernel/head.S | 103 ++++++++++++++++++--
2 files changed, 109 insertions(+), 9 deletions(-)
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 300add3b8023..fe4a2cd1f15c 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -1825,6 +1825,21 @@ config XEN
help
Say Y if you want to run Linux in a Virtual Machine on Xen on ARM.
+config RANDOMIZE_BASE
+ bool "Randomize the address of the kernel image"
+ depends on MMU && AUTO_ZRELADDR
+ depends on !XIP_KERNEL && !ZBOOT_ROM
+ select RELOCATABLE
+ help
+ Randomizes the virtual and physical address at which the kernel
+ image is loaded, as a security feature that deters exploit attempts
+ relying on knowledge of the location of kernel internals.
+
+ This relies on the UEFI stub to invoke the EFI_RNG_PROTOCOL to
+ randomize the load address of the decompressed kernel in the
+ physical space. The same offset is applied to the virtual mapping
+ of the kernel in the virtual space by the kernel proper.
+
endmenu
menu "Boot options"
diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
index 71bc0d037bc9..0795da990dde 100644
--- a/arch/arm/kernel/head.S
+++ b/arch/arm/kernel/head.S
@@ -48,6 +48,28 @@
#define PMD_ORDER 2
#endif
+ .macro get_kaslr_offset, reg
+#ifdef CONFIG_RANDOMIZE_BASE
+ ldr_l \reg, __kaslr_offset
+#else
+ mov \reg, #0
+#endif
+ .endm
+
+ .macro add_kaslr_offset, reg, tmp
+#ifdef CONFIG_RANDOMIZE_BASE
+ get_kaslr_offset \tmp
+ add \reg, \reg, \tmp
+#endif
+ .endm
+
+ .macro sub_kaslr_offset, reg, tmp
+#ifdef CONFIG_RANDOMIZE_BASE
+ get_kaslr_offset \tmp
+ sub \reg, \reg, \tmp
+#endif
+ .endm
+
/*
* Kernel startup entry point.
* ---------------------------
@@ -73,6 +95,7 @@
.equ swapper_pg_dir, . - PG_DIR_SIZE
ENTRY(stext)
+ mov r3, #0 @ normal entry point - clear r3
ARM_BE8(setend be ) @ ensure we are in BE8 mode
THUMB( badr r9, 1f ) @ Kernel is always entered in ARM.
@@ -80,6 +103,16 @@ ENTRY(stext)
THUMB( .thumb ) @ switch to Thumb now.
THUMB(1: )
+#ifdef CONFIG_RANDOMIZE_BASE
+ str_l r3, __kaslr_offset, r9 @ offset in r3 if entered via kaslr ep
+
+ .section ".bss", "aw", %nobits
+ .align 2
+__kaslr_offset:
+ .long 0 @ will be wiped before entering C code
+ .previous
+#endif
+
#ifdef CONFIG_ARM_VIRT_EXT
bl __hyp_stub_install
#endif
@@ -103,6 +136,7 @@ ENTRY(stext)
#ifndef CONFIG_XIP_KERNEL
adr_l r8, _text @ __pa(_text)
sub r8, r8, #TEXT_OFFSET @ PHYS_OFFSET
+ sub_kaslr_offset r8, r12
#else
ldr r8, =PLAT_PHYS_OFFSET @ always constant in this case
#endif
@@ -139,8 +173,8 @@ ENTRY(stext)
* r0 will hold the CPU control register value, r1, r2, r4, and
* r9 will be preserved. r5 will also be preserved if LPAE.
*/
- ldr r13, =__mmap_switched @ address to jump to after
- @ mmu has been enabled
+ adr_l lr, __primary_switch @ address to jump to after
+ mov r13, lr @ mmu has been enabled
badr lr, 1f @ return (PIC) address
#ifdef CONFIG_ARM_LPAE
mov r5, #0 @ high TTBR0
@@ -151,7 +185,8 @@ ENTRY(stext)
ldr r12, [r10, #PROCINFO_INITFUNC]
add r12, r12, r10
ret r12
-1: b __enable_mmu
+1: get_kaslr_offset r12 @ get before turning MMU on
+ b __enable_mmu
ENDPROC(stext)
.ltorg
@@ -230,9 +265,14 @@ __create_page_tables:
/*
* Map our RAM from the start to the end of the kernel .bss section.
*/
- add r0, r4, #PAGE_OFFSET >> (SECTION_SHIFT - PMD_ORDER)
- ldr r6, =(_end - 1)
- orr r3, r8, r7
+ get_kaslr_offset r3
+ add r0, r3, #PAGE_OFFSET
+ add r0, r4, r0, lsr #(SECTION_SHIFT - PMD_ORDER)
+ adr_l r6, _end - 1
+ sub r6, r6, r8
+ add r6, r6, #PAGE_OFFSET
+ add r3, r3, r8
+ orr r3, r3, r7
add r6, r4, r6, lsr #(SECTION_SHIFT - PMD_ORDER)
1: str r3, [r0], #1 << PMD_ORDER
add r3, r3, #1 << SECTION_SHIFT
@@ -376,7 +416,7 @@ ENTRY(secondary_startup)
* Use the page tables supplied from __cpu_up.
*/
adr_l r3, secondary_data
- mov_l r12, __secondary_switched
+ adr_l r12, __secondary_switch
ldrd r4, [r3, #0] @ get secondary_data.pgdir
ARM_BE8(eor r4, r4, r5) @ Swap r5 and r4 in BE:
ARM_BE8(eor r5, r4, r5) @ it can be done in 3 steps
@@ -414,6 +454,7 @@ ENDPROC(__secondary_switched)
* r4 = TTBR pointer (low word)
* r5 = TTBR pointer (high word if LPAE)
* r9 = processor ID
+ * r12 = KASLR offset
* r13 = *virtual* address to jump to upon completion
*/
__enable_mmu:
@@ -451,6 +492,7 @@ ENDPROC(__enable_mmu)
* r1 = machine ID
* r2 = atags or dtb pointer
* r9 = processor ID
+ * r12 = KASLR offset
* r13 = *virtual* address to jump to upon completion
*
* other registers depend on the function called upon completion
@@ -466,10 +508,52 @@ ENTRY(__turn_mmu_on)
mov r3, r3
mov r3, r13
ret r3
-__turn_mmu_on_end:
ENDPROC(__turn_mmu_on)
- .popsection
+__primary_switch:
+#ifdef CONFIG_RELOCATABLE
+ adr_l r7, _text @ r7 := __pa(_text)
+ sub r7, r7, #TEXT_OFFSET @ r7 := PHYS_OFFSET
+
+ adr_l r5, __rel_begin
+ adr_l r6, __rel_end
+ sub r5, r5, r7
+ sub r6, r6, r7
+
+ add r5, r5, #PAGE_OFFSET
+ add r6, r6, #PAGE_OFFSET
+ add r5, r5, r12
+ add r6, r6, r12
+
+ adr_l r3, __stubs_start @ __pa(__stubs_start)
+ sub r3, r3, r7 @ offset of __stubs_start
+ add r3, r3, #PAGE_OFFSET @ __va(__stubs_start)
+ sub r3, r3, #0xffff1000 @ subtract VA of stubs section
+
+0: cmp r5, r6
+ bge 1f
+ ldm r5!, {r7, r8} @ load next relocation entry
+ cmp r8, #23 @ R_ARM_RELATIVE
+ bne 0b
+ cmp r7, #0xff000000 @ vector page?
+ addgt r7, r7, r3 @ fix up VA offset
+ ldr r8, [r7, r12]
+ add r8, r8, r12
+ str r8, [r7, r12]
+ b 0b
+1:
+#endif
+ ldr pc, =__mmap_switched
+ENDPROC(__primary_switch)
+
+#ifdef CONFIG_SMP
+__secondary_switch:
+ ldr pc, =__secondary_switched
+ENDPROC(__secondary_switch)
+#endif
+ .ltorg
+__turn_mmu_on_end:
+ .popsection
#ifdef CONFIG_SMP_ON_UP
__HEAD
@@ -570,6 +654,7 @@ __fixup_pv_table:
adr_l r6, __pv_phys_pfn_offset
adr_l r7, __pv_offset @ __pa(__pv_offset)
mov_l r3, __pv_offset @ __va(__pv_offset)
+ add_kaslr_offset r3, ip
mvn ip, #0
subs r3, r7, r3 @ PHYS_OFFSET - PAGE_OFFSET
mov r0, r8, lsr #PAGE_SHIFT @ convert to PFN
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 24/29] ARM: kernel: implement randomization of the kernel load address
2017-09-03 12:07 ` [PATCH v2 24/29] ARM: kernel: implement randomization of the kernel load address Ard Biesheuvel
@ 2017-09-04 18:44 ` Nicolas Pitre
2017-09-04 19:29 ` Ard Biesheuvel
0 siblings, 1 reply; 83+ messages in thread
From: Nicolas Pitre @ 2017-09-04 18:44 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
> This implements randomization of the placement of the kernel image
> inside the lowmem region. It is intended to work together with the
> decompressor to place the kernel at an offset in physical memory
> that is a multiple of 2 MB, and to take the same offset into account
> when creating the virtual mapping.
>
> This uses runtime relocation of the kernel built as a PIE binary, to
> fix up all absolute symbol references to refer to their runtime virtual
> address. The physical-to-virtual mapping remains unchanged.
>
> In order to allow the decompressor to hand over to the core kernel
> without making assumptions that are not guaranteed to hold when
> invoking the core kernel directly using bootloaders that are not
> KASLR aware, the KASLR offset is expected to be placed in r3 when
> entering the kernel 4 bytes past the entry point, skipping the first
> instruction.
>
> Cc: Russell King <linux@armlinux.org.uk>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> ---
> arch/arm/Kconfig | 15 +++
> arch/arm/kernel/head.S | 103 ++++++++++++++++++--
> 2 files changed, 109 insertions(+), 9 deletions(-)
>
> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
> index 300add3b8023..fe4a2cd1f15c 100644
> --- a/arch/arm/Kconfig
> +++ b/arch/arm/Kconfig
> @@ -1825,6 +1825,21 @@ config XEN
> help
> Say Y if you want to run Linux in a Virtual Machine on Xen on ARM.
>
> +config RANDOMIZE_BASE
> + bool "Randomize the address of the kernel image"
> + depends on MMU && AUTO_ZRELADDR
> + depends on !XIP_KERNEL && !ZBOOT_ROM
This should work even if ZBOOT_ROM is selected. The ZBOOT_ROM option
allows for the decompressor executing directly from ROM and
decompressing the kernel anywhere in RAM.
> + select RELOCATABLE
> + help
> + Randomizes the virtual and physical address at which the kernel
> + image is loaded, as a security feature that deters exploit attempts
> + relying on knowledge of the location of kernel internals.
> +
> + This relies on the UEFI stub to invoke the EFI_RNG_PROTOCOL to
> + randomize the load address of the decompressed kernel in the
> + physical space. The same offset is applied to the virtual mapping
> + of the kernel in the virtual space by the kernel proper.
> +
> endmenu
>
> menu "Boot options"
> diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
> index 71bc0d037bc9..0795da990dde 100644
> --- a/arch/arm/kernel/head.S
> +++ b/arch/arm/kernel/head.S
> @@ -48,6 +48,28 @@
> #define PMD_ORDER 2
> #endif
>
> + .macro get_kaslr_offset, reg
> +#ifdef CONFIG_RANDOMIZE_BASE
> + ldr_l \reg, __kaslr_offset
> +#else
> + mov \reg, #0
> +#endif
> + .endm
> +
> + .macro add_kaslr_offset, reg, tmp
> +#ifdef CONFIG_RANDOMIZE_BASE
> + get_kaslr_offset \tmp
> + add \reg, \reg, \tmp
> +#endif
> + .endm
> +
> + .macro sub_kaslr_offset, reg, tmp
> +#ifdef CONFIG_RANDOMIZE_BASE
> + get_kaslr_offset \tmp
> + sub \reg, \reg, \tmp
> +#endif
> + .endm
> +
> /*
> * Kernel startup entry point.
> * ---------------------------
> @@ -73,6 +95,7 @@
> .equ swapper_pg_dir, . - PG_DIR_SIZE
>
> ENTRY(stext)
> + mov r3, #0 @ normal entry point - clear r3
> ARM_BE8(setend be ) @ ensure we are in BE8 mode
>
> THUMB( badr r9, 1f ) @ Kernel is always entered in ARM.
> @@ -80,6 +103,16 @@ ENTRY(stext)
> THUMB( .thumb ) @ switch to Thumb now.
> THUMB(1: )
>
> +#ifdef CONFIG_RANDOMIZE_BASE
> + str_l r3, __kaslr_offset, r9 @ offset in r3 if entered via kaslr ep
> +
> + .section ".bss", "aw", %nobits
> + .align 2
> +__kaslr_offset:
> + .long 0 @ will be wiped before entering C code
> + .previous
> +#endif
> +
> #ifdef CONFIG_ARM_VIRT_EXT
> bl __hyp_stub_install
> #endif
> @@ -103,6 +136,7 @@ ENTRY(stext)
> #ifndef CONFIG_XIP_KERNEL
> adr_l r8, _text @ __pa(_text)
> sub r8, r8, #TEXT_OFFSET @ PHYS_OFFSET
> + sub_kaslr_offset r8, r12
> #else
> ldr r8, =PLAT_PHYS_OFFSET @ always constant in this case
> #endif
> @@ -139,8 +173,8 @@ ENTRY(stext)
> * r0 will hold the CPU control register value, r1, r2, r4, and
> * r9 will be preserved. r5 will also be preserved if LPAE.
> */
> - ldr r13, =__mmap_switched @ address to jump to after
> - @ mmu has been enabled
> + adr_l lr, __primary_switch @ address to jump to after
> + mov r13, lr @ mmu has been enabled
> badr lr, 1f @ return (PIC) address
> #ifdef CONFIG_ARM_LPAE
> mov r5, #0 @ high TTBR0
> @@ -151,7 +185,8 @@ ENTRY(stext)
> ldr r12, [r10, #PROCINFO_INITFUNC]
> add r12, r12, r10
> ret r12
> -1: b __enable_mmu
> +1: get_kaslr_offset r12 @ get before turning MMU on
> + b __enable_mmu
> ENDPROC(stext)
> .ltorg
>
> @@ -230,9 +265,14 @@ __create_page_tables:
> /*
> * Map our RAM from the start to the end of the kernel .bss section.
> */
> - add r0, r4, #PAGE_OFFSET >> (SECTION_SHIFT - PMD_ORDER)
> - ldr r6, =(_end - 1)
> - orr r3, r8, r7
> + get_kaslr_offset r3
> + add r0, r3, #PAGE_OFFSET
> + add r0, r4, r0, lsr #(SECTION_SHIFT - PMD_ORDER)
> + adr_l r6, _end - 1
> + sub r6, r6, r8
> + add r6, r6, #PAGE_OFFSET
> + add r3, r3, r8
> + orr r3, r3, r7
> add r6, r4, r6, lsr #(SECTION_SHIFT - PMD_ORDER)
> 1: str r3, [r0], #1 << PMD_ORDER
> add r3, r3, #1 << SECTION_SHIFT
> @@ -376,7 +416,7 @@ ENTRY(secondary_startup)
> * Use the page tables supplied from __cpu_up.
> */
> adr_l r3, secondary_data
> - mov_l r12, __secondary_switched
> + adr_l r12, __secondary_switch
> ldrd r4, [r3, #0] @ get secondary_data.pgdir
> ARM_BE8(eor r4, r4, r5) @ Swap r5 and r4 in BE:
> ARM_BE8(eor r5, r4, r5) @ it can be done in 3 steps
> @@ -414,6 +454,7 @@ ENDPROC(__secondary_switched)
> * r4 = TTBR pointer (low word)
> * r5 = TTBR pointer (high word if LPAE)
> * r9 = processor ID
> + * r12 = KASLR offset
> * r13 = *virtual* address to jump to upon completion
> */
> __enable_mmu:
> @@ -451,6 +492,7 @@ ENDPROC(__enable_mmu)
> * r1 = machine ID
> * r2 = atags or dtb pointer
> * r9 = processor ID
> + * r12 = KASLR offset
> * r13 = *virtual* address to jump to upon completion
> *
> * other registers depend on the function called upon completion
> @@ -466,10 +508,52 @@ ENTRY(__turn_mmu_on)
> mov r3, r3
> mov r3, r13
> ret r3
> -__turn_mmu_on_end:
> ENDPROC(__turn_mmu_on)
> - .popsection
>
> +__primary_switch:
> +#ifdef CONFIG_RELOCATABLE
> + adr_l r7, _text @ r7 := __pa(_text)
> + sub r7, r7, #TEXT_OFFSET @ r7 := PHYS_OFFSET
> +
> + adr_l r5, __rel_begin
> + adr_l r6, __rel_end
> + sub r5, r5, r7
> + sub r6, r6, r7
> +
> + add r5, r5, #PAGE_OFFSET
> + add r6, r6, #PAGE_OFFSET
> + add r5, r5, r12
> + add r6, r6, r12
> +
> + adr_l r3, __stubs_start @ __pa(__stubs_start)
> + sub r3, r3, r7 @ offset of __stubs_start
> + add r3, r3, #PAGE_OFFSET @ __va(__stubs_start)
> + sub r3, r3, #0xffff1000 @ subtract VA of stubs section
> +
> +0: cmp r5, r6
> + bge 1f
> + ldm r5!, {r7, r8} @ load next relocation entry
> + cmp r8, #23 @ R_ARM_RELATIVE
> + bne 0b
> + cmp r7, #0xff000000 @ vector page?
> + addgt r7, r7, r3 @ fix up VA offset
> + ldr r8, [r7, r12]
> + add r8, r8, r12
> + str r8, [r7, r12]
> + b 0b
> +1:
> +#endif
> + ldr pc, =__mmap_switched
> +ENDPROC(__primary_switch)
> +
> +#ifdef CONFIG_SMP
> +__secondary_switch:
> + ldr pc, =__secondary_switched
> +ENDPROC(__secondary_switch)
> +#endif
> + .ltorg
> +__turn_mmu_on_end:
> + .popsection
>
> #ifdef CONFIG_SMP_ON_UP
> __HEAD
> @@ -570,6 +654,7 @@ __fixup_pv_table:
> adr_l r6, __pv_phys_pfn_offset
> adr_l r7, __pv_offset @ __pa(__pv_offset)
> mov_l r3, __pv_offset @ __va(__pv_offset)
> + add_kaslr_offset r3, ip
> mvn ip, #0
> subs r3, r7, r3 @ PHYS_OFFSET - PAGE_OFFSET
> mov r0, r8, lsr #PAGE_SHIFT @ convert to PFN
> --
> 2.11.0
>
>
^ permalink raw reply [flat|nested] 83+ messages in thread* [PATCH v2 24/29] ARM: kernel: implement randomization of the kernel load address
2017-09-04 18:44 ` Nicolas Pitre
@ 2017-09-04 19:29 ` Ard Biesheuvel
0 siblings, 0 replies; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-04 19:29 UTC (permalink / raw)
To: linux-arm-kernel
On 4 September 2017 at 19:44, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
>
>> This implements randomization of the placement of the kernel image
>> inside the lowmem region. It is intended to work together with the
>> decompressor to place the kernel at an offset in physical memory
>> that is a multiple of 2 MB, and to take the same offset into account
>> when creating the virtual mapping.
>>
>> This uses runtime relocation of the kernel built as a PIE binary, to
>> fix up all absolute symbol references to refer to their runtime virtual
>> address. The physical-to-virtual mapping remains unchanged.
>>
>> In order to allow the decompressor to hand over to the core kernel
>> without making assumptions that are not guaranteed to hold when
>> invoking the core kernel directly using bootloaders that are not
>> KASLR aware, the KASLR offset is expected to be placed in r3 when
>> entering the kernel 4 bytes past the entry point, skipping the first
>> instruction.
>>
>> Cc: Russell King <linux@armlinux.org.uk>
>> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>> ---
>> arch/arm/Kconfig | 15 +++
>> arch/arm/kernel/head.S | 103 ++++++++++++++++++--
>> 2 files changed, 109 insertions(+), 9 deletions(-)
>>
>> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
>> index 300add3b8023..fe4a2cd1f15c 100644
>> --- a/arch/arm/Kconfig
>> +++ b/arch/arm/Kconfig
>> @@ -1825,6 +1825,21 @@ config XEN
>> help
>> Say Y if you want to run Linux in a Virtual Machine on Xen on ARM.
>>
>> +config RANDOMIZE_BASE
>> + bool "Randomize the address of the kernel image"
>> + depends on MMU && AUTO_ZRELADDR
>> + depends on !XIP_KERNEL && !ZBOOT_ROM
>
> This should work even if ZBOOT_ROM is selected. The ZBOOT_ROM option
> allows for the decompressor executing directly from ROM and
> decompressing the kernel anywhere in RAM.
>
In principle, yes. But currently, the code is not entirely XIP clean,
i.e., the kaslr_offset and __efi_boot variables are in .text but
assumed to be writable.
For UEFI, this does not really matter, because that implies !XIP.
However, I agree it would be nice if ZBOOT_ROM could run this code as
well, but it requires a global variable. I will try to make that work.
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 25/29] ARM: decompressor: explicitly map decompressor binary cacheable
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (23 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 24/29] ARM: kernel: implement randomization of the kernel load address Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-04 18:47 ` Nicolas Pitre
2017-09-03 12:07 ` [PATCH v2 26/29] ARM: decompressor: add KASLR support Ard Biesheuvel
` (4 subsequent siblings)
29 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
When randomizing the kernel load address, there may be a large
distance in memory between the decompressor binary and its payload
and the destination area in memory. Ensure that the decompressor
itself is mapped cacheable in this case, by tweaking the existing
routine that takes care of this for XIP decompressors.
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/boot/compressed/head.S | 16 ++++++++++------
1 file changed, 10 insertions(+), 6 deletions(-)
diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
index 5884e8151376..583cc6899d98 100644
--- a/arch/arm/boot/compressed/head.S
+++ b/arch/arm/boot/compressed/head.S
@@ -706,20 +706,24 @@ __setup_mmu: sub r3, r4, #16384 @ Page directory size
teq r0, r2
bne 1b
/*
- * If ever we are running from Flash, then we surely want the cache
- * to be enabled also for our execution instance... We map 2MB of it
- * so there is no map overlap problem for up to 1 MB compressed kernel.
- * If the execution is in RAM then we would only be duplicating the above.
+ * Make sure our entire executable image (including payload) is mapped
+ * cacheable, in case it is located outside the region we covered above.
+ * (This may be the case if running from flash or with randomization enabled)
+ * If the regions happen to overlap, we just duplicate some of the above.
*/
orr r1, r6, #0x04 @ ensure B is set for this
orr r1, r1, #3 << 10
mov r2, pc
+ adr_l r9, _end
mov r2, r2, lsr #20
+ mov r9, r9, lsr #20
orr r1, r1, r2, lsl #20
add r0, r3, r2, lsl #2
- str r1, [r0], #4
+ add r9, r3, r9, lsl #2
+0: str r1, [r0], #4
add r1, r1, #1048576
- str r1, [r0]
+ cmp r0, r9
+ bls 0b
mov pc, lr
ENDPROC(__setup_mmu)
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 25/29] ARM: decompressor: explicitly map decompressor binary cacheable
2017-09-03 12:07 ` [PATCH v2 25/29] ARM: decompressor: explicitly map decompressor binary cacheable Ard Biesheuvel
@ 2017-09-04 18:47 ` Nicolas Pitre
0 siblings, 0 replies; 83+ messages in thread
From: Nicolas Pitre @ 2017-09-04 18:47 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
> When randomizing the kernel load address, there may be a large
> distance in memory between the decompressor binary and its payload
> and the destination area in memory. Ensure that the decompressor
> itself is mapped cacheable in this case, by tweaking the existing
> routine that takes care of this for XIP decompressors.
>
> Cc: Russell King <linux@armlinux.org.uk>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
> ---
> arch/arm/boot/compressed/head.S | 16 ++++++++++------
> 1 file changed, 10 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
> index 5884e8151376..583cc6899d98 100644
> --- a/arch/arm/boot/compressed/head.S
> +++ b/arch/arm/boot/compressed/head.S
> @@ -706,20 +706,24 @@ __setup_mmu: sub r3, r4, #16384 @ Page directory size
> teq r0, r2
> bne 1b
> /*
> - * If ever we are running from Flash, then we surely want the cache
> - * to be enabled also for our execution instance... We map 2MB of it
> - * so there is no map overlap problem for up to 1 MB compressed kernel.
> - * If the execution is in RAM then we would only be duplicating the above.
> + * Make sure our entire executable image (including payload) is mapped
> + * cacheable, in case it is located outside the region we covered above.
> + * (This may be the case if running from flash or with randomization enabled)
> + * If the regions happen to overlap, we just duplicate some of the above.
> */
> orr r1, r6, #0x04 @ ensure B is set for this
> orr r1, r1, #3 << 10
> mov r2, pc
> + adr_l r9, _end
> mov r2, r2, lsr #20
> + mov r9, r9, lsr #20
> orr r1, r1, r2, lsl #20
> add r0, r3, r2, lsl #2
> - str r1, [r0], #4
> + add r9, r3, r9, lsl #2
> +0: str r1, [r0], #4
> add r1, r1, #1048576
> - str r1, [r0]
> + cmp r0, r9
> + bls 0b
> mov pc, lr
> ENDPROC(__setup_mmu)
>
> --
> 2.11.0
>
>
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 26/29] ARM: decompressor: add KASLR support
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (24 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 25/29] ARM: decompressor: explicitly map decompressor binary cacheable Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-04 18:53 ` Nicolas Pitre
2017-09-03 12:07 ` [PATCH v2 27/29] efi/libstub: add 'max' parameter to efi_random_alloc() Ard Biesheuvel
` (3 subsequent siblings)
29 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
Add support to the decompressor to load the kernel at a randomized
offset, and invoke the kernel proper while passing on the information
about the offset at which the kernel was loaded.
This implementation will extract some pseudo-randomness from the low
bits of the generic timer (if available), and use CRC-16 to combine
it with the build ID string and the device tree binary (which ideally
has a /chosen/kaslr-seed property, but may also have other properties
that differ between boots). This seed is used to select one of the
candidate offsets in the lowmem region that don't overlap the zImage
itself, the DTB, the initrd and /memreserve/s and/or /reserved-memory
nodes that should be left alone.
When booting via the UEFI stub, it is left up to the firmware to supply
a suitable seed and select an offset.
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/boot/compressed/Makefile | 8 +-
arch/arm/boot/compressed/head.S | 84 ++++-
arch/arm/boot/compressed/kaslr.c | 398 ++++++++++++++++++++
3 files changed, 484 insertions(+), 6 deletions(-)
diff --git a/arch/arm/boot/compressed/Makefile b/arch/arm/boot/compressed/Makefile
index d50430c40045..771b1ba1baa3 100644
--- a/arch/arm/boot/compressed/Makefile
+++ b/arch/arm/boot/compressed/Makefile
@@ -85,8 +85,14 @@ $(addprefix $(obj)/,$(libfdt) $(libfdt_hdrs)): $(obj)/%: $(srctree)/scripts/dtc/
$(addprefix $(obj)/,$(libfdt_objs) atags_to_fdt.o): \
$(addprefix $(obj)/,$(libfdt_hdrs))
+ifneq ($(CONFIG_ARM_ATAG_DTB_COMPAT)$(CONFIG_RANDOMIZE_BASE),)
+OBJS += $(libfdt_objs)
ifeq ($(CONFIG_ARM_ATAG_DTB_COMPAT),y)
-OBJS += $(libfdt_objs) atags_to_fdt.o
+OBJS += atags_to_fdt.o
+endif
+ifeq ($(CONFIG_RANDOMIZE_BASE),y)
+OBJS += kaslr.o
+endif
endif
targets := vmlinux vmlinux.lds piggy_data piggy.o \
diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
index 583cc6899d98..79b4033b0ed4 100644
--- a/arch/arm/boot/compressed/head.S
+++ b/arch/arm/boot/compressed/head.S
@@ -381,6 +381,42 @@ restart: adr r0, LC0
dtb_check_done:
#endif
+#ifdef CONFIG_RANDOMIZE_BASE
+ ldr r1, kaslr_offset @ check if the kaslr_offset is
+ cmp r1, #0 @ already set
+#ifdef CONFIG_EFI_STUB
+ ldreq r1, __efi_boot @ UEFI has its own KASLR init
+ cmpeq r1, #0 @ routine so skip over this one
+#endif
+ bne 1f
+
+ stmfd sp!, {r0-r3, ip, lr}
+ adr_l r2, _text @ start of zImage
+ stmfd sp!, {r2, r8, r10} @ pass stack arguments
+
+#if defined(CONFIG_CPU_V6) || defined(CONFIG_CPU_V6K) || defined(CONFIG_CPU_V7)
+ /*
+ * Get some pseudo-entropy from the low bits of the generic
+ * timer if it is implemented.
+ */
+ mrc p15, 0, r1, c0, c1, 1 @ read ID_PFR1 register
+ tst r1, #0x10000 @ have generic timer?
+ beq 0f
+ mrrc p15, 1, r3, r1, c14 @ read CNTVCT
+#endif
+0: adr_l r0, kaslr_offset @ pass &kaslr_offset in r0
+ mov r1, r4 @ pass base address
+ mov r2, r9 @ pass decompressed image size
+ eor r3, r3, r3, ror #16 @ pass initial seed
+ bl kaslr_early_init
+ add sp, sp, #12
+ cmp r0, #0
+ addne r4, r4, r0 @ add offset to base address
+ ldmfd sp!, {r0-r3, ip, lr}
+ bne restart
+1:
+#endif
+
/*
* Check to see if we will overwrite ourselves.
* r4 = final kernel address (possibly with LSB set)
@@ -1356,25 +1392,63 @@ __hyp_reentry_vectors:
__enter_kernel:
mov r0, #0 @ must be 0
+#ifdef CONFIG_RANDOMIZE_BASE
+ ldr r3, kaslr_offset
+ add r4, r4, #4 @ skip first instruction
+#endif
ARM( mov pc, r4 ) @ call kernel
M_CLASS( add r4, r4, #1 ) @ enter in Thumb mode for M class
THUMB( bx r4 ) @ entry point is always ARM for A/R classes
+#ifdef CONFIG_RANDOMIZE_BASE
+ /*
+ * Minimal implementation of CRC-16 that does not use a
+ * lookup table and uses 32-bit wide loads, so it still
+ * performs reasonably well with the D-cache off. Equivalent
+ * to lib/crc16.c for input sizes that are 4 byte multiples.
+ */
+ENTRY(__crc16)
+ push {r4, lr}
+ ldr r3, =0xa001 @ CRC-16 polynomial
+0: subs r2, r2, #4
+ popmi {r4, pc}
+ ldr r4, [r1], #4
+#ifdef __ARMEB__
+ eor ip, r4, r4, ror #16 @ endian swap
+ bic ip, ip, #0x00ff0000
+ mov r4, r4, ror #8
+ eor r4, r4, ip, lsr #8
+#endif
+ eor r0, r0, r4
+ .rept 32
+ lsrs r0, r0, #1
+ eorcs r0, r0, r3
+ .endr
+ b 0b
+ENDPROC(__crc16)
+
+ .align 2
+kaslr_offset: .long 0
+#ifdef CONFIG_EFI_STUB
+__efi_boot: .long 0
+#endif
+#endif
+
reloc_code_end:
#ifdef CONFIG_EFI_STUB
.align 2
-_start: .long start - .
-
ENTRY(efi_stub_entry)
@ allocate space on stack for passing current zImage address
@ and for the EFI stub to return of new entry point of
@ zImage, as EFI stub may copy the kernel. Pointer address
@ is passed in r2. r0 and r1 are passed through from the
@ EFI firmware to efi_entry
- adr ip, _start
- ldr r3, [ip]
- add r3, r3, ip
+#ifdef CONFIG_RANDOMIZE_BASE
+ adr r3, __efi_boot
+ str r3, [r3] @ set __efi_boot != 0
+#endif
+ adr_l r3, start
stmfd sp!, {r3, lr}
mov r2, sp @ pass zImage address in r2
bl efi_entry
diff --git a/arch/arm/boot/compressed/kaslr.c b/arch/arm/boot/compressed/kaslr.c
new file mode 100644
index 000000000000..c68bdea764b6
--- /dev/null
+++ b/arch/arm/boot/compressed/kaslr.c
@@ -0,0 +1,398 @@
+/*
+ * Copyright (C) 2017 Linaro Ltd; <ard.biesheuvel@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#include <libfdt.h>
+#include <linux/types.h>
+#include <generated/compile.h>
+#include <generated/utsrelease.h>
+#include <asm/pgtable.h>
+
+struct regions {
+ u32 pa_start;
+ u32 pa_end;
+ u32 image_size;
+ u32 zimage_start;
+ u32 zimage_size;
+ u32 dtb_start;
+ u32 dtb_size;
+ u32 initrd_start;
+ u32 initrd_size;
+ int reserved_mem;
+ int reserved_mem_addr_cells;
+ int reserved_mem_size_cells;
+};
+
+extern u32 __crc16(u32 crc, u32 const input[], int byte_count);
+
+static u32 __memparse(const char *val, const char **retptr)
+{
+ int base = 10;
+ u32 ret = 0;
+
+ if (*val == '0') {
+ val++;
+ if (*val == 'x' || *val == 'X') {
+ val++;
+ base = 16;
+ } else {
+ base = 8;
+ }
+ }
+
+ while (*val != ',' && *val != ' ' && *val != '\0') {
+ char c = *val++;
+
+ switch (c) {
+ case '0' ... '9':
+ ret = ret * base + (c - '0');
+ continue;
+ case 'a' ... 'f':
+ ret = ret * base + (c - 'a' + 10);
+ continue;
+ case 'A' ... 'F':
+ ret = ret * base + (c - 'A' + 10);
+ continue;
+ case 'g':
+ case 'G':
+ ret <<= 10;
+ case 'm':
+ case 'M':
+ ret <<= 10;
+ case 'k':
+ case 'K':
+ ret <<= 10;
+ break;
+ default:
+ if (retptr)
+ *retptr = NULL;
+ return 0;
+ }
+ }
+ if (retptr)
+ *retptr = val;
+ return ret;
+}
+
+static bool regions_intersect(u32 s1, u32 e1, u32 s2, u32 e2)
+{
+ return e1 >= s2 && e2 >= s1;
+}
+
+static bool intersects_reserved_region(const void *fdt, u32 start,
+ u32 end, struct regions *regions)
+{
+ int subnode, len, i;
+ u64 base, size;
+
+ /* check for overlap with /memreserve/ entries */
+ for (i = 0; i < fdt_num_mem_rsv(fdt); i++) {
+ if (fdt_get_mem_rsv(fdt, i, &base, &size) < 0)
+ continue;
+ if (regions_intersect(start, end, base, base + size))
+ return true;
+ }
+
+ if (regions->reserved_mem < 0)
+ return false;
+
+ /* check for overlap with static reservations in /reserved-memory */
+ for (subnode = fdt_first_subnode(fdt, regions->reserved_mem);
+ subnode >= 0;
+ subnode = fdt_next_subnode(fdt, subnode)) {
+ const fdt32_t *reg;
+
+ len = 0;
+ reg = fdt_getprop(fdt, subnode, "reg", &len);
+ while (len >= (regions->reserved_mem_addr_cells +
+ regions->reserved_mem_size_cells)) {
+
+ base = fdt32_to_cpu(reg[0]);
+ if (regions->reserved_mem_addr_cells == 2)
+ base = (base << 32) | fdt32_to_cpu(reg[1]);
+
+ reg += regions->reserved_mem_addr_cells;
+ len -= 4 * regions->reserved_mem_addr_cells;
+
+ size = fdt32_to_cpu(reg[0]);
+ if (regions->reserved_mem_size_cells == 2)
+ size = (size << 32) | fdt32_to_cpu(reg[1]);
+
+ reg += regions->reserved_mem_size_cells;
+ len -= 4 * regions->reserved_mem_size_cells;
+
+ if (base >= regions->pa_end)
+ continue;
+
+ if (regions_intersect(start, end, base,
+ min(base + size, (u64)U32_MAX)))
+ return true;
+ }
+ }
+ return false;
+}
+
+static bool intersects_occupied_region(const void *fdt, u32 start,
+ u32 end, struct regions *regions)
+{
+ if (regions_intersect(start, end, regions->zimage_start,
+ regions->zimage_start + regions->zimage_size))
+ return true;
+
+ if (regions_intersect(start, end, regions->initrd_start,
+ regions->initrd_start + regions->initrd_size))
+ return true;
+
+ if (regions_intersect(start, end, regions->dtb_start,
+ regions->dtb_start + regions->dtb_size))
+ return true;
+
+ return intersects_reserved_region(fdt, start, end, regions);
+}
+
+static u32 count_suitable_regions(const void *fdt, struct regions *regions)
+{
+ u32 pa, ret = 0;
+
+ for (pa = regions->pa_start; pa < regions->pa_end; pa += SZ_2M) {
+ if (!intersects_occupied_region(fdt, pa,
+ pa + regions->image_size,
+ regions))
+ ret++;
+ }
+ return ret;
+}
+
+static u32 get_numbered_region(const void *fdt,
+ struct regions *regions,
+ int num)
+{
+ u32 pa;
+
+ for (pa = regions->pa_start; pa < regions->pa_end; pa += SZ_2M) {
+ if (!intersects_occupied_region(fdt, pa,
+ pa + regions->image_size,
+ regions))
+ if (num-- == 0)
+ return pa;
+ }
+ return regions->pa_start; /* should not happen */
+}
+
+static void get_cell_sizes(const void *fdt, int node, int *addr_cells,
+ int *size_cells)
+{
+ const int *prop;
+ int len;
+
+ /*
+ * Retrieve the #address-cells and #size-cells properties
+ * from the 'node', or use the default if not provided.
+ */
+ *addr_cells = *size_cells = 1;
+
+ prop = fdt_getprop(fdt, node, "#address-cells", &len);
+ if (len == 4)
+ *addr_cells = fdt32_to_cpu(*prop);
+ prop = fdt_getprop(fdt, node, "#size-cells", &len);
+ if (len == 4)
+ *size_cells = fdt32_to_cpu(*prop);
+}
+
+static u32 get_memory_end(const void *fdt)
+{
+ int mem_node, address_cells, size_cells, len;
+ const fdt32_t *reg;
+ u64 memory_end = 0;
+
+ /* Look for a node called "memory" at the lowest level of the tree */
+ mem_node = fdt_path_offset(fdt, "/memory");
+ if (mem_node <= 0)
+ return 0;
+
+ get_cell_sizes(fdt, 0, &address_cells, &size_cells);
+
+ /*
+ * Now find the 'reg' property of the /memory node, and iterate over
+ * the base/size pairs.
+ */
+ len = 0;
+ reg = fdt_getprop(fdt, mem_node, "reg", &len);
+ while (len >= 4 * (address_cells + size_cells)) {
+ u64 base, size;
+
+ base = fdt32_to_cpu(reg[0]);
+ if (address_cells == 2)
+ base = (base << 32) | fdt32_to_cpu(reg[1]);
+
+ reg += address_cells;
+ len -= 4 * address_cells;
+
+ size = fdt32_to_cpu(reg[0]);
+ if (size_cells == 2)
+ size = (size << 32) | fdt32_to_cpu(reg[1]);
+
+ reg += size_cells;
+ len -= 4 * size_cells;
+
+ memory_end = max(memory_end, base + size);
+ }
+ return min(memory_end, (u64)U32_MAX);
+}
+
+static char *__strstr(const char *s1, const char *s2, int l2)
+{
+ int l1;
+
+ l1 = strlen(s1);
+ while (l1 >= l2) {
+ l1--;
+ if (!memcmp(s1, s2, l2))
+ return (char *)s1;
+ s1++;
+ }
+ return NULL;
+}
+
+static const char *get_cmdline_param(const char *cmdline, const char *param,
+ int param_size)
+{
+ static const char default_cmdline[] = CONFIG_CMDLINE;
+ const char *p;
+
+ if (!IS_ENABLED(CONFIG_CMDLINE_FORCE) && cmdline != NULL) {
+ p = __strstr(cmdline, param, param_size);
+ if (p == cmdline ||
+ (p > cmdline && *(p - 1) == ' '))
+ return p;
+ }
+
+ if (IS_ENABLED(CONFIG_CMDLINE_FORCE) ||
+ IS_ENABLED(CONFIG_CMDLINE_EXTEND)) {
+ p = __strstr(default_cmdline, param, param_size);
+ if (p == default_cmdline ||
+ (p > default_cmdline && *(p - 1) == ' '))
+ return p;
+ }
+ return NULL;
+}
+
+u32 kaslr_early_init(u32 *kaslr_offset, u32 image_base, u32 image_size,
+ u32 seed, u32 zimage_start, const void *fdt,
+ u32 zimage_end)
+{
+ static const char __aligned(4) build_id[] = UTS_VERSION UTS_RELEASE;
+ struct regions regions;
+ const char *command_line;
+ const char *p;
+ int chosen, len;
+ u32 lowmem_top, num;
+
+ if (fdt_check_header(fdt))
+ return 0;
+
+ chosen = fdt_path_offset(fdt, "/chosen");
+ if (chosen < 0)
+ return 0;
+
+ command_line = fdt_getprop(fdt, chosen, "bootargs", &len);
+
+ /* check the command line for the presence of 'nokaslr' */
+ p = get_cmdline_param(command_line, "nokaslr", sizeof("nokaslr") - 1);
+ if (p != NULL)
+ return 0;
+
+ /* check the command line for the presence of 'vmalloc=' */
+ p = get_cmdline_param(command_line, "vmalloc=", sizeof("vmalloc=") - 1);
+ if (p != NULL)
+ lowmem_top = VMALLOC_END - __memparse(p + 8, NULL) -
+ VMALLOC_OFFSET;
+ else
+ lowmem_top = VMALLOC_DEFAULT_BASE;
+
+ regions.image_size = round_up(image_size, SZ_2M);
+ regions.pa_start = round_down(image_base, SZ_128M);
+ regions.pa_end = lowmem_top - PAGE_OFFSET + regions.pa_start -
+ regions.image_size;
+ regions.zimage_start = zimage_start;
+ regions.zimage_size = zimage_end - zimage_start;
+ regions.dtb_start = (u32)fdt;
+ regions.dtb_size = fdt_totalsize(fdt);
+
+ /*
+ * Stir up the seed a bit by taking the CRC of the DTB:
+ * hopefully there's a /chosen/kaslr-seed in there.
+ */
+ seed = __crc16(seed, fdt, regions.dtb_size);
+
+ /* stir a bit more using data that changes between builds */
+ seed = __crc16(seed, (u32 *)build_id, sizeof(build_id));
+
+ /* check for initrd on the command line */
+ regions.initrd_start = regions.initrd_size = 0;
+ p = get_cmdline_param(command_line, "initrd=", sizeof("initrd=") - 1);
+ if (p != NULL) {
+ regions.initrd_start = __memparse(p + 7, &p);
+ if (*p++ == ',')
+ regions.initrd_size = __memparse(p, NULL);
+ if (regions.initrd_size == 0)
+ regions.initrd_start = 0;
+ }
+
+ /* ... or in /chosen */
+ if (regions.initrd_size == 0) {
+ const fdt32_t *prop;
+ u64 start = 0, end = 0;
+
+ prop = fdt_getprop(fdt, chosen, "linux,initrd-start", &len);
+ if (prop) {
+ start = fdt32_to_cpu(prop[0]);
+ if (len == 8)
+ start = (start << 32) | fdt32_to_cpu(prop[1]);
+ }
+
+ prop = fdt_getprop(fdt, chosen, "linux,initrd-end", &len);
+ if (prop) {
+ end = fdt32_to_cpu(prop[0]);
+ if (len == 8)
+ end = (end << 32) | fdt32_to_cpu(prop[1]);
+ }
+ if (start != 0 && end != 0 && start < U32_MAX) {
+ regions.initrd_start = start;
+ regions.initrd_size = max_t(u64, end, U32_MAX) - start;
+ }
+ }
+
+ /* check the memory nodes for the size of the lowmem region */
+ regions.pa_end = min(regions.pa_end, get_memory_end(fdt));
+
+ /* check for a reserved-memory node and record its cell sizes */
+ regions.reserved_mem = fdt_path_offset(fdt, "/reserved-memory");
+ if (regions.reserved_mem >= 0)
+ get_cell_sizes(fdt, regions.reserved_mem,
+ ®ions.reserved_mem_addr_cells,
+ ®ions.reserved_mem_size_cells);
+
+ /*
+ * Iterate over the physical memory range covered by the lowmem region
+ * in 2 MB increments, and count each offset at which we don't overlap
+ * with any of the reserved regions for the zImage itself, the DTB,
+ * the initrd and any regions described as reserved in the device tree.
+ * This produces a count, which we will scale by multiplying by a 16-bit
+ * random value and shifting right by 16 places.
+ * Using this random value, we iterate over the physical memory range
+ * again until we counted enough iterations, and return the offset we
+ * ended up at.
+ */
+ num = ((u16)seed * count_suitable_regions(fdt, ®ions)) >> 16;
+
+ *kaslr_offset = get_numbered_region(fdt, ®ions, num) -
+ regions.pa_start;
+
+ return *kaslr_offset;
+}
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 26/29] ARM: decompressor: add KASLR support
2017-09-03 12:07 ` [PATCH v2 26/29] ARM: decompressor: add KASLR support Ard Biesheuvel
@ 2017-09-04 18:53 ` Nicolas Pitre
2017-09-04 19:33 ` Ard Biesheuvel
0 siblings, 1 reply; 83+ messages in thread
From: Nicolas Pitre @ 2017-09-04 18:53 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
> Add support to the decompressor to load the kernel at a randomized
> offset, and invoke the kernel proper while passing on the information
> about the offset at which the kernel was loaded.
>
> This implementation will extract some pseudo-randomness from the low
> bits of the generic timer (if available), and use CRC-16 to combine
> it with the build ID string and the device tree binary (which ideally
> has a /chosen/kaslr-seed property, but may also have other properties
> that differ between boots). This seed is used to select one of the
> candidate offsets in the lowmem region that don't overlap the zImage
> itself, the DTB, the initrd and /memreserve/s and/or /reserved-memory
> nodes that should be left alone.
>
> When booting via the UEFI stub, it is left up to the firmware to supply
> a suitable seed and select an offset.
Why did you remove the entropy contribution from general regs upon
entry? That was an easy way to enable KASLR on those platforms not using
DT as they would simply have to put some random value in any of the
regs.
>
> Cc: Russell King <linux@armlinux.org.uk>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> ---
> arch/arm/boot/compressed/Makefile | 8 +-
> arch/arm/boot/compressed/head.S | 84 ++++-
> arch/arm/boot/compressed/kaslr.c | 398 ++++++++++++++++++++
> 3 files changed, 484 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm/boot/compressed/Makefile b/arch/arm/boot/compressed/Makefile
> index d50430c40045..771b1ba1baa3 100644
> --- a/arch/arm/boot/compressed/Makefile
> +++ b/arch/arm/boot/compressed/Makefile
> @@ -85,8 +85,14 @@ $(addprefix $(obj)/,$(libfdt) $(libfdt_hdrs)): $(obj)/%: $(srctree)/scripts/dtc/
> $(addprefix $(obj)/,$(libfdt_objs) atags_to_fdt.o): \
> $(addprefix $(obj)/,$(libfdt_hdrs))
>
> +ifneq ($(CONFIG_ARM_ATAG_DTB_COMPAT)$(CONFIG_RANDOMIZE_BASE),)
> +OBJS += $(libfdt_objs)
> ifeq ($(CONFIG_ARM_ATAG_DTB_COMPAT),y)
> -OBJS += $(libfdt_objs) atags_to_fdt.o
> +OBJS += atags_to_fdt.o
> +endif
> +ifeq ($(CONFIG_RANDOMIZE_BASE),y)
> +OBJS += kaslr.o
> +endif
> endif
>
> targets := vmlinux vmlinux.lds piggy_data piggy.o \
> diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
> index 583cc6899d98..79b4033b0ed4 100644
> --- a/arch/arm/boot/compressed/head.S
> +++ b/arch/arm/boot/compressed/head.S
> @@ -381,6 +381,42 @@ restart: adr r0, LC0
> dtb_check_done:
> #endif
>
> +#ifdef CONFIG_RANDOMIZE_BASE
> + ldr r1, kaslr_offset @ check if the kaslr_offset is
> + cmp r1, #0 @ already set
> +#ifdef CONFIG_EFI_STUB
> + ldreq r1, __efi_boot @ UEFI has its own KASLR init
> + cmpeq r1, #0 @ routine so skip over this one
> +#endif
> + bne 1f
> +
> + stmfd sp!, {r0-r3, ip, lr}
> + adr_l r2, _text @ start of zImage
> + stmfd sp!, {r2, r8, r10} @ pass stack arguments
> +
> +#if defined(CONFIG_CPU_V6) || defined(CONFIG_CPU_V6K) || defined(CONFIG_CPU_V7)
> + /*
> + * Get some pseudo-entropy from the low bits of the generic
> + * timer if it is implemented.
> + */
> + mrc p15, 0, r1, c0, c1, 1 @ read ID_PFR1 register
> + tst r1, #0x10000 @ have generic timer?
> + beq 0f
> + mrrc p15, 1, r3, r1, c14 @ read CNTVCT
> +#endif
> +0: adr_l r0, kaslr_offset @ pass &kaslr_offset in r0
> + mov r1, r4 @ pass base address
> + mov r2, r9 @ pass decompressed image size
> + eor r3, r3, r3, ror #16 @ pass initial seed
> + bl kaslr_early_init
> + add sp, sp, #12
> + cmp r0, #0
> + addne r4, r4, r0 @ add offset to base address
> + ldmfd sp!, {r0-r3, ip, lr}
> + bne restart
> +1:
> +#endif
> +
> /*
> * Check to see if we will overwrite ourselves.
> * r4 = final kernel address (possibly with LSB set)
> @@ -1356,25 +1392,63 @@ __hyp_reentry_vectors:
>
> __enter_kernel:
> mov r0, #0 @ must be 0
> +#ifdef CONFIG_RANDOMIZE_BASE
> + ldr r3, kaslr_offset
> + add r4, r4, #4 @ skip first instruction
> +#endif
> ARM( mov pc, r4 ) @ call kernel
> M_CLASS( add r4, r4, #1 ) @ enter in Thumb mode for M class
> THUMB( bx r4 ) @ entry point is always ARM for A/R classes
>
> +#ifdef CONFIG_RANDOMIZE_BASE
> + /*
> + * Minimal implementation of CRC-16 that does not use a
> + * lookup table and uses 32-bit wide loads, so it still
> + * performs reasonably well with the D-cache off. Equivalent
> + * to lib/crc16.c for input sizes that are 4 byte multiples.
> + */
> +ENTRY(__crc16)
> + push {r4, lr}
> + ldr r3, =0xa001 @ CRC-16 polynomial
> +0: subs r2, r2, #4
> + popmi {r4, pc}
> + ldr r4, [r1], #4
> +#ifdef __ARMEB__
> + eor ip, r4, r4, ror #16 @ endian swap
> + bic ip, ip, #0x00ff0000
> + mov r4, r4, ror #8
> + eor r4, r4, ip, lsr #8
> +#endif
> + eor r0, r0, r4
> + .rept 32
> + lsrs r0, r0, #1
> + eorcs r0, r0, r3
> + .endr
> + b 0b
> +ENDPROC(__crc16)
> +
> + .align 2
> +kaslr_offset: .long 0
> +#ifdef CONFIG_EFI_STUB
> +__efi_boot: .long 0
> +#endif
> +#endif
> +
> reloc_code_end:
>
> #ifdef CONFIG_EFI_STUB
> .align 2
> -_start: .long start - .
> -
> ENTRY(efi_stub_entry)
> @ allocate space on stack for passing current zImage address
> @ and for the EFI stub to return of new entry point of
> @ zImage, as EFI stub may copy the kernel. Pointer address
> @ is passed in r2. r0 and r1 are passed through from the
> @ EFI firmware to efi_entry
> - adr ip, _start
> - ldr r3, [ip]
> - add r3, r3, ip
> +#ifdef CONFIG_RANDOMIZE_BASE
> + adr r3, __efi_boot
> + str r3, [r3] @ set __efi_boot != 0
> +#endif
> + adr_l r3, start
> stmfd sp!, {r3, lr}
> mov r2, sp @ pass zImage address in r2
> bl efi_entry
> diff --git a/arch/arm/boot/compressed/kaslr.c b/arch/arm/boot/compressed/kaslr.c
> new file mode 100644
> index 000000000000..c68bdea764b6
> --- /dev/null
> +++ b/arch/arm/boot/compressed/kaslr.c
> @@ -0,0 +1,398 @@
> +/*
> + * Copyright (C) 2017 Linaro Ltd; <ard.biesheuvel@linaro.org>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +
> +#include <libfdt.h>
> +#include <linux/types.h>
> +#include <generated/compile.h>
> +#include <generated/utsrelease.h>
> +#include <asm/pgtable.h>
> +
> +struct regions {
> + u32 pa_start;
> + u32 pa_end;
> + u32 image_size;
> + u32 zimage_start;
> + u32 zimage_size;
> + u32 dtb_start;
> + u32 dtb_size;
> + u32 initrd_start;
> + u32 initrd_size;
> + int reserved_mem;
> + int reserved_mem_addr_cells;
> + int reserved_mem_size_cells;
> +};
> +
> +extern u32 __crc16(u32 crc, u32 const input[], int byte_count);
> +
> +static u32 __memparse(const char *val, const char **retptr)
> +{
> + int base = 10;
> + u32 ret = 0;
> +
> + if (*val == '0') {
> + val++;
> + if (*val == 'x' || *val == 'X') {
> + val++;
> + base = 16;
> + } else {
> + base = 8;
> + }
> + }
> +
> + while (*val != ',' && *val != ' ' && *val != '\0') {
> + char c = *val++;
> +
> + switch (c) {
> + case '0' ... '9':
> + ret = ret * base + (c - '0');
> + continue;
> + case 'a' ... 'f':
> + ret = ret * base + (c - 'a' + 10);
> + continue;
> + case 'A' ... 'F':
> + ret = ret * base + (c - 'A' + 10);
> + continue;
> + case 'g':
> + case 'G':
> + ret <<= 10;
> + case 'm':
> + case 'M':
> + ret <<= 10;
> + case 'k':
> + case 'K':
> + ret <<= 10;
> + break;
> + default:
> + if (retptr)
> + *retptr = NULL;
> + return 0;
> + }
> + }
> + if (retptr)
> + *retptr = val;
> + return ret;
> +}
> +
> +static bool regions_intersect(u32 s1, u32 e1, u32 s2, u32 e2)
> +{
> + return e1 >= s2 && e2 >= s1;
> +}
> +
> +static bool intersects_reserved_region(const void *fdt, u32 start,
> + u32 end, struct regions *regions)
> +{
> + int subnode, len, i;
> + u64 base, size;
> +
> + /* check for overlap with /memreserve/ entries */
> + for (i = 0; i < fdt_num_mem_rsv(fdt); i++) {
> + if (fdt_get_mem_rsv(fdt, i, &base, &size) < 0)
> + continue;
> + if (regions_intersect(start, end, base, base + size))
> + return true;
> + }
> +
> + if (regions->reserved_mem < 0)
> + return false;
> +
> + /* check for overlap with static reservations in /reserved-memory */
> + for (subnode = fdt_first_subnode(fdt, regions->reserved_mem);
> + subnode >= 0;
> + subnode = fdt_next_subnode(fdt, subnode)) {
> + const fdt32_t *reg;
> +
> + len = 0;
> + reg = fdt_getprop(fdt, subnode, "reg", &len);
> + while (len >= (regions->reserved_mem_addr_cells +
> + regions->reserved_mem_size_cells)) {
> +
> + base = fdt32_to_cpu(reg[0]);
> + if (regions->reserved_mem_addr_cells == 2)
> + base = (base << 32) | fdt32_to_cpu(reg[1]);
> +
> + reg += regions->reserved_mem_addr_cells;
> + len -= 4 * regions->reserved_mem_addr_cells;
> +
> + size = fdt32_to_cpu(reg[0]);
> + if (regions->reserved_mem_size_cells == 2)
> + size = (size << 32) | fdt32_to_cpu(reg[1]);
> +
> + reg += regions->reserved_mem_size_cells;
> + len -= 4 * regions->reserved_mem_size_cells;
> +
> + if (base >= regions->pa_end)
> + continue;
> +
> + if (regions_intersect(start, end, base,
> + min(base + size, (u64)U32_MAX)))
> + return true;
> + }
> + }
> + return false;
> +}
> +
> +static bool intersects_occupied_region(const void *fdt, u32 start,
> + u32 end, struct regions *regions)
> +{
> + if (regions_intersect(start, end, regions->zimage_start,
> + regions->zimage_start + regions->zimage_size))
> + return true;
> +
> + if (regions_intersect(start, end, regions->initrd_start,
> + regions->initrd_start + regions->initrd_size))
> + return true;
> +
> + if (regions_intersect(start, end, regions->dtb_start,
> + regions->dtb_start + regions->dtb_size))
> + return true;
> +
> + return intersects_reserved_region(fdt, start, end, regions);
> +}
> +
> +static u32 count_suitable_regions(const void *fdt, struct regions *regions)
> +{
> + u32 pa, ret = 0;
> +
> + for (pa = regions->pa_start; pa < regions->pa_end; pa += SZ_2M) {
> + if (!intersects_occupied_region(fdt, pa,
> + pa + regions->image_size,
> + regions))
> + ret++;
> + }
> + return ret;
> +}
> +
> +static u32 get_numbered_region(const void *fdt,
> + struct regions *regions,
> + int num)
> +{
> + u32 pa;
> +
> + for (pa = regions->pa_start; pa < regions->pa_end; pa += SZ_2M) {
> + if (!intersects_occupied_region(fdt, pa,
> + pa + regions->image_size,
> + regions))
> + if (num-- == 0)
> + return pa;
> + }
> + return regions->pa_start; /* should not happen */
> +}
> +
> +static void get_cell_sizes(const void *fdt, int node, int *addr_cells,
> + int *size_cells)
> +{
> + const int *prop;
> + int len;
> +
> + /*
> + * Retrieve the #address-cells and #size-cells properties
> + * from the 'node', or use the default if not provided.
> + */
> + *addr_cells = *size_cells = 1;
> +
> + prop = fdt_getprop(fdt, node, "#address-cells", &len);
> + if (len == 4)
> + *addr_cells = fdt32_to_cpu(*prop);
> + prop = fdt_getprop(fdt, node, "#size-cells", &len);
> + if (len == 4)
> + *size_cells = fdt32_to_cpu(*prop);
> +}
> +
> +static u32 get_memory_end(const void *fdt)
> +{
> + int mem_node, address_cells, size_cells, len;
> + const fdt32_t *reg;
> + u64 memory_end = 0;
> +
> + /* Look for a node called "memory" at the lowest level of the tree */
> + mem_node = fdt_path_offset(fdt, "/memory");
> + if (mem_node <= 0)
> + return 0;
> +
> + get_cell_sizes(fdt, 0, &address_cells, &size_cells);
> +
> + /*
> + * Now find the 'reg' property of the /memory node, and iterate over
> + * the base/size pairs.
> + */
> + len = 0;
> + reg = fdt_getprop(fdt, mem_node, "reg", &len);
> + while (len >= 4 * (address_cells + size_cells)) {
> + u64 base, size;
> +
> + base = fdt32_to_cpu(reg[0]);
> + if (address_cells == 2)
> + base = (base << 32) | fdt32_to_cpu(reg[1]);
> +
> + reg += address_cells;
> + len -= 4 * address_cells;
> +
> + size = fdt32_to_cpu(reg[0]);
> + if (size_cells == 2)
> + size = (size << 32) | fdt32_to_cpu(reg[1]);
> +
> + reg += size_cells;
> + len -= 4 * size_cells;
> +
> + memory_end = max(memory_end, base + size);
> + }
> + return min(memory_end, (u64)U32_MAX);
> +}
> +
> +static char *__strstr(const char *s1, const char *s2, int l2)
> +{
> + int l1;
> +
> + l1 = strlen(s1);
> + while (l1 >= l2) {
> + l1--;
> + if (!memcmp(s1, s2, l2))
> + return (char *)s1;
> + s1++;
> + }
> + return NULL;
> +}
> +
> +static const char *get_cmdline_param(const char *cmdline, const char *param,
> + int param_size)
> +{
> + static const char default_cmdline[] = CONFIG_CMDLINE;
> + const char *p;
> +
> + if (!IS_ENABLED(CONFIG_CMDLINE_FORCE) && cmdline != NULL) {
> + p = __strstr(cmdline, param, param_size);
> + if (p == cmdline ||
> + (p > cmdline && *(p - 1) == ' '))
> + return p;
> + }
> +
> + if (IS_ENABLED(CONFIG_CMDLINE_FORCE) ||
> + IS_ENABLED(CONFIG_CMDLINE_EXTEND)) {
> + p = __strstr(default_cmdline, param, param_size);
> + if (p == default_cmdline ||
> + (p > default_cmdline && *(p - 1) == ' '))
> + return p;
> + }
> + return NULL;
> +}
> +
> +u32 kaslr_early_init(u32 *kaslr_offset, u32 image_base, u32 image_size,
> + u32 seed, u32 zimage_start, const void *fdt,
> + u32 zimage_end)
> +{
> + static const char __aligned(4) build_id[] = UTS_VERSION UTS_RELEASE;
> + struct regions regions;
> + const char *command_line;
> + const char *p;
> + int chosen, len;
> + u32 lowmem_top, num;
> +
> + if (fdt_check_header(fdt))
> + return 0;
> +
> + chosen = fdt_path_offset(fdt, "/chosen");
> + if (chosen < 0)
> + return 0;
> +
> + command_line = fdt_getprop(fdt, chosen, "bootargs", &len);
> +
> + /* check the command line for the presence of 'nokaslr' */
> + p = get_cmdline_param(command_line, "nokaslr", sizeof("nokaslr") - 1);
> + if (p != NULL)
> + return 0;
> +
> + /* check the command line for the presence of 'vmalloc=' */
> + p = get_cmdline_param(command_line, "vmalloc=", sizeof("vmalloc=") - 1);
> + if (p != NULL)
> + lowmem_top = VMALLOC_END - __memparse(p + 8, NULL) -
> + VMALLOC_OFFSET;
> + else
> + lowmem_top = VMALLOC_DEFAULT_BASE;
> +
> + regions.image_size = round_up(image_size, SZ_2M);
> + regions.pa_start = round_down(image_base, SZ_128M);
> + regions.pa_end = lowmem_top - PAGE_OFFSET + regions.pa_start -
> + regions.image_size;
> + regions.zimage_start = zimage_start;
> + regions.zimage_size = zimage_end - zimage_start;
> + regions.dtb_start = (u32)fdt;
> + regions.dtb_size = fdt_totalsize(fdt);
> +
> + /*
> + * Stir up the seed a bit by taking the CRC of the DTB:
> + * hopefully there's a /chosen/kaslr-seed in there.
> + */
> + seed = __crc16(seed, fdt, regions.dtb_size);
> +
> + /* stir a bit more using data that changes between builds */
> + seed = __crc16(seed, (u32 *)build_id, sizeof(build_id));
> +
> + /* check for initrd on the command line */
> + regions.initrd_start = regions.initrd_size = 0;
> + p = get_cmdline_param(command_line, "initrd=", sizeof("initrd=") - 1);
> + if (p != NULL) {
> + regions.initrd_start = __memparse(p + 7, &p);
> + if (*p++ == ',')
> + regions.initrd_size = __memparse(p, NULL);
> + if (regions.initrd_size == 0)
> + regions.initrd_start = 0;
> + }
> +
> + /* ... or in /chosen */
> + if (regions.initrd_size == 0) {
> + const fdt32_t *prop;
> + u64 start = 0, end = 0;
> +
> + prop = fdt_getprop(fdt, chosen, "linux,initrd-start", &len);
> + if (prop) {
> + start = fdt32_to_cpu(prop[0]);
> + if (len == 8)
> + start = (start << 32) | fdt32_to_cpu(prop[1]);
> + }
> +
> + prop = fdt_getprop(fdt, chosen, "linux,initrd-end", &len);
> + if (prop) {
> + end = fdt32_to_cpu(prop[0]);
> + if (len == 8)
> + end = (end << 32) | fdt32_to_cpu(prop[1]);
> + }
> + if (start != 0 && end != 0 && start < U32_MAX) {
> + regions.initrd_start = start;
> + regions.initrd_size = max_t(u64, end, U32_MAX) - start;
> + }
> + }
> +
> + /* check the memory nodes for the size of the lowmem region */
> + regions.pa_end = min(regions.pa_end, get_memory_end(fdt));
> +
> + /* check for a reserved-memory node and record its cell sizes */
> + regions.reserved_mem = fdt_path_offset(fdt, "/reserved-memory");
> + if (regions.reserved_mem >= 0)
> + get_cell_sizes(fdt, regions.reserved_mem,
> + ®ions.reserved_mem_addr_cells,
> + ®ions.reserved_mem_size_cells);
> +
> + /*
> + * Iterate over the physical memory range covered by the lowmem region
> + * in 2 MB increments, and count each offset at which we don't overlap
> + * with any of the reserved regions for the zImage itself, the DTB,
> + * the initrd and any regions described as reserved in the device tree.
> + * This produces a count, which we will scale by multiplying by a 16-bit
> + * random value and shifting right by 16 places.
> + * Using this random value, we iterate over the physical memory range
> + * again until we counted enough iterations, and return the offset we
> + * ended up at.
> + */
> + num = ((u16)seed * count_suitable_regions(fdt, ®ions)) >> 16;
> +
> + *kaslr_offset = get_numbered_region(fdt, ®ions, num) -
> + regions.pa_start;
> +
> + return *kaslr_offset;
> +}
> --
> 2.11.0
>
>
^ permalink raw reply [flat|nested] 83+ messages in thread* [PATCH v2 26/29] ARM: decompressor: add KASLR support
2017-09-04 18:53 ` Nicolas Pitre
@ 2017-09-04 19:33 ` Ard Biesheuvel
0 siblings, 0 replies; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-04 19:33 UTC (permalink / raw)
To: linux-arm-kernel
On 4 September 2017 at 19:53, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Sun, 3 Sep 2017, Ard Biesheuvel wrote:
>
>> Add support to the decompressor to load the kernel at a randomized
>> offset, and invoke the kernel proper while passing on the information
>> about the offset at which the kernel was loaded.
>>
>> This implementation will extract some pseudo-randomness from the low
>> bits of the generic timer (if available), and use CRC-16 to combine
>> it with the build ID string and the device tree binary (which ideally
>> has a /chosen/kaslr-seed property, but may also have other properties
>> that differ between boots). This seed is used to select one of the
>> candidate offsets in the lowmem region that don't overlap the zImage
>> itself, the DTB, the initrd and /memreserve/s and/or /reserved-memory
>> nodes that should be left alone.
>>
>> When booting via the UEFI stub, it is left up to the firmware to supply
>> a suitable seed and select an offset.
>
> Why did you remove the entropy contribution from general regs upon
> entry? That was an easy way to enable KASLR on those platforms not using
> DT as they would simply have to put some random value in any of the
> regs.
>
I simply did not consider non-DTB, to be honest. I implemented the
suggestion to stir the entropy using some build time data and the DTB
contents, and given that the kaslr entry code parses the command line
and other DTB structures (/memreserve/, /reserved-memory,
/chosen/initrd-xxx) to decide where to uncompress the kernel, the
thought never crossed my mind to add support for !DTB as well. And to
be perfectly honest, I'd rather rely on the ATAG to DTB compat code so
I don't have to implement an ATAGS version of kaslr_early_init()
Of course, that still does not rule out eor'ing all the register
together instead of only r3, so I will add that back.
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 27/29] efi/libstub: add 'max' parameter to efi_random_alloc()
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (25 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 26/29] ARM: decompressor: add KASLR support Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-03 12:07 ` [PATCH v2 28/29] efi/libstub: check for vmalloc= command line argument Ard Biesheuvel
` (2 subsequent siblings)
29 siblings, 0 replies; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
Add an upper limit to efi_random_alloc() so we can use it to randomly
allocate the ARM kernel in lowmem.
Cc: Matt Fleming <matt@codeblueprint.co.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
drivers/firmware/efi/libstub/arm64-stub.c | 2 +-
drivers/firmware/efi/libstub/efistub.h | 3 ++-
drivers/firmware/efi/libstub/random.c | 11 +++++++----
3 files changed, 10 insertions(+), 6 deletions(-)
diff --git a/drivers/firmware/efi/libstub/arm64-stub.c b/drivers/firmware/efi/libstub/arm64-stub.c
index b4c2589d7c91..940766f90adb 100644
--- a/drivers/firmware/efi/libstub/arm64-stub.c
+++ b/drivers/firmware/efi/libstub/arm64-stub.c
@@ -94,7 +94,7 @@ efi_status_t handle_kernel_image(efi_system_table_t *sys_table_arg,
*reserve_size = kernel_memsize + offset;
status = efi_random_alloc(sys_table_arg, *reserve_size,
MIN_KIMG_ALIGN, reserve_addr,
- (u32)phys_seed);
+ (u32)phys_seed, ULONG_MAX);
*image_addr = *reserve_addr + offset;
} else {
diff --git a/drivers/firmware/efi/libstub/efistub.h b/drivers/firmware/efi/libstub/efistub.h
index 83f268c05007..3a670a5f759f 100644
--- a/drivers/firmware/efi/libstub/efistub.h
+++ b/drivers/firmware/efi/libstub/efistub.h
@@ -60,7 +60,8 @@ efi_status_t efi_get_random_bytes(efi_system_table_t *sys_table,
efi_status_t efi_random_alloc(efi_system_table_t *sys_table_arg,
unsigned long size, unsigned long align,
- unsigned long *addr, unsigned long random_seed);
+ unsigned long *addr, unsigned long random_seed,
+ unsigned long max);
efi_status_t check_platform_features(efi_system_table_t *sys_table_arg);
diff --git a/drivers/firmware/efi/libstub/random.c b/drivers/firmware/efi/libstub/random.c
index 7e72954d5860..810e60e76211 100644
--- a/drivers/firmware/efi/libstub/random.c
+++ b/drivers/firmware/efi/libstub/random.c
@@ -42,7 +42,8 @@ efi_status_t efi_get_random_bytes(efi_system_table_t *sys_table_arg,
*/
static unsigned long get_entry_num_slots(efi_memory_desc_t *md,
unsigned long size,
- unsigned long align_shift)
+ unsigned long align_shift,
+ unsigned long max)
{
unsigned long align = 1UL << align_shift;
u64 first_slot, last_slot, region_end;
@@ -50,7 +51,8 @@ static unsigned long get_entry_num_slots(efi_memory_desc_t *md,
if (md->type != EFI_CONVENTIONAL_MEMORY)
return 0;
- region_end = min((u64)ULONG_MAX, md->phys_addr + md->num_pages*EFI_PAGE_SIZE - 1);
+ region_end = min_t(u64, max, md->phys_addr +
+ md->num_pages * EFI_PAGE_SIZE - 1);
first_slot = round_up(md->phys_addr, align);
last_slot = round_down(region_end - size + 1, align);
@@ -73,7 +75,8 @@ efi_status_t efi_random_alloc(efi_system_table_t *sys_table_arg,
unsigned long size,
unsigned long align,
unsigned long *addr,
- unsigned long random_seed)
+ unsigned long random_seed,
+ unsigned long max)
{
unsigned long map_size, desc_size, total_slots = 0, target_slot;
unsigned long buff_size;
@@ -101,7 +104,7 @@ efi_status_t efi_random_alloc(efi_system_table_t *sys_table_arg,
efi_memory_desc_t *md = (void *)memory_map + map_offset;
unsigned long slots;
- slots = get_entry_num_slots(md, size, ilog2(align));
+ slots = get_entry_num_slots(md, size, ilog2(align), max);
MD_NUM_SLOTS(md) = slots;
total_slots += slots;
}
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 28/29] efi/libstub: check for vmalloc= command line argument
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (26 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 27/29] efi/libstub: add 'max' parameter to efi_random_alloc() Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-03 12:07 ` [PATCH v2 29/29] efi/libstub: arm: implement KASLR Ard Biesheuvel
2017-09-05 16:45 ` [PATCH v2 00/29] implement KASLR for ARM Tony Lindgren
29 siblings, 0 replies; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
Check for and record the presence of a vmalloc= argument on the
kernel command line. We need this information on ARM systems when
implementing KASLR, given that the size of the vmalloc region will
affect the size of the lowmem region, therefore affecting the
available randomization range as well.
Cc: Matt Fleming <matt@codeblueprint.co.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
drivers/firmware/efi/libstub/efi-stub-helper.c | 9 +++++++++
drivers/firmware/efi/libstub/efistub.h | 1 +
2 files changed, 10 insertions(+)
diff --git a/drivers/firmware/efi/libstub/efi-stub-helper.c b/drivers/firmware/efi/libstub/efi-stub-helper.c
index b0184360efc6..f3e9d43030ac 100644
--- a/drivers/firmware/efi/libstub/efi-stub-helper.c
+++ b/drivers/firmware/efi/libstub/efi-stub-helper.c
@@ -34,6 +34,7 @@ static unsigned long __chunk_size = EFI_READ_CHUNK_SIZE;
static int __section(.data) __nokaslr;
static int __section(.data) __quiet;
+static int __section(.data) __vmalloc_arg;
int __pure nokaslr(void)
{
@@ -43,6 +44,10 @@ int __pure is_quiet(void)
{
return __quiet;
}
+int __pure have_vmalloc(void)
+{
+ return __vmalloc_arg;
+}
#define EFI_MMAP_NR_SLACK_SLOTS 8
@@ -433,6 +438,10 @@ efi_status_t efi_parse_options(char const *cmdline)
if (str == cmdline || (str && str > cmdline && *(str - 1) == ' '))
__quiet = 1;
+ str = strstr(cmdline, "vmalloc=");
+ if (str == cmdline || (str && str > cmdline && *(str - 1) == ' '))
+ __vmalloc_arg = 1;
+
/*
* If no EFI parameters were specified on the cmdline we've got
* nothing to do.
diff --git a/drivers/firmware/efi/libstub/efistub.h b/drivers/firmware/efi/libstub/efistub.h
index 3a670a5f759f..aaf2aeb785ea 100644
--- a/drivers/firmware/efi/libstub/efistub.h
+++ b/drivers/firmware/efi/libstub/efistub.h
@@ -26,6 +26,7 @@
extern int __pure nokaslr(void);
extern int __pure is_quiet(void);
+extern int __pure have_vmalloc(void);
#define pr_efi(sys_table, msg) do { \
if (!is_quiet()) efi_printk(sys_table, "EFI stub: "msg); \
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 29/29] efi/libstub: arm: implement KASLR
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (27 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 28/29] efi/libstub: check for vmalloc= command line argument Ard Biesheuvel
@ 2017-09-03 12:07 ` Ard Biesheuvel
2017-09-05 16:45 ` [PATCH v2 00/29] implement KASLR for ARM Tony Lindgren
29 siblings, 0 replies; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-03 12:07 UTC (permalink / raw)
To: linux-arm-kernel
This wires up the new KASLR implementation for ARM to the random number
generator and memory allocation routines in the UEFI stub.
Given how the UEFI stub keeps track of the placement of the DTB and
potentially an initrd via its memory map, we can quite simply use
efi_random_alloc() to carve out a window for the kernel proper, and
inform the decompressor about this by setting the kaslr_offset variable
directly.
Since the presence of a vmalloc= command line option complicates the
calculations involved, let's just disable KASLR for now if a vmalloc=
command line argument was provided.
Cc: Matt Fleming <matt@codeblueprint.co.uk>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/boot/compressed/head.S | 12 ++++-
drivers/firmware/efi/libstub/arm32-stub.c | 47 +++++++++++++++++++-
2 files changed, 56 insertions(+), 3 deletions(-)
diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
index 79b4033b0ed4..68d803dfdf3f 100644
--- a/arch/arm/boot/compressed/head.S
+++ b/arch/arm/boot/compressed/head.S
@@ -203,6 +203,15 @@ not_angel:
*/
mov r4, pc
and r4, r4, #0xf8000000
+#if defined(CONFIG_RANDOMIZE_BASE) && defined(CONFIG_EFI_STUB)
+ /*
+ * The KASLR randomization may have already been performed by
+ * the UEFI stub, in which case kaslr_offset will already have
+ * a value.
+ */
+ ldr r0, kaslr_offset
+ add r4, r4, r0
+#endif
/* Determine final kernel image address. */
add r4, r4, #TEXT_OFFSET
#else
@@ -1428,7 +1437,8 @@ ENTRY(__crc16)
ENDPROC(__crc16)
.align 2
-kaslr_offset: .long 0
+ENTRY(kaslr_offset)
+ .long 0
#ifdef CONFIG_EFI_STUB
__efi_boot: .long 0
#endif
diff --git a/drivers/firmware/efi/libstub/arm32-stub.c b/drivers/firmware/efi/libstub/arm32-stub.c
index becbda445913..73d3694d2fbb 100644
--- a/drivers/firmware/efi/libstub/arm32-stub.c
+++ b/drivers/firmware/efi/libstub/arm32-stub.c
@@ -8,9 +8,12 @@
*/
#include <linux/efi.h>
#include <asm/efi.h>
+#include <asm/pgtable.h>
#include "efistub.h"
+extern u32 kaslr_offset;
+
efi_status_t check_platform_features(efi_system_table_t *sys_table_arg)
{
int block;
@@ -200,6 +203,29 @@ efi_status_t handle_kernel_image(efi_system_table_t *sys_table,
efi_loaded_image_t *image)
{
efi_status_t status;
+ u32 phys_seed = 0;
+
+ if (IS_ENABLED(CONFIG_RANDOMIZE_BASE)) {
+ if (have_vmalloc()) {
+ pr_efi(sys_table,
+ "vmalloc= command line argument found, disabling KASLR\n");
+ } else if (!nokaslr()) {
+ status = efi_get_random_bytes(sys_table,
+ sizeof(phys_seed),
+ (u8 *)&phys_seed);
+ if (status == EFI_NOT_FOUND) {
+ pr_efi(sys_table,
+ "EFI_RNG_PROTOCOL unavailable, no randomness supplied\n");
+ } else if (status != EFI_SUCCESS) {
+ pr_efi_err(sys_table,
+ "efi_get_random_bytes() failed\n");
+ return status;
+ }
+ } else {
+ pr_efi(sys_table,
+ "KASLR disabled on kernel command line\n");
+ }
+ }
/*
* Verify that the DRAM base address is compatible with the ARM
@@ -210,8 +236,25 @@ efi_status_t handle_kernel_image(efi_system_table_t *sys_table,
*/
dram_base = round_up(dram_base, SZ_128M);
- status = reserve_kernel_base(sys_table, dram_base, reserve_addr,
- reserve_size);
+ if (!IS_ENABLED(CONFIG_RANDOMIZE_BASE) || phys_seed == 0) {
+ status = reserve_kernel_base(sys_table, dram_base, reserve_addr,
+ reserve_size);
+ } else {
+ /* the end of the lowmem region */
+ unsigned long max = dram_base + VMALLOC_DEFAULT_BASE
+ - PAGE_OFFSET - 1;
+ /*
+ * The DTB and initrd are covered by allocations in the UEFI
+ * memory map, so we can create a random allocation for the
+ * uncompressed kernel, and inform the decompressor about the
+ * offset with respect to the base of memory.
+ */
+ *reserve_size = MAX_UNCOMP_KERNEL_SIZE;
+ status = efi_random_alloc(sys_table, *reserve_size, SZ_2M,
+ reserve_addr, phys_seed, max);
+ kaslr_offset = *reserve_addr - dram_base;
+ }
+
if (status != EFI_SUCCESS) {
pr_efi_err(sys_table, "Unable to allocate memory for uncompressed kernel.\n");
return status;
--
2.11.0
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 00/29] implement KASLR for ARM
2017-09-03 12:07 [PATCH v2 00/29] implement KASLR for ARM Ard Biesheuvel
` (28 preceding siblings ...)
2017-09-03 12:07 ` [PATCH v2 29/29] efi/libstub: arm: implement KASLR Ard Biesheuvel
@ 2017-09-05 16:45 ` Tony Lindgren
2017-09-05 16:48 ` Ard Biesheuvel
29 siblings, 1 reply; 83+ messages in thread
From: Tony Lindgren @ 2017-09-05 16:45 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
* Ard Biesheuvel <ard.biesheuvel@linaro.org> [170903 05:08]:
> This series implements randomization of the placement of the core ARM kernel
> inside the lowmem region. It consists of the following parts:
>
> - changes that allow us to build vmlinux as a PIE executable which retains
> the metadata required to fix up all absolute symbol references at runtime
> - changes that eliminate absolute references from low-level code that may
> execute with the MMU off: this removes the need to perform explicit cache
> maintenance after the absolute references have been fixed up at runtime with
> the caches enabled
> - changes to the core kernel startup code to take the physical offset into
> account when creating the virtual mapping (the pa-to-va mapping remains
> unchanged)
> - changes to the decompressor to collect some pseudo-entropy, and randomize
> the physical offset of the decompressed kernel, taking placement of DTB,
> initrd and reserved regions into account
> - changes to the UEFI stub code to choose the KASLR offset and communicate
> it to the decompressor
>
> To test these changes, boot a multi_v7_defconfig+CONFIG_RANDOMIZE_BASE=y
I gave a quick try using your arm-kaslr-v3 branch, hopefully that's
the right one.
The good news is that now omap3 boots with omap2plus_defconfig with
and without CONFIG_RANDOMIZE_BASE=y and I did not see any compiler
errors with my gcc 6.2.0 like earlier :)
I did see boot attempts fail with randomize enable where no output
was produced. It seems this is happening for me maybe 1 out of 5 boots.
Enabling DEBUG_LL did not show anything either.
Then loading modules with CONFIG_RANDOMIZE_BASE=y seems to fail with:
$ sudo modprobe rtc-twl
rtc_twl: disagrees about version of symbol module_layout
modprobe: ERROR: could not insert 'rtc_twl': Exec format error
Regards,
Tony
^ permalink raw reply [flat|nested] 83+ messages in thread* [PATCH v2 00/29] implement KASLR for ARM
2017-09-05 16:45 ` [PATCH v2 00/29] implement KASLR for ARM Tony Lindgren
@ 2017-09-05 16:48 ` Ard Biesheuvel
2017-09-05 19:37 ` Tony Lindgren
0 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-05 16:48 UTC (permalink / raw)
To: linux-arm-kernel
On 5 September 2017 at 17:45, Tony Lindgren <tony@atomide.com> wrote:
> Hi,
>
> * Ard Biesheuvel <ard.biesheuvel@linaro.org> [170903 05:08]:
>> This series implements randomization of the placement of the core ARM kernel
>> inside the lowmem region. It consists of the following parts:
>>
>> - changes that allow us to build vmlinux as a PIE executable which retains
>> the metadata required to fix up all absolute symbol references at runtime
>> - changes that eliminate absolute references from low-level code that may
>> execute with the MMU off: this removes the need to perform explicit cache
>> maintenance after the absolute references have been fixed up at runtime with
>> the caches enabled
>> - changes to the core kernel startup code to take the physical offset into
>> account when creating the virtual mapping (the pa-to-va mapping remains
>> unchanged)
>> - changes to the decompressor to collect some pseudo-entropy, and randomize
>> the physical offset of the decompressed kernel, taking placement of DTB,
>> initrd and reserved regions into account
>> - changes to the UEFI stub code to choose the KASLR offset and communicate
>> it to the decompressor
>>
>> To test these changes, boot a multi_v7_defconfig+CONFIG_RANDOMIZE_BASE=y
>
> I gave a quick try using your arm-kaslr-v3 branch, hopefully that's
> the right one.
>
> The good news is that now omap3 boots with omap2plus_defconfig with
> and without CONFIG_RANDOMIZE_BASE=y and I did not see any compiler
> errors with my gcc 6.2.0 like earlier :)
>
Good! Thanks a lot for taking the time to test this stuff.
> I did see boot attempts fail with randomize enable where no output
> was produced. It seems this is happening for me maybe 1 out of 5 boots.
> Enabling DEBUG_LL did not show anything either.
>
Yes. I am looking into a couple of kernelci boot reports that look
suspicious, but it is rather difficult to reproduce, for obvious
reasons :-)
Which hardware are you testing this on?
> Then loading modules with CONFIG_RANDOMIZE_BASE=y seems to fail with:
>
> $ sudo modprobe rtc-twl
> rtc_twl: disagrees about version of symbol module_layout
> modprobe: ERROR: could not insert 'rtc_twl': Exec format error
>
Is this with CONFIG_MODVERSIONS enabled?
Thanks,
Ard.
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 00/29] implement KASLR for ARM
2017-09-05 16:48 ` Ard Biesheuvel
@ 2017-09-05 19:37 ` Tony Lindgren
2017-09-05 19:42 ` Ard Biesheuvel
0 siblings, 1 reply; 83+ messages in thread
From: Tony Lindgren @ 2017-09-05 19:37 UTC (permalink / raw)
To: linux-arm-kernel
* Ard Biesheuvel <ard.biesheuvel@linaro.org> [170905 09:49]:
> On 5 September 2017 at 17:45, Tony Lindgren <tony@atomide.com> wrote:
> > I did see boot attempts fail with randomize enable where no output
> > was produced. It seems this is happening for me maybe 1 out of 5 boots.
> > Enabling DEBUG_LL did not show anything either.
> >
>
> Yes. I am looking into a couple of kernelci boot reports that look
> suspicious, but it is rather difficult to reproduce, for obvious
> reasons :-)
>
> Which hardware are you testing this on?
For testing on omap3, I'm mostly using logicpd torpedo devkit as
it works out of box with PM with mainline Linux and has NFSroot
usable too.
> > Then loading modules with CONFIG_RANDOMIZE_BASE=y seems to fail with:
> >
> > $ sudo modprobe rtc-twl
> > rtc_twl: disagrees about version of symbol module_layout
> > modprobe: ERROR: could not insert 'rtc_twl': Exec format error
> >
>
> Is this with CONFIG_MODVERSIONS enabled?
Yes, but disabling that did not seem to make any difference
based on just one test.
Regards,
Tony
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 00/29] implement KASLR for ARM
2017-09-05 19:37 ` Tony Lindgren
@ 2017-09-05 19:42 ` Ard Biesheuvel
2017-09-05 21:27 ` Tony Lindgren
0 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-05 19:42 UTC (permalink / raw)
To: linux-arm-kernel
On 5 September 2017 at 20:37, Tony Lindgren <tony@atomide.com> wrote:
> * Ard Biesheuvel <ard.biesheuvel@linaro.org> [170905 09:49]:
>> On 5 September 2017 at 17:45, Tony Lindgren <tony@atomide.com> wrote:
>> > I did see boot attempts fail with randomize enable where no output
>> > was produced. It seems this is happening for me maybe 1 out of 5 boots.
>> > Enabling DEBUG_LL did not show anything either.
>> >
>>
>> Yes. I am looking into a couple of kernelci boot reports that look
>> suspicious, but it is rather difficult to reproduce, for obvious
>> reasons :-)
>>
>> Which hardware are you testing this on?
>
> For testing on omap3, I'm mostly using logicpd torpedo devkit as
> it works out of box with PM with mainline Linux and has NFSroot
> usable too.
>
Right. Well, I will try to reproduce with the BB white I have.
Are you booting with an initrd?
>> > Then loading modules with CONFIG_RANDOMIZE_BASE=y seems to fail with:
>> >
>> > $ sudo modprobe rtc-twl
>> > rtc_twl: disagrees about version of symbol module_layout
>> > modprobe: ERROR: could not insert 'rtc_twl': Exec format error
>> >
>>
>> Is this with CONFIG_MODVERSIONS enabled?
>
> Yes, but disabling that did not seem to make any difference
> based on just one test.
>
Yeah, well, it appears I missed a couple of details :-)
This should fix the module loading issues:
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 1a0304dd388d..bbefd5f32ec2 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -1830,6 +1830,8 @@ config RANDOMIZE_BASE
depends on MMU && AUTO_ZRELADDR
depends on !XIP_KERNEL && !ZBOOT_ROM
select RELOCATABLE
+ select ARM_MODULE_PLTS if MODULES
+ select MODULE_REL_CRCS if MODVERSIONS
help
Randomizes the virtual and physical address at which the kernel
image is loaded, as a security feature that deters exploit attempts
diff --git a/arch/arm/include/asm/elf.h b/arch/arm/include/asm/elf.h
index f13ae153fb24..b56fc4dd27b6 100644
--- a/arch/arm/include/asm/elf.h
+++ b/arch/arm/include/asm/elf.h
@@ -50,6 +50,7 @@ typedef struct user_fp elf_fpregset_t;
#define R_ARM_NONE 0
#define R_ARM_PC24 1
#define R_ARM_ABS32 2
+#define R_ARM_REL32 3
#define R_ARM_CALL 28
#define R_ARM_JUMP24 29
#define R_ARM_TARGET1 38
diff --git a/arch/arm/kernel/module.c b/arch/arm/kernel/module.c
index 3ff571c2c71c..aa4d72837cd5 100644
--- a/arch/arm/kernel/module.c
+++ b/arch/arm/kernel/module.c
@@ -175,6 +175,10 @@
*(u32 *)loc |= offset & 0x7fffffff;
break;
+ case R_ARM_REL32:
+ *(u32 *)loc += sym->st_value - loc;
+ break;
+
case R_ARM_MOVW_ABS_NC:
case R_ARM_MOVT_ABS:
offset = tmp = __mem_to_opcode_arm(*(u32 *)loc);
^ permalink raw reply related [flat|nested] 83+ messages in thread
* [PATCH v2 00/29] implement KASLR for ARM
2017-09-05 19:42 ` Ard Biesheuvel
@ 2017-09-05 21:27 ` Tony Lindgren
2017-09-05 21:31 ` Ard Biesheuvel
0 siblings, 1 reply; 83+ messages in thread
From: Tony Lindgren @ 2017-09-05 21:27 UTC (permalink / raw)
To: linux-arm-kernel
* Ard Biesheuvel <ard.biesheuvel@linaro.org> [170905 12:43]:
> Right. Well, I will try to reproduce with the BB white I have.
Yeah that should be reproducable, I got it to happen on BBB here
after about 5 boots.
> Are you booting with an initrd?
Not on this one, on beagleboard xm I do.
> >> > Then loading modules with CONFIG_RANDOMIZE_BASE=y seems to fail with:
> >> >
> >> > $ sudo modprobe rtc-twl
> >> > rtc_twl: disagrees about version of symbol module_layout
> >> > modprobe: ERROR: could not insert 'rtc_twl': Exec format error
> >> >
> >>
> >> Is this with CONFIG_MODVERSIONS enabled?
> >
> > Yes, but disabling that did not seem to make any difference
> > based on just one test.
> >
>
> Yeah, well, it appears I missed a couple of details :-)
>
> This should fix the module loading issues:
Yeah now modprobe works :) That's after manually applying
it as the tabs got munched in your patch somewhere, see below.
Regards,
Tony
> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
> index 1a0304dd388d..bbefd5f32ec2 100644
> --- a/arch/arm/Kconfig
> +++ b/arch/arm/Kconfig
> @@ -1830,6 +1830,8 @@ config RANDOMIZE_BASE
> depends on MMU && AUTO_ZRELADDR
> depends on !XIP_KERNEL && !ZBOOT_ROM
> select RELOCATABLE
> + select ARM_MODULE_PLTS if MODULES
> + select MODULE_REL_CRCS if MODVERSIONS
> help
> Randomizes the virtual and physical address at which the kernel
> image is loaded, as a security feature that deters exploit attempts
> diff --git a/arch/arm/include/asm/elf.h b/arch/arm/include/asm/elf.h
> index f13ae153fb24..b56fc4dd27b6 100644
> --- a/arch/arm/include/asm/elf.h
> +++ b/arch/arm/include/asm/elf.h
> @@ -50,6 +50,7 @@ typedef struct user_fp elf_fpregset_t;
> #define R_ARM_NONE 0
> #define R_ARM_PC24 1
> #define R_ARM_ABS32 2
> +#define R_ARM_REL32 3
> #define R_ARM_CALL 28
> #define R_ARM_JUMP24 29
> #define R_ARM_TARGET1 38
> diff --git a/arch/arm/kernel/module.c b/arch/arm/kernel/module.c
> index 3ff571c2c71c..aa4d72837cd5 100644
> --- a/arch/arm/kernel/module.c
> +++ b/arch/arm/kernel/module.c
> @@ -175,6 +175,10 @@
> *(u32 *)loc |= offset & 0x7fffffff;
> break;
>
> + case R_ARM_REL32:
> + *(u32 *)loc += sym->st_value - loc;
> + break;
> +
> case R_ARM_MOVW_ABS_NC:
> case R_ARM_MOVT_ABS:
> offset = tmp = __mem_to_opcode_arm(*(u32 *)loc);
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 00/29] implement KASLR for ARM
2017-09-05 21:27 ` Tony Lindgren
@ 2017-09-05 21:31 ` Ard Biesheuvel
2017-09-06 10:40 ` Ard Biesheuvel
0 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-05 21:31 UTC (permalink / raw)
To: linux-arm-kernel
On 5 September 2017 at 22:27, Tony Lindgren <tony@atomide.com> wrote:
> * Ard Biesheuvel <ard.biesheuvel@linaro.org> [170905 12:43]:
>> Right. Well, I will try to reproduce with the BB white I have.
>
> Yeah that should be reproducable, I got it to happen on BBB here
> after about 5 boots.
>
>> Are you booting with an initrd?
>
> Not on this one, on beagleboard xm I do.
>
Interesting.
>> >> > Then loading modules with CONFIG_RANDOMIZE_BASE=y seems to fail with:
>> >> >
>> >> > $ sudo modprobe rtc-twl
>> >> > rtc_twl: disagrees about version of symbol module_layout
>> >> > modprobe: ERROR: could not insert 'rtc_twl': Exec format error
>> >> >
>> >>
>> >> Is this with CONFIG_MODVERSIONS enabled?
>> >
>> > Yes, but disabling that did not seem to make any difference
>> > based on just one test.
>> >
>>
>> Yeah, well, it appears I missed a couple of details :-)
>>
>> This should fix the module loading issues:
>
> Yeah now modprobe works :) That's after manually applying
> it as the tabs got munched in your patch somewhere, see below.
>
Yeah, Gmail tends to do that now and again. Apologies.
Thanks for confirming that the module loading works for you, I will
fold this into my v3.
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 00/29] implement KASLR for ARM
2017-09-05 21:31 ` Ard Biesheuvel
@ 2017-09-06 10:40 ` Ard Biesheuvel
2017-09-06 16:22 ` Tony Lindgren
0 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-06 10:40 UTC (permalink / raw)
To: linux-arm-kernel
On 5 September 2017 at 22:31, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> On 5 September 2017 at 22:27, Tony Lindgren <tony@atomide.com> wrote:
>> * Ard Biesheuvel <ard.biesheuvel@linaro.org> [170905 12:43]:
>>> Right. Well, I will try to reproduce with the BB white I have.
>>
>> Yeah that should be reproducable, I got it to happen on BBB here
>> after about 5 boots.
>>
Unfortunately, my BB white boots happily every time.
I have added DEBUG_UNCOMPRESS support to mach-omap2, and added some
debug output to the kaslr init routine
For example,
Starting kernel ...
regions.image_size:00e00000
regions.pa_start:80000000
regions.pa_end:90000000
regions.zimage_start:80200000
regions.zimage_size:00430228
regions.dtb_start:82000000
regions.dtb_size:00009000
num:00000016
*kaslr_offset:04400000
Uncompressing Linux... done, booting the kernel.
[ 0.000000] Booting Linux on physical CPU 0x0
....
[ 0.000000] Virtual kernel memory layout:
[ 0.000000] vector : 0xffff0000 - 0xffff1000 ( 4 kB)
[ 0.000000] fixmap : 0xffc00000 - 0xfff00000 (3072 kB)
[ 0.000000] vmalloc : 0xd0000000 - 0xff800000 ( 760 MB)
[ 0.000000] lowmem : 0xc0000000 - 0xcfe00000 ( 254 MB)
[ 0.000000] pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB)
[ 0.000000] modules : 0xbf000000 - 0xbfe00000 ( 14 MB)
[ 0.000000] .text : 0xc4408000 - 0xc4d00000 (9184 kB)
[ 0.000000] .init : 0xc5000000 - 0xc5100000 (1024 kB)
[ 0.000000] .data : 0xc5100000 - 0xc5149df0 ( 296 kB)
[ 0.000000] .bss : 0xc51cb000 - 0xc59a9a8c (8059 kB)
I have pushed the changes to my arm-kaslr-v3 branch. If you are
willing to spend more cycles on this, could you please run it with
DEBUG_LL enabled, and capture the output of the failed boots?
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 00/29] implement KASLR for ARM
2017-09-06 10:40 ` Ard Biesheuvel
@ 2017-09-06 16:22 ` Tony Lindgren
2017-09-06 16:25 ` Ard Biesheuvel
0 siblings, 1 reply; 83+ messages in thread
From: Tony Lindgren @ 2017-09-06 16:22 UTC (permalink / raw)
To: linux-arm-kernel
* Ard Biesheuvel <ard.biesheuvel@linaro.org> [170906 03:41]:
> On 5 September 2017 at 22:31, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> > On 5 September 2017 at 22:27, Tony Lindgren <tony@atomide.com> wrote:
> >> * Ard Biesheuvel <ard.biesheuvel@linaro.org> [170905 12:43]:
> >>> Right. Well, I will try to reproduce with the BB white I have.
> >>
> >> Yeah that should be reproducable, I got it to happen on BBB here
> >> after about 5 boots.
> >>
>
> Unfortunately, my BB white boots happily every time.
>
> I have added DEBUG_UNCOMPRESS support to mach-omap2, and added some
> debug output to the kaslr init routine
OK. FYI, the DEBUG_UNCOMPRESS support works in Linux next as
it's now using the generic 8250 debug_ll code.
> For example,
>
> Starting kernel ...
>
> regions.image_size:00e00000
> regions.pa_start:80000000
> regions.pa_end:90000000
> regions.zimage_start:80200000
> regions.zimage_size:00430228
> regions.dtb_start:82000000
> regions.dtb_size:00009000
> num:00000016
> *kaslr_offset:04400000
> Uncompressing Linux... done, booting the kernel.
> [ 0.000000] Booting Linux on physical CPU 0x0
> ....
> [ 0.000000] Virtual kernel memory layout:
> [ 0.000000] vector : 0xffff0000 - 0xffff1000 ( 4 kB)
> [ 0.000000] fixmap : 0xffc00000 - 0xfff00000 (3072 kB)
> [ 0.000000] vmalloc : 0xd0000000 - 0xff800000 ( 760 MB)
> [ 0.000000] lowmem : 0xc0000000 - 0xcfe00000 ( 254 MB)
> [ 0.000000] pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB)
> [ 0.000000] modules : 0xbf000000 - 0xbfe00000 ( 14 MB)
> [ 0.000000] .text : 0xc4408000 - 0xc4d00000 (9184 kB)
> [ 0.000000] .init : 0xc5000000 - 0xc5100000 (1024 kB)
> [ 0.000000] .data : 0xc5100000 - 0xc5149df0 ( 296 kB)
> [ 0.000000] .bss : 0xc51cb000 - 0xc59a9a8c (8059 kB)
>
> I have pushed the changes to my arm-kaslr-v3 branch. If you are
> willing to spend more cycles on this, could you please run it with
> DEBUG_LL enabled, and capture the output of the failed boots?
Sure was not able to reproduce it so far on BBB. But here's
failed boot output from logicpd-torpedo-37xx-devkit. Will
try some more booting on BBB too.
Regards,
Tony
8< -------------------
Kernel image @ 0x81000000 [ 0x000000 - 0x426810 ]
## Flattened Device Tree blob at 84000000
Booting using the fdt blob@0x84000000
Loading Device Tree to 86feb000, end 86fff2d5 ... OK
Starting kernel ...
regions.image_size:00e00000
regions.pa_start:80000000
regions.pa_end:88000000
regions.zimage_start:81000000
regions.zimage_size:00437830
regions.dtb_start:86feb000
regions.dtb_size:00012000
regions.initrd_start:00000000
regions.initrd_size:00000000
num:0000002f
num:00000029
*kaslr_offset:07400000
Uncompressing Linux...
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 00/29] implement KASLR for ARM
2017-09-06 16:22 ` Tony Lindgren
@ 2017-09-06 16:25 ` Ard Biesheuvel
2017-09-06 16:31 ` Tony Lindgren
0 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-06 16:25 UTC (permalink / raw)
To: linux-arm-kernel
On 6 September 2017 at 17:22, Tony Lindgren <tony@atomide.com> wrote:
> * Ard Biesheuvel <ard.biesheuvel@linaro.org> [170906 03:41]:
>> On 5 September 2017 at 22:31, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
>> > On 5 September 2017 at 22:27, Tony Lindgren <tony@atomide.com> wrote:
>> >> * Ard Biesheuvel <ard.biesheuvel@linaro.org> [170905 12:43]:
>> >>> Right. Well, I will try to reproduce with the BB white I have.
>> >>
>> >> Yeah that should be reproducable, I got it to happen on BBB here
>> >> after about 5 boots.
>> >>
>>
>> Unfortunately, my BB white boots happily every time.
>>
>> I have added DEBUG_UNCOMPRESS support to mach-omap2, and added some
>> debug output to the kaslr init routine
>
> OK. FYI, the DEBUG_UNCOMPRESS support works in Linux next as
> it's now using the generic 8250 debug_ll code.
>
>> For example,
>>
>> Starting kernel ...
>>
>> regions.image_size:00e00000
>> regions.pa_start:80000000
>> regions.pa_end:90000000
>> regions.zimage_start:80200000
>> regions.zimage_size:00430228
>> regions.dtb_start:82000000
>> regions.dtb_size:00009000
>> num:00000016
>> *kaslr_offset:04400000
>> Uncompressing Linux... done, booting the kernel.
>> [ 0.000000] Booting Linux on physical CPU 0x0
>> ....
>> [ 0.000000] Virtual kernel memory layout:
>> [ 0.000000] vector : 0xffff0000 - 0xffff1000 ( 4 kB)
>> [ 0.000000] fixmap : 0xffc00000 - 0xfff00000 (3072 kB)
>> [ 0.000000] vmalloc : 0xd0000000 - 0xff800000 ( 760 MB)
>> [ 0.000000] lowmem : 0xc0000000 - 0xcfe00000 ( 254 MB)
>> [ 0.000000] pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB)
>> [ 0.000000] modules : 0xbf000000 - 0xbfe00000 ( 14 MB)
>> [ 0.000000] .text : 0xc4408000 - 0xc4d00000 (9184 kB)
>> [ 0.000000] .init : 0xc5000000 - 0xc5100000 (1024 kB)
>> [ 0.000000] .data : 0xc5100000 - 0xc5149df0 ( 296 kB)
>> [ 0.000000] .bss : 0xc51cb000 - 0xc59a9a8c (8059 kB)
>>
>> I have pushed the changes to my arm-kaslr-v3 branch. If you are
>> willing to spend more cycles on this, could you please run it with
>> DEBUG_LL enabled, and capture the output of the failed boots?
>
> Sure was not able to reproduce it so far on BBB. But here's
> failed boot output from logicpd-torpedo-37xx-devkit. Will
> try some more booting on BBB too.
>
Thanks Tony.
> 8< -------------------
> Kernel image @ 0x81000000 [ 0x000000 - 0x426810 ]
> ## Flattened Device Tree blob at 84000000
> Booting using the fdt blob at 0x84000000
> Loading Device Tree to 86feb000, end 86fff2d5 ... OK
>
> Starting kernel ...
>
> regions.image_size:00e00000
> regions.pa_start:80000000
> regions.pa_end:88000000
> regions.zimage_start:81000000
> regions.zimage_size:00437830
> regions.dtb_start:86feb000
> regions.dtb_size:00012000
> regions.initrd_start:00000000
> regions.initrd_size:00000000
> num:0000002f
> num:00000029
> *kaslr_offset:07400000
> Uncompressing Linux...
Is that all? Does it hang while decompressing the kernel?
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 00/29] implement KASLR for ARM
2017-09-06 16:25 ` Ard Biesheuvel
@ 2017-09-06 16:31 ` Tony Lindgren
2017-09-06 16:35 ` Ard Biesheuvel
0 siblings, 1 reply; 83+ messages in thread
From: Tony Lindgren @ 2017-09-06 16:31 UTC (permalink / raw)
To: linux-arm-kernel
* Ard Biesheuvel <ard.biesheuvel@linaro.org> [170906 09:26]:
> On 6 September 2017 at 17:22, Tony Lindgren <tony@atomide.com> wrote:
> > Sure was not able to reproduce it so far on BBB. But here's
> > failed boot output from logicpd-torpedo-37xx-devkit. Will
> > try some more booting on BBB too.
...
> > 8< -------------------
> > Kernel image @ 0x81000000 [ 0x000000 - 0x426810 ]
> > ## Flattened Device Tree blob at 84000000
> > Booting using the fdt blob at 0x84000000
> > Loading Device Tree to 86feb000, end 86fff2d5 ... OK
> >
> > Starting kernel ...
> >
> > regions.image_size:00e00000
> > regions.pa_start:80000000
> > regions.pa_end:88000000
> > regions.zimage_start:81000000
> > regions.zimage_size:00437830
> > regions.dtb_start:86feb000
> > regions.dtb_size:00012000
> > regions.initrd_start:00000000
> > regions.initrd_size:00000000
> > num:0000002f
> > num:00000029
> > *kaslr_offset:07400000
> > Uncompressing Linux...
>
> Is that all? Does it hang while decompressing the kernel?
Yeah so it seems. If we had uncompress overwriting something
because of the increase in size it should happen on every
boot though, not once every five boots or so.
Regards,
TOny
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 00/29] implement KASLR for ARM
2017-09-06 16:31 ` Tony Lindgren
@ 2017-09-06 16:35 ` Ard Biesheuvel
2017-09-06 17:12 ` Tony Lindgren
0 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-06 16:35 UTC (permalink / raw)
To: linux-arm-kernel
On 6 September 2017 at 17:31, Tony Lindgren <tony@atomide.com> wrote:
> * Ard Biesheuvel <ard.biesheuvel@linaro.org> [170906 09:26]:
>> On 6 September 2017 at 17:22, Tony Lindgren <tony@atomide.com> wrote:
>> > Sure was not able to reproduce it so far on BBB. But here's
>> > failed boot output from logicpd-torpedo-37xx-devkit. Will
>> > try some more booting on BBB too.
> ...
>> > 8< -------------------
>> > Kernel image @ 0x81000000 [ 0x000000 - 0x426810 ]
>> > ## Flattened Device Tree blob at 84000000
>> > Booting using the fdt blob at 0x84000000
>> > Loading Device Tree to 86feb000, end 86fff2d5 ... OK
>> >
>> > Starting kernel ...
>> >
>> > regions.image_size:00e00000
>> > regions.pa_start:80000000
>> > regions.pa_end:88000000
>> > regions.zimage_start:81000000
>> > regions.zimage_size:00437830
>> > regions.dtb_start:86feb000
>> > regions.dtb_size:00012000
>> > regions.initrd_start:00000000
>> > regions.initrd_size:00000000
>> > num:0000002f
>> > num:00000029
>> > *kaslr_offset:07400000
>> > Uncompressing Linux...
>>
>> Is that all? Does it hang while decompressing the kernel?
>
> Yeah so it seems. If we had uncompress overwriting something
> because of the increase in size it should happen on every
> boot though, not once every five boots or so.
>
Turns out I am calculating the top of DRAM incorrectly for boards
where less memory is present than the size of the lowmem region.
Could you try this please? (Apologies for the whitespace)
diff --git a/arch/arm/boot/compressed/kaslr.c b/arch/arm/boot/compressed/kaslr.c
index d43c0be9b47b..e9c86809c857 100644
--- a/arch/arm/boot/compressed/kaslr.c
+++ b/arch/arm/boot/compressed/kaslr.c
@@ -339,8 +339,7 @@ u32 kaslr_early_init(u32 *kaslr_offset, u32
image_base, u32 image_size,
regions.image_size = round_up(image_size, SZ_2M);
regions.pa_start = round_down(image_base, SZ_128M);
- regions.pa_end = lowmem_top - PAGE_OFFSET + regions.pa_start -
- regions.image_size;
+ regions.pa_end = lowmem_top - PAGE_OFFSET + regions.pa_start;
regions.zimage_start = zimage_start;
regions.zimage_size = zimage_end - zimage_start;
regions.dtb_start = (u32)fdt;
@@ -391,7 +390,8 @@ u32 kaslr_early_init(u32 *kaslr_offset, u32
image_base, u32 image_size,
}
/* check the memory nodes for the size of the lowmem region */
- regions.pa_end = min(regions.pa_end, get_memory_end(fdt));
+ regions.pa_end = min(regions.pa_end, get_memory_end(fdt)) -
+ regions.image_size;
puthex32(regions.image_size);
puthex32(regions.pa_start);
^ permalink raw reply related [flat|nested] 83+ messages in thread* [PATCH v2 00/29] implement KASLR for ARM
2017-09-06 16:35 ` Ard Biesheuvel
@ 2017-09-06 17:12 ` Tony Lindgren
2017-09-06 17:30 ` Ard Biesheuvel
0 siblings, 1 reply; 83+ messages in thread
From: Tony Lindgren @ 2017-09-06 17:12 UTC (permalink / raw)
To: linux-arm-kernel
* Ard Biesheuvel <ard.biesheuvel@linaro.org> [170906 09:36]:
> Turns out I am calculating the top of DRAM incorrectly for boards
> where less memory is present than the size of the lowmem region.
>
> Could you try this please? (Apologies for the whitespace)
I think for 10 or so units per year you can actually buy
a non-mangling outgoing SMTP service if nothing else helps..
Anyways, you patch manually applied fixed most of the random
boot hangs for me, but I did see a new one after 11 boot
attempts, see below.
Regards,
Tony
8< --------------
Kernel image @ 0x81000000 [ 0x000000 - 0x426300 ]
## Flattened Device Tree blob at 84000000
Booting using the fdt blob@0x84000000
Loading Device Tree to 86feb000, end 86fff2d5 ... OK
Starting kernel ...
regions.image_size:00e00000
regions.pa_start:80000000
regions.pa_end:87200000
regions.zimage_start:81000000
regions.zimage_size:00437320
regions.dtb_start:86feb000
regions.dtb_size:00012000
regions.initrd_start:00000000
regions.initrd_size:00000000
num:00000028
num:00000025
*kaslr_offset:05e00000
Uncompressing Linux... done, booting the kernel.
Warning: Neither atags nor dtb found
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 00/29] implement KASLR for ARM
2017-09-06 17:12 ` Tony Lindgren
@ 2017-09-06 17:30 ` Ard Biesheuvel
2017-09-06 17:53 ` Tony Lindgren
0 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-06 17:30 UTC (permalink / raw)
To: linux-arm-kernel
On 6 September 2017 at 18:12, Tony Lindgren <tony@atomide.com> wrote:
> * Ard Biesheuvel <ard.biesheuvel@linaro.org> [170906 09:36]:
>> Turns out I am calculating the top of DRAM incorrectly for boards
>> where less memory is present than the size of the lowmem region.
>>
>> Could you try this please? (Apologies for the whitespace)
>
> I think for 10 or so units per year you can actually buy
> a non-mangling outgoing SMTP service if nothing else helps..
>
Yeah, you're right, apologies. I use git-send-email mostly, but not
for inline snippets like this. And it is not actually the SMTP service
but the web client that mangles the whitespace
> Anyways, you patch manually applied fixed most of the random
> boot hangs for me, but I did see a new one after 11 boot
> attempts, see below.
>
> Regards,
>
> Tony
>
> 8< --------------
> Kernel image @ 0x81000000 [ 0x000000 - 0x426300 ]
> ## Flattened Device Tree blob at 84000000
> Booting using the fdt blob at 0x84000000
> Loading Device Tree to 86feb000, end 86fff2d5 ... OK
>
> Starting kernel ...
>
> regions.image_size:00e00000
> regions.pa_start:80000000
> regions.pa_end:87200000
> regions.zimage_start:81000000
> regions.zimage_size:00437320
> regions.dtb_start:86feb000
> regions.dtb_size:00012000
> regions.initrd_start:00000000
> regions.initrd_size:00000000
> num:00000028
> num:00000025
> *kaslr_offset:05e00000
> Uncompressing Linux... done, booting the kernel.
> Warning: Neither atags nor dtb found
OK, so in this case, 80000000 + 00e00000 + 05e00000 == 86c00000, which
is still below the DTB, but apparently, it has corrupted it anyway.
I will try to figure out what's going on here.
Thanks again for taking the time,
Ard.
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 00/29] implement KASLR for ARM
2017-09-06 17:30 ` Ard Biesheuvel
@ 2017-09-06 17:53 ` Tony Lindgren
2017-09-06 18:04 ` Ard Biesheuvel
0 siblings, 1 reply; 83+ messages in thread
From: Tony Lindgren @ 2017-09-06 17:53 UTC (permalink / raw)
To: linux-arm-kernel
* Ard Biesheuvel <ard.biesheuvel@linaro.org> [170906 10:31]:
> On 6 September 2017 at 18:12, Tony Lindgren <tony@atomide.com> wrote:
> > Kernel image @ 0x81000000 [ 0x000000 - 0x426300 ]
> > ## Flattened Device Tree blob at 84000000
> > Booting using the fdt blob at 0x84000000
> > Loading Device Tree to 86feb000, end 86fff2d5 ... OK
> >
> > Starting kernel ...
> >
> > regions.image_size:00e00000
> > regions.pa_start:80000000
> > regions.pa_end:87200000
> > regions.zimage_start:81000000
> > regions.zimage_size:00437320
> > regions.dtb_start:86feb000
> > regions.dtb_size:00012000
> > regions.initrd_start:00000000
> > regions.initrd_size:00000000
> > num:00000028
> > num:00000025
> > *kaslr_offset:05e00000
> > Uncompressing Linux... done, booting the kernel.
> > Warning: Neither atags nor dtb found
>
> OK, so in this case, 80000000 + 00e00000 + 05e00000 == 86c00000, which
> is still below the DTB, but apparently, it has corrupted it anyway.
>
> I will try to figure out what's going on here.
Do you need to have kaslr_offset beyond the uncompressed
kernel size maybe?
> Thanks again for taking the time,
No problem, I'm happy to test these changes.
Regards,
Tony
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 00/29] implement KASLR for ARM
2017-09-06 17:53 ` Tony Lindgren
@ 2017-09-06 18:04 ` Ard Biesheuvel
2017-09-06 18:22 ` Tony Lindgren
0 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-06 18:04 UTC (permalink / raw)
To: linux-arm-kernel
On 6 September 2017 at 18:53, Tony Lindgren <tony@atomide.com> wrote:
> * Ard Biesheuvel <ard.biesheuvel@linaro.org> [170906 10:31]:
>> On 6 September 2017 at 18:12, Tony Lindgren <tony@atomide.com> wrote:
>> > Kernel image @ 0x81000000 [ 0x000000 - 0x426300 ]
>> > ## Flattened Device Tree blob at 84000000
>> > Booting using the fdt blob at 0x84000000
>> > Loading Device Tree to 86feb000, end 86fff2d5 ... OK
>> >
>> > Starting kernel ...
>> >
>> > regions.image_size:00e00000
>> > regions.pa_start:80000000
>> > regions.pa_end:87200000
>> > regions.zimage_start:81000000
>> > regions.zimage_size:00437320
>> > regions.dtb_start:86feb000
>> > regions.dtb_size:00012000
>> > regions.initrd_start:00000000
>> > regions.initrd_size:00000000
>> > num:00000028
>> > num:00000025
>> > *kaslr_offset:05e00000
>> > Uncompressing Linux... done, booting the kernel.
>> > Warning: Neither atags nor dtb found
>>
>> OK, so in this case, 80000000 + 00e00000 + 05e00000 == 86c00000, which
>> is still below the DTB, but apparently, it has corrupted it anyway.
>>
>> I will try to figure out what's going on here.
>
> Do you need to have kaslr_offset beyond the uncompressed
> kernel size maybe?
>
I think the problem is in the rounding of region.pa_start.
I have now changed this to
regions.image_size = image_base % SZ_128M + round_up(image_size, SZ_2M);
regions.pa_start = round_down(image_base, SZ_128M);
>> Thanks again for taking the time,
>
> No problem, I'm happy to test these changes.
>
I have updated my arm-kaslr-v3 with all the fixes from this discussion
(and more)
Thanks,
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 00/29] implement KASLR for ARM
2017-09-06 18:04 ` Ard Biesheuvel
@ 2017-09-06 18:22 ` Tony Lindgren
2017-09-06 18:25 ` Ard Biesheuvel
0 siblings, 1 reply; 83+ messages in thread
From: Tony Lindgren @ 2017-09-06 18:22 UTC (permalink / raw)
To: linux-arm-kernel
* Ard Biesheuvel <ard.biesheuvel@linaro.org> [170906 11:05]:
> On 6 September 2017 at 18:53, Tony Lindgren <tony@atomide.com> wrote:
> > * Ard Biesheuvel <ard.biesheuvel@linaro.org> [170906 10:31]:
> >> On 6 September 2017 at 18:12, Tony Lindgren <tony@atomide.com> wrote:
> >> > Kernel image @ 0x81000000 [ 0x000000 - 0x426300 ]
> >> > ## Flattened Device Tree blob at 84000000
> >> > Booting using the fdt blob at 0x84000000
> >> > Loading Device Tree to 86feb000, end 86fff2d5 ... OK
> >> >
> >> > Starting kernel ...
> >> >
> >> > regions.image_size:00e00000
> >> > regions.pa_start:80000000
> >> > regions.pa_end:87200000
> >> > regions.zimage_start:81000000
> >> > regions.zimage_size:00437320
> >> > regions.dtb_start:86feb000
> >> > regions.dtb_size:00012000
> >> > regions.initrd_start:00000000
> >> > regions.initrd_size:00000000
> >> > num:00000028
> >> > num:00000025
> >> > *kaslr_offset:05e00000
> >> > Uncompressing Linux... done, booting the kernel.
> >> > Warning: Neither atags nor dtb found
> >>
> >> OK, so in this case, 80000000 + 00e00000 + 05e00000 == 86c00000, which
> >> is still below the DTB, but apparently, it has corrupted it anyway.
> >>
> >> I will try to figure out what's going on here.
> >
> > Do you need to have kaslr_offset beyond the uncompressed
> > kernel size maybe?
> >
>
> I think the problem is in the rounding of region.pa_start.
>
> I have now changed this to
>
> regions.image_size = image_base % SZ_128M + round_up(image_size, SZ_2M);
> regions.pa_start = round_down(image_base, SZ_128M);
...
> I have updated my arm-kaslr-v3 with all the fixes from this discussion
> (and more)
Looks like your branch at commit 5221c86ad2e7 still failed
after boot attempt #4, see below. Is that the right commit?
Regards,
Tony
8< -----------------
regions.zimage_size:004379e8
regions.dtb_start:86feb000
regions.dtb_size:00012000
regions.initrd_start:00000000
regions.initrd_size:00000000
count:00000028
num:00000025
*kaslr_offset:05e00000
Uncompressing Linux... done, booting the kernel.
Warning: Neither atags nor dtb found
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 00/29] implement KASLR for ARM
2017-09-06 18:22 ` Tony Lindgren
@ 2017-09-06 18:25 ` Ard Biesheuvel
2017-09-06 20:08 ` Tony Lindgren
0 siblings, 1 reply; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-06 18:25 UTC (permalink / raw)
To: linux-arm-kernel
On 6 September 2017 at 19:22, Tony Lindgren <tony@atomide.com> wrote:
> * Ard Biesheuvel <ard.biesheuvel@linaro.org> [170906 11:05]:
>> On 6 September 2017 at 18:53, Tony Lindgren <tony@atomide.com> wrote:
>> > * Ard Biesheuvel <ard.biesheuvel@linaro.org> [170906 10:31]:
>> >> On 6 September 2017 at 18:12, Tony Lindgren <tony@atomide.com> wrote:
>> >> > Kernel image @ 0x81000000 [ 0x000000 - 0x426300 ]
>> >> > ## Flattened Device Tree blob at 84000000
>> >> > Booting using the fdt blob at 0x84000000
>> >> > Loading Device Tree to 86feb000, end 86fff2d5 ... OK
>> >> >
>> >> > Starting kernel ...
>> >> >
>> >> > regions.image_size:00e00000
>> >> > regions.pa_start:80000000
>> >> > regions.pa_end:87200000
>> >> > regions.zimage_start:81000000
>> >> > regions.zimage_size:00437320
>> >> > regions.dtb_start:86feb000
>> >> > regions.dtb_size:00012000
>> >> > regions.initrd_start:00000000
>> >> > regions.initrd_size:00000000
>> >> > num:00000028
>> >> > num:00000025
>> >> > *kaslr_offset:05e00000
>> >> > Uncompressing Linux... done, booting the kernel.
>> >> > Warning: Neither atags nor dtb found
>> >>
>> >> OK, so in this case, 80000000 + 00e00000 + 05e00000 == 86c00000, which
>> >> is still below the DTB, but apparently, it has corrupted it anyway.
>> >>
>> >> I will try to figure out what's going on here.
>> >
>> > Do you need to have kaslr_offset beyond the uncompressed
>> > kernel size maybe?
>> >
>>
>> I think the problem is in the rounding of region.pa_start.
>>
>> I have now changed this to
>>
>> regions.image_size = image_base % SZ_128M + round_up(image_size, SZ_2M);
>> regions.pa_start = round_down(image_base, SZ_128M);
> ...
>
>> I have updated my arm-kaslr-v3 with all the fixes from this discussion
>> (and more)
>
> Looks like your branch at commit 5221c86ad2e7 still failed
> after boot attempt #4, see below. Is that the right commit?
>
It should be, yes.
> 8< -----------------
> regions.zimage_size:004379e8
> regions.dtb_start:86feb000
> regions.dtb_size:00012000
> regions.initrd_start:00000000
> regions.initrd_size:00000000
> count:00000028
> num:00000025
> *kaslr_offset:05e00000
> Uncompressing Linux... done, booting the kernel.
> Warning: Neither atags nor dtb found
Did you capture the image_size and pa_start/pa_end as well?
In any case, this is the exact same offset that failed before, so the
rounding of pa_start wasn't the problem.
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 00/29] implement KASLR for ARM
2017-09-06 18:25 ` Ard Biesheuvel
@ 2017-09-06 20:08 ` Tony Lindgren
2017-09-12 6:51 ` Ard Biesheuvel
0 siblings, 1 reply; 83+ messages in thread
From: Tony Lindgren @ 2017-09-06 20:08 UTC (permalink / raw)
To: linux-arm-kernel
* Ard Biesheuvel <ard.biesheuvel@linaro.org> [170906 11:25]:
> Did you capture the image_size and pa_start/pa_end as well?
Sorry I guess that was an incomplete copy & paste, but here's
another one:
regions.image_size:00e08000
regions.pa_start:80000000
regions.pa_end:871f8000
regions.zimage_start:81000000
regions.zimage_size:004379e8
regions.dtb_start:86feb000
regions.dtb_size:00012000
regions.initrd_start:00000000
regions.initrd_size:00000000
count:00000028
num:00000025
*kaslr_offset:05e00000
Uncompressing Linux... done, booting the kernel.
Warning: Neither atags nor dtb found
> In any case, this is the exact same offset that failed before, so the
> rounding of pa_start wasn't the problem.
OK
Regards,
Tony
^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH v2 00/29] implement KASLR for ARM
2017-09-06 20:08 ` Tony Lindgren
@ 2017-09-12 6:51 ` Ard Biesheuvel
0 siblings, 0 replies; 83+ messages in thread
From: Ard Biesheuvel @ 2017-09-12 6:51 UTC (permalink / raw)
To: linux-arm-kernel
On 6 September 2017 at 21:08, Tony Lindgren <tony@atomide.com> wrote:
> * Ard Biesheuvel <ard.biesheuvel@linaro.org> [170906 11:25]:
>> Did you capture the image_size and pa_start/pa_end as well?
>
> Sorry I guess that was an incomplete copy & paste, but here's
> another one:
>
> regions.image_size:00e08000
> regions.pa_start:80000000
> regions.pa_end:871f8000
> regions.zimage_start:81000000
> regions.zimage_size:004379e8
> regions.dtb_start:86feb000
> regions.dtb_size:00012000
> regions.initrd_start:00000000
> regions.initrd_size:00000000
> count:00000028
> num:00000025
> *kaslr_offset:05e00000
> Uncompressing Linux... done, booting the kernel.
> Warning: Neither atags nor dtb found
>
>> In any case, this is the exact same offset that failed before, so the
>> rounding of pa_start wasn't the problem.
>
> OK
>
I am pretty sure this is an issue with .bss, which is not covered by
image_size (and which is absolutely *huge* btw [8+ MB] on
omap2plus_defconfig, due to lockdep being enabled by default)
^ permalink raw reply [flat|nested] 83+ messages in thread