[PATCH v2 0/2] arm64: harden shadow call stack pointer handling

[PATCH v2 0/2] arm64: harden shadow call stack pointer handling

[Ard Biesheuvel]

A couple of tweaks to the arm64 entry code to avoid loading the shadow
call stack pointer in a way that could potentially be unsafe in the
context of ROP attacks.

Changes since v1:
- rebase onto v6.2-rc1

Cc: Sami Tolvanen <samitolvanen@google.com>
Cc: Kees Cook <keescook@chromium.org>

Ard Biesheuvel (2):
  arm64: Always load shadow stack pointer directly from the task struct
  arm64: Stash shadow stack pointer in the task struct on interrupt

 arch/arm64/include/asm/scs.h |  7 ++++---
 arch/arm64/kernel/entry.S    | 16 +++++++---------
 arch/arm64/kernel/head.S     |  2 +-
 3 files changed, 12 insertions(+), 13 deletions(-)

-- 
2.39.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v2 1/2] arm64: Always load shadow stack pointer directly from the task struct

[Ard Biesheuvel]

All occurrences of the scs_load macro load the value of the shadow call
stack pointer from the task which is current at that point. So instead
of taking a task struct register argument in the scs_load macro to
specify the task struct to load from, let's always reference the current
task directly. This should make it much harder to exploit any
instruction sequences reloading the shadow call stack pointer register
from memory.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
 arch/arm64/include/asm/scs.h | 7 ++++---
 arch/arm64/kernel/entry.S    | 4 ++--
 arch/arm64/kernel/head.S     | 2 +-
 3 files changed, 7 insertions(+), 6 deletions(-)

diff --git a/arch/arm64/include/asm/scs.h b/arch/arm64/include/asm/scs.h
index ff7da1268a52ab79..13df982a080805e6 100644
--- a/arch/arm64/include/asm/scs.h
+++ b/arch/arm64/include/asm/scs.h
@@ -10,15 +10,16 @@
 #ifdef CONFIG_SHADOW_CALL_STACK
 	scs_sp	.req	x18
 
-	.macro scs_load tsk
-	ldr	scs_sp, [\tsk, #TSK_TI_SCS_SP]
+	.macro scs_load_current
+	get_current_task scs_sp
+	ldr	scs_sp, [scs_sp, #TSK_TI_SCS_SP]
 	.endm
 
 	.macro scs_save tsk
 	str	scs_sp, [\tsk, #TSK_TI_SCS_SP]
 	.endm
 #else
-	.macro scs_load tsk
+	.macro scs_load_current
 	.endm
 
 	.macro scs_save tsk
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index 11cb99c4d298784d..546f7773238ea45d 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -275,7 +275,7 @@ alternative_if ARM64_HAS_ADDRESS_AUTH
 alternative_else_nop_endif
 1:
 
-	scs_load tsk
+	scs_load_current
 	.else
 	add	x21, sp, #PT_REGS_SIZE
 	get_current_task tsk
@@ -848,7 +848,7 @@ SYM_FUNC_START(cpu_switch_to)
 	msr	sp_el0, x1
 	ptrauth_keys_install_kernel x1, x8, x9, x10
 	scs_save x0
-	scs_load x1
+	scs_load_current
 	ret
 SYM_FUNC_END(cpu_switch_to)
 NOKPROBE(cpu_switch_to)
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 952e17bd1c0b4f91..b9c1a506798ea315 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -404,7 +404,7 @@ SYM_FUNC_END(create_kernel_mapping)
 	stp	xzr, xzr, [sp, #S_STACKFRAME]
 	add	x29, sp, #S_STACKFRAME
 
-	scs_load \tsk
+	scs_load_current
 
 	adr_l	\tmp1, __per_cpu_offset
 	ldr	w\tmp2, [\tsk, #TSK_TI_CPU]
-- 
2.39.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 1/2] arm64: Always load shadow stack pointer directly from the task struct

[Mark Rutland]

On Mon, Jan 09, 2023 at 06:47:59PM +0100, Ard Biesheuvel wrote:
> All occurrences of the scs_load macro load the value of the shadow call
> stack pointer from the task which is current at that point. So instead
> of taking a task struct register argument in the scs_load macro to
> specify the task struct to load from, let's always reference the current
> task directly. This should make it much harder to exploit any
> instruction sequences reloading the shadow call stack pointer register
> from memory.
> 
> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>

Makes sense to me.

Acked-by: Mark Rutland <mark.rutland@arm.com>

Mark.

> ---
>  arch/arm64/include/asm/scs.h | 7 ++++---
>  arch/arm64/kernel/entry.S    | 4 ++--
>  arch/arm64/kernel/head.S     | 2 +-
>  3 files changed, 7 insertions(+), 6 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/scs.h b/arch/arm64/include/asm/scs.h
> index ff7da1268a52ab79..13df982a080805e6 100644
> --- a/arch/arm64/include/asm/scs.h
> +++ b/arch/arm64/include/asm/scs.h
> @@ -10,15 +10,16 @@
>  #ifdef CONFIG_SHADOW_CALL_STACK
>  	scs_sp	.req	x18
>  
> -	.macro scs_load tsk
> -	ldr	scs_sp, [\tsk, #TSK_TI_SCS_SP]
> +	.macro scs_load_current
> +	get_current_task scs_sp
> +	ldr	scs_sp, [scs_sp, #TSK_TI_SCS_SP]
>  	.endm
>  
>  	.macro scs_save tsk
>  	str	scs_sp, [\tsk, #TSK_TI_SCS_SP]
>  	.endm
>  #else
> -	.macro scs_load tsk
> +	.macro scs_load_current
>  	.endm
>  
>  	.macro scs_save tsk
> diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
> index 11cb99c4d298784d..546f7773238ea45d 100644
> --- a/arch/arm64/kernel/entry.S
> +++ b/arch/arm64/kernel/entry.S
> @@ -275,7 +275,7 @@ alternative_if ARM64_HAS_ADDRESS_AUTH
>  alternative_else_nop_endif
>  1:
>  
> -	scs_load tsk
> +	scs_load_current
>  	.else
>  	add	x21, sp, #PT_REGS_SIZE
>  	get_current_task tsk
> @@ -848,7 +848,7 @@ SYM_FUNC_START(cpu_switch_to)
>  	msr	sp_el0, x1
>  	ptrauth_keys_install_kernel x1, x8, x9, x10
>  	scs_save x0
> -	scs_load x1
> +	scs_load_current
>  	ret
>  SYM_FUNC_END(cpu_switch_to)
>  NOKPROBE(cpu_switch_to)
> diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
> index 952e17bd1c0b4f91..b9c1a506798ea315 100644
> --- a/arch/arm64/kernel/head.S
> +++ b/arch/arm64/kernel/head.S
> @@ -404,7 +404,7 @@ SYM_FUNC_END(create_kernel_mapping)
>  	stp	xzr, xzr, [sp, #S_STACKFRAME]
>  	add	x29, sp, #S_STACKFRAME
>  
> -	scs_load \tsk
> +	scs_load_current
>  
>  	adr_l	\tmp1, __per_cpu_offset
>  	ldr	w\tmp2, [\tsk, #TSK_TI_CPU]
> -- 
> 2.39.0
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 1/2] arm64: Always load shadow stack pointer directly from the task struct

[Kees Cook]

On Mon, Jan 09, 2023 at 06:47:59PM +0100, Ard Biesheuvel wrote:
> All occurrences of the scs_load macro load the value of the shadow call
> stack pointer from the task which is current at that point. So instead
> of taking a task struct register argument in the scs_load macro to
> specify the task struct to load from, let's always reference the current
> task directly. This should make it much harder to exploit any
> instruction sequences reloading the shadow call stack pointer register
> from memory.
> 
> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>

Reviewed-by: Kees Cook <keescook@chromium.org>

-- 
Kees Cook

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v2 2/2] arm64: Stash shadow stack pointer in the task struct on interrupt

[Ard Biesheuvel]

Instead of reloading the shadow call stack pointer from the ordinary
stack, which may be vulnerable to the kind of gadget based attacks
shadow call stacks were designed to prevent, let's store a task's shadow
call stack pointer in the task struct when switching to the shadow IRQ
stack.

Given that currently, the task_struct::scs_sp field is only used to
preserve the shadow call stack pointer while a task is scheduled out or
running in user space, reusing this field to preserve and restore it
while running off the IRQ stack must be safe, as those occurrences are
guaranteed to never overlap. (The stack switching logic only switches
stacks when running from the task stack, and so the value being saved
here always corresponds to the task mode shadow stack)

While at it, fold a mov/add/mov sequence into a single add.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
 arch/arm64/kernel/entry.S | 12 +++++-------
 1 file changed, 5 insertions(+), 7 deletions(-)

diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index 546f7773238ea45d..80d763e165fc5856 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -876,19 +876,19 @@ NOKPROBE(ret_from_fork)
  */
 SYM_FUNC_START(call_on_irq_stack)
 #ifdef CONFIG_SHADOW_CALL_STACK
-	stp	scs_sp, xzr, [sp, #-16]!
+	get_current_task x16
+	scs_save x16
 	ldr_this_cpu scs_sp, irq_shadow_call_stack_ptr, x17
 #endif
+
 	/* Create a frame record to save our LR and SP (implicit in FP) */
 	stp	x29, x30, [sp, #-16]!
 	mov	x29, sp
 
 	ldr_this_cpu x16, irq_stack_ptr, x17
-	mov	x15, #IRQ_STACK_SIZE
-	add	x16, x16, x15
 
 	/* Move to the new stack and call the function there */
-	mov	sp, x16
+	add	sp, x16, #IRQ_STACK_SIZE
 	blr	x1
 
 	/*
@@ -897,9 +897,7 @@ SYM_FUNC_START(call_on_irq_stack)
 	 */
 	mov	sp, x29
 	ldp	x29, x30, [sp], #16
-#ifdef CONFIG_SHADOW_CALL_STACK
-	ldp	scs_sp, xzr, [sp], #16
-#endif
+	scs_load_current
 	ret
 SYM_FUNC_END(call_on_irq_stack)
 NOKPROBE(call_on_irq_stack)
-- 
2.39.0


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 2/2] arm64: Stash shadow stack pointer in the task struct on interrupt

[Mark Rutland]

On Mon, Jan 09, 2023 at 06:48:00PM +0100, Ard Biesheuvel wrote:
> Instead of reloading the shadow call stack pointer from the ordinary
> stack, which may be vulnerable to the kind of gadget based attacks
> shadow call stacks were designed to prevent, let's store a task's shadow
> call stack pointer in the task struct when switching to the shadow IRQ
> stack.
> 
> Given that currently, the task_struct::scs_sp field is only used to
> preserve the shadow call stack pointer while a task is scheduled out or
> running in user space, reusing this field to preserve and restore it
> while running off the IRQ stack must be safe, as those occurrences are
> guaranteed to never overlap. (The stack switching logic only switches
> stacks when running from the task stack, and so the value being saved
> here always corresponds to the task mode shadow stack)
> 
> While at it, fold a mov/add/mov sequence into a single add.
> 
> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>

Acked-by: Mark Rutland <mark.rutland@arm.com>

Mark.

> ---
>  arch/arm64/kernel/entry.S | 12 +++++-------
>  1 file changed, 5 insertions(+), 7 deletions(-)
> 
> diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
> index 546f7773238ea45d..80d763e165fc5856 100644
> --- a/arch/arm64/kernel/entry.S
> +++ b/arch/arm64/kernel/entry.S
> @@ -876,19 +876,19 @@ NOKPROBE(ret_from_fork)
>   */
>  SYM_FUNC_START(call_on_irq_stack)
>  #ifdef CONFIG_SHADOW_CALL_STACK
> -	stp	scs_sp, xzr, [sp, #-16]!
> +	get_current_task x16
> +	scs_save x16
>  	ldr_this_cpu scs_sp, irq_shadow_call_stack_ptr, x17
>  #endif
> +
>  	/* Create a frame record to save our LR and SP (implicit in FP) */
>  	stp	x29, x30, [sp, #-16]!
>  	mov	x29, sp
>  
>  	ldr_this_cpu x16, irq_stack_ptr, x17
> -	mov	x15, #IRQ_STACK_SIZE
> -	add	x16, x16, x15
>  
>  	/* Move to the new stack and call the function there */
> -	mov	sp, x16
> +	add	sp, x16, #IRQ_STACK_SIZE
>  	blr	x1
>  
>  	/*
> @@ -897,9 +897,7 @@ SYM_FUNC_START(call_on_irq_stack)
>  	 */
>  	mov	sp, x29
>  	ldp	x29, x30, [sp], #16
> -#ifdef CONFIG_SHADOW_CALL_STACK
> -	ldp	scs_sp, xzr, [sp], #16
> -#endif
> +	scs_load_current
>  	ret
>  SYM_FUNC_END(call_on_irq_stack)
>  NOKPROBE(call_on_irq_stack)
> -- 
> 2.39.0
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 2/2] arm64: Stash shadow stack pointer in the task struct on interrupt

[Kees Cook]

On Mon, Jan 09, 2023 at 06:48:00PM +0100, Ard Biesheuvel wrote:
> Instead of reloading the shadow call stack pointer from the ordinary
> stack, which may be vulnerable to the kind of gadget based attacks
> shadow call stacks were designed to prevent, let's store a task's shadow
> call stack pointer in the task struct when switching to the shadow IRQ
> stack.
> 
> Given that currently, the task_struct::scs_sp field is only used to
> preserve the shadow call stack pointer while a task is scheduled out or
> running in user space, reusing this field to preserve and restore it
> while running off the IRQ stack must be safe, as those occurrences are
> guaranteed to never overlap. (The stack switching logic only switches
> stacks when running from the task stack, and so the value being saved
> here always corresponds to the task mode shadow stack)
> 
> While at it, fold a mov/add/mov sequence into a single add.
> 
> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>

Reviewed-by: Kees Cook <keescook@chromium.org>

-- 
Kees Cook

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 0/2] arm64: harden shadow call stack pointer handling

[Catalin Marinas]

On Mon, 09 Jan 2023 18:47:58 +0100, Ard Biesheuvel wrote:
> A couple of tweaks to the arm64 entry code to avoid loading the shadow
> call stack pointer in a way that could potentially be unsafe in the
> context of ROP attacks.
> 
> Changes since v1:
> - rebase onto v6.2-rc1
> 
> [...]

Applied to arm64 (for-next/scs), thanks!

[1/2] arm64: Always load shadow stack pointer directly from the task struct
      https://git.kernel.org/arm64/c/2198d07c509f
[2/2] arm64: Stash shadow stack pointer in the task struct on interrupt
      https://git.kernel.org/arm64/c/59b37fe52f49

-- 
Catalin


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel