[PATCH v2] KVM: arm64: Add missing BTI instructions

[PATCH v2] KVM: arm64: Add missing BTI instructions

[Mostafa Saleh]

Some bti instructions were missing from
commit b53d4a272349 ("KVM: arm64: Use BTI for nvhe")

1) kvm_host_psci_cpu_entry
kvm_host_psci_cpu_entry is called from __kvm_hyp_init_cpu through "br"
instruction as __kvm_hyp_init_cpu resides in idmap section while
kvm_host_psci_cpu_entry is in hyp .text so the offset is larger than
128MB range covered by "b".
Which means that this function should start with "bti j" instruction.

LLVM which is the only compiler supporting BTI for Linux, adds "bti j"
for jump tables or by when taking the address of the block [1].
Same behaviour is observed with GCC.

As kvm_host_psci_cpu_entry is a C function, this must be done in
assembly.

Another solution is to use X16/X17 with "br", as according to ARM
ARM DDI0487I.a RLJHCL/IGMGRS, PACIASP has an implicit branch
target identification instruction that is compatible with
PSTATE.BTYPE 0b01 which includes "br X16/X17"
And the kvm_host_psci_cpu_entry has PACIASP as it is an external
function.
Although, using explicit "bti" makes it more clear than relying on
which register is used.

A third solution is to clear SCTLR_EL2.BT, which would make PACIASP
compatible PSTATE.BTYPE 0b11 ("br" to other registers).
However this deviates from the kernel behaviour (in bti_enable()).

2) Spectre vector table
"br" instructions are generated at runtime for the vector table
(__bp_harden_hyp_vecs).
These branches would land on vectors in __kvm_hyp_vector at offset 8.
As all the macros are defined with valid_vect/invalid_vect, it is
sufficient to add "bti j" at the correct offset.

[1] https://reviews.llvm.org/D52867

Fixes: b53d4a272349 ("KVM: arm64: Use BTI for nvhe")
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Reported-by: Sudeep Holla <sudeep.holla@arm.com>
---
v1 -> v2:
- Add another missing bti in spectre vector table.
v1: https://lore.kernel.org/all/20230705171632.3912123-1-smostafa@google.com/
---
 arch/arm64/kvm/hyp/hyp-entry.S       |  8 ++++++++
 arch/arm64/kvm/hyp/nvhe/host.S       | 10 ++++++++++
 arch/arm64/kvm/hyp/nvhe/psci-relay.c |  2 +-
 3 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
index 8f3f93fa119e..03f97d71984c 100644
--- a/arch/arm64/kvm/hyp/hyp-entry.S
+++ b/arch/arm64/kvm/hyp/hyp-entry.S
@@ -154,6 +154,12 @@ SYM_CODE_END(\label)
 	esb
 	stp	x0, x1, [sp, #-16]!
 662:
+	/*
+	 * spectre vectors __bp_harden_hyp_vecs generate br instructions at runtime
+	 * that jump at offset 8 at __kvm_hyp_vector.
+	 * As hyp .text is guarded section, it needs bti j.
+	 */
+	bti j
 	b	\target
 
 check_preamble_length 661b, 662b
@@ -165,6 +171,8 @@ check_preamble_length 661b, 662b
 	nop
 	stp	x0, x1, [sp, #-16]!
 662:
+	/* Check valid_vect */
+	bti j
 	b	\target
 
 check_preamble_length 661b, 662b
diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S
index c87c63133e10..7693a6757cd7 100644
--- a/arch/arm64/kvm/hyp/nvhe/host.S
+++ b/arch/arm64/kvm/hyp/nvhe/host.S
@@ -297,3 +297,13 @@ SYM_CODE_START(__kvm_hyp_host_forward_smc)
 
 	ret
 SYM_CODE_END(__kvm_hyp_host_forward_smc)
+
+/*
+ * kvm_host_psci_cpu_entry is called through br instruction, which requires
+ * bti j instruction as compilers (gcc and llvm) doesn't insert bti j for external
+ * functions, but bti c instead.
+ */
+SYM_CODE_START(kvm_host_psci_cpu_entry)
+       bti j
+       b __kvm_host_psci_cpu_entry
+SYM_CODE_END(kvm_host_psci_cpu_entry)
diff --git a/arch/arm64/kvm/hyp/nvhe/psci-relay.c b/arch/arm64/kvm/hyp/nvhe/psci-relay.c
index 08508783ec3d..24543d2a3490 100644
--- a/arch/arm64/kvm/hyp/nvhe/psci-relay.c
+++ b/arch/arm64/kvm/hyp/nvhe/psci-relay.c
@@ -200,7 +200,7 @@ static int psci_system_suspend(u64 func_id, struct kvm_cpu_context *host_ctxt)
 			 __hyp_pa(init_params), 0);
 }
 
-asmlinkage void __noreturn kvm_host_psci_cpu_entry(bool is_cpu_on)
+asmlinkage void __noreturn __kvm_host_psci_cpu_entry(bool is_cpu_on)
 {
 	struct psci_boot_args *boot_args;
 	struct kvm_cpu_context *host_ctxt;
-- 
2.41.0.255.g8b1d071c50-goog


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2] KVM: arm64: Add missing BTI instructions

[Sudeep Holla]

On Thu, Jul 06, 2023 at 03:22:40PM +0000, Mostafa Saleh wrote:
> Some bti instructions were missing from
> commit b53d4a272349 ("KVM: arm64: Use BTI for nvhe")
> 
> 1) kvm_host_psci_cpu_entry
> kvm_host_psci_cpu_entry is called from __kvm_hyp_init_cpu through "br"
> instruction as __kvm_hyp_init_cpu resides in idmap section while
> kvm_host_psci_cpu_entry is in hyp .text so the offset is larger than
> 128MB range covered by "b".
> Which means that this function should start with "bti j" instruction.
> 
> LLVM which is the only compiler supporting BTI for Linux, adds "bti j"
> for jump tables or by when taking the address of the block [1].
> Same behaviour is observed with GCC.
> 
> As kvm_host_psci_cpu_entry is a C function, this must be done in
> assembly.
> 
> Another solution is to use X16/X17 with "br", as according to ARM
> ARM DDI0487I.a RLJHCL/IGMGRS, PACIASP has an implicit branch
> target identification instruction that is compatible with
> PSTATE.BTYPE 0b01 which includes "br X16/X17"
> And the kvm_host_psci_cpu_entry has PACIASP as it is an external
> function.
> Although, using explicit "bti" makes it more clear than relying on
> which register is used.
> 
> A third solution is to clear SCTLR_EL2.BT, which would make PACIASP
> compatible PSTATE.BTYPE 0b11 ("br" to other registers).
> However this deviates from the kernel behaviour (in bti_enable()).
> 
> 2) Spectre vector table
> "br" instructions are generated at runtime for the vector table
> (__bp_harden_hyp_vecs).
> These branches would land on vectors in __kvm_hyp_vector at offset 8.
> As all the macros are defined with valid_vect/invalid_vect, it is
> sufficient to add "bti j" at the correct offset.
> 
> [1] https://reviews.llvm.org/D52867
> 
> Fixes: b53d4a272349 ("KVM: arm64: Use BTI for nvhe")
> Signed-off-by: Mostafa Saleh <smostafa@google.com>
> Reported-by: Sudeep Holla <sudeep.holla@arm.com>

Nothing change w.r.t cpu suspend-resume path in v2 anyways, but I assure
I tested this again just be absolutely sure and it still fixes the issue
I reported 😄, so

Tested-by: Sudeep Holla <sudeep.holla@arm.com>

-- 
Regards,
Sudeep

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2] KVM: arm64: Add missing BTI instructions

[Mostafa Saleh]

On Thu, Jul 06, 2023 at 05:23:08PM +0100, Sudeep Holla wrote:
> On Thu, Jul 06, 2023 at 03:22:40PM +0000, Mostafa Saleh wrote:
> > Some bti instructions were missing from
> > commit b53d4a272349 ("KVM: arm64: Use BTI for nvhe")
> > 
> > 1) kvm_host_psci_cpu_entry
> > kvm_host_psci_cpu_entry is called from __kvm_hyp_init_cpu through "br"
> > instruction as __kvm_hyp_init_cpu resides in idmap section while
> > kvm_host_psci_cpu_entry is in hyp .text so the offset is larger than
> > 128MB range covered by "b".
> > Which means that this function should start with "bti j" instruction.
> > 
> > LLVM which is the only compiler supporting BTI for Linux, adds "bti j"
> > for jump tables or by when taking the address of the block [1].
> > Same behaviour is observed with GCC.
> > 
> > As kvm_host_psci_cpu_entry is a C function, this must be done in
> > assembly.
> > 
> > Another solution is to use X16/X17 with "br", as according to ARM
> > ARM DDI0487I.a RLJHCL/IGMGRS, PACIASP has an implicit branch
> > target identification instruction that is compatible with
> > PSTATE.BTYPE 0b01 which includes "br X16/X17"
> > And the kvm_host_psci_cpu_entry has PACIASP as it is an external
> > function.
> > Although, using explicit "bti" makes it more clear than relying on
> > which register is used.
> > 
> > A third solution is to clear SCTLR_EL2.BT, which would make PACIASP
> > compatible PSTATE.BTYPE 0b11 ("br" to other registers).
> > However this deviates from the kernel behaviour (in bti_enable()).
> > 
> > 2) Spectre vector table
> > "br" instructions are generated at runtime for the vector table
> > (__bp_harden_hyp_vecs).
> > These branches would land on vectors in __kvm_hyp_vector at offset 8.
> > As all the macros are defined with valid_vect/invalid_vect, it is
> > sufficient to add "bti j" at the correct offset.
> > 
> > [1] https://reviews.llvm.org/D52867
> > 
> > Fixes: b53d4a272349 ("KVM: arm64: Use BTI for nvhe")
> > Signed-off-by: Mostafa Saleh <smostafa@google.com>
> > Reported-by: Sudeep Holla <sudeep.holla@arm.com>
> 
> Nothing change w.r.t cpu suspend-resume path in v2 anyways, but I assure
> I tested this again just be absolutely sure and it still fixes the issue
> I reported 😄, so
> 
> Tested-by: Sudeep Holla <sudeep.holla@arm.com>

Thanks for testing the patch again!


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2] KVM: arm64: Add missing BTI instructions

[Marc Zyngier]

On Thu, 06 Jul 2023 16:22:40 +0100,
Mostafa Saleh <smostafa@google.com> wrote:
> 
> Some bti instructions were missing from
> commit b53d4a272349 ("KVM: arm64: Use BTI for nvhe")
> 
> 1) kvm_host_psci_cpu_entry
> kvm_host_psci_cpu_entry is called from __kvm_hyp_init_cpu through "br"
> instruction as __kvm_hyp_init_cpu resides in idmap section while
> kvm_host_psci_cpu_entry is in hyp .text so the offset is larger than
> 128MB range covered by "b".
> Which means that this function should start with "bti j" instruction.
> 
> LLVM which is the only compiler supporting BTI for Linux, adds "bti j"
> for jump tables or by when taking the address of the block [1].
> Same behaviour is observed with GCC.
> 
> As kvm_host_psci_cpu_entry is a C function, this must be done in
> assembly.
> 
> Another solution is to use X16/X17 with "br", as according to ARM
> ARM DDI0487I.a RLJHCL/IGMGRS, PACIASP has an implicit branch
> target identification instruction that is compatible with
> PSTATE.BTYPE 0b01 which includes "br X16/X17"
> And the kvm_host_psci_cpu_entry has PACIASP as it is an external
> function.
> Although, using explicit "bti" makes it more clear than relying on
> which register is used.
> 
> A third solution is to clear SCTLR_EL2.BT, which would make PACIASP
> compatible PSTATE.BTYPE 0b11 ("br" to other registers).
> However this deviates from the kernel behaviour (in bti_enable()).
> 
> 2) Spectre vector table
> "br" instructions are generated at runtime for the vector table
> (__bp_harden_hyp_vecs).
> These branches would land on vectors in __kvm_hyp_vector at offset 8.
> As all the macros are defined with valid_vect/invalid_vect, it is
> sufficient to add "bti j" at the correct offset.
> 
> [1] https://reviews.llvm.org/D52867
> 
> Fixes: b53d4a272349 ("KVM: arm64: Use BTI for nvhe")
> Signed-off-by: Mostafa Saleh <smostafa@google.com>
> Reported-by: Sudeep Holla <sudeep.holla@arm.com>
> ---
> v1 -> v2:
> - Add another missing bti in spectre vector table.
> v1: https://lore.kernel.org/all/20230705171632.3912123-1-smostafa@google.com/

Acked-by: Marc Zyngier <maz@kernel.org>

	M.

-- 
Without deviation from the norm, progress is not possible.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2] KVM: arm64: Add missing BTI instructions

[Oliver Upton]

On Thu, 6 Jul 2023 15:22:40 +0000, Mostafa Saleh wrote:
> Some bti instructions were missing from
> commit b53d4a272349 ("KVM: arm64: Use BTI for nvhe")
> 
> 1) kvm_host_psci_cpu_entry
> kvm_host_psci_cpu_entry is called from __kvm_hyp_init_cpu through "br"
> instruction as __kvm_hyp_init_cpu resides in idmap section while
> kvm_host_psci_cpu_entry is in hyp .text so the offset is larger than
> 128MB range covered by "b".
> Which means that this function should start with "bti j" instruction.
> 
> [...]

Applied to kvmarm/fixes. Sorry this took longer than expected, Mostafa. I
was hitting an unrelated issue with pKVM initialization on the only system
I have that supports BTI.

[1/1] KVM: arm64: Add missing BTI instructions
      https://git.kernel.org/kvmarm/kvmarm/c/dcf89d111199

--
Best,
Oliver

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2] KVM: arm64: Add missing BTI instructions

[Mostafa Saleh]

On Wed, Jul 12, 2023 at 10:50:34PM +0000, Oliver Upton wrote:
> On Thu, 6 Jul 2023 15:22:40 +0000, Mostafa Saleh wrote:
> > Some bti instructions were missing from
> > commit b53d4a272349 ("KVM: arm64: Use BTI for nvhe")
> > 
> > 1) kvm_host_psci_cpu_entry
> > kvm_host_psci_cpu_entry is called from __kvm_hyp_init_cpu through "br"
> > instruction as __kvm_hyp_init_cpu resides in idmap section while
> > kvm_host_psci_cpu_entry is in hyp .text so the offset is larger than
> > 128MB range covered by "b".
> > Which means that this function should start with "bti j" instruction.
> > 
> > [...]
> 
> Applied to kvmarm/fixes. Sorry this took longer than expected, Mostafa. I
> was hitting an unrelated issue with pKVM initialization on the only system
> I have that supports BTI.
> 
> [1/1] KVM: arm64: Add missing BTI instructions
>       https://git.kernel.org/kvmarm/kvmarm/c/dcf89d111199
> 

Thanks Oliver, hopefully everything is sorted out now.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel