[PATCH 0/5] Backport ARM64 VHE boot fixes to 6.6.y

[PATCH 0/5] Backport ARM64 VHE boot fixes to 6.6.y

[Colton Lewis]

This series backports VHE CPU boot fixes to the 6.6.y stable branch.

These fixes are already present in the 6.12.y stable branch (and
newer), but are missing in 6.6.y. They are required to enable booting
L1 guests with nested virtualization enabled (kvm-arm.mode=nested).

Without these patches, a 6.6.y guest boots with HCR_EL2.E2H
incorrectly configured (because it misses VHE-only detection or early
initialization), causing early boot hangs/trap loops.

Conflict resolutions:
- Patch 4 (KVM: arm64: Initialize HCR_EL2.E2H early) had conflicts in
  arch/arm64/kvm/hyp/nvhe/hyp-init.S due to differences in state
  initialization. Resolved by extracting EL2 state initialization into
  __kvm_init_el2_state.
- Patch 5 (arm64: Revamp HCR_EL2.E2H RES1 detection) had conflicts in
  arch/arm64/include/asm/el2_setup.h. Resolved by using raw msr hcr_el2
  instead of the missing msr_hcr_el2 macro.


Marc Zyngier (4):
  arm64: sysreg: Add layout for ID_AA64MMFR4_EL1
  arm64: Treat HCR_EL2.E2H as RES1 when ID_AA64MMFR4_EL1.E2H0 is
    negative
  arm64: Fix early handling of FEAT_E2H0 not being implemented
  arm64: Revamp HCR_EL2.E2H RES1 detection

Mark Rutland (1):
  KVM: arm64: Initialize HCR_EL2.E2H early

 arch/arm64/include/asm/el2_setup.h | 52 ++++++++++++++++++++++++++++++
 arch/arm64/kernel/head.S           | 17 +++-------
 arch/arm64/kvm/hyp/nvhe/hyp-init.S | 16 +++++++--
 arch/arm64/tools/sysreg            | 37 +++++++++++++++++++++
 4 files changed, 107 insertions(+), 15 deletions(-)


base-commit: d1cfde2d5d15be14123bdd1689162bd27f995a90
--
2.55.0.rc2.803.g1fd1e6609c-goog

[PATCH 1/5] arm64: sysreg: Add layout for ID_AA64MMFR4_EL1

[Colton Lewis]

From: Marc Zyngier <maz@kernel.org>

[ Upstream commit cfc680bb04c54e61faa51a34d8383a0aa25b583f ]

ARMv9.5 has infroduced ID_AA64MMFR4_EL1 with a bunch of new features.
Add the corresponding layout.

This is extracted from the public ARM SysReg_xml_A_profile-2023-09
delivery, timestamped d55f5af8e09052abe92a02adf820deea2eaed717.

Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Miguel Luis <miguel.luis@oracle.com>
Link: https://lore.kernel.org/r/20240122181344.258974-5-maz@kernel.org
Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
---
 arch/arm64/tools/sysreg | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)

diff --git a/arch/arm64/tools/sysreg b/arch/arm64/tools/sysreg
index 76ce150e7347e..f7180d391f829 100644
--- a/arch/arm64/tools/sysreg
+++ b/arch/arm64/tools/sysreg
@@ -1669,6 +1669,43 @@ UnsignedEnum	3:0	TCRX
 EndEnum
 EndSysreg
 
+Sysreg	ID_AA64MMFR4_EL1	3	0	0	7	4
+Res0	63:40
+UnsignedEnum	39:36	E3DSE
+	0b0000	NI
+	0b0001	IMP
+EndEnum
+Res0	35:28
+SignedEnum	27:24	E2H0
+	0b0000	IMP
+	0b1110	NI_NV1
+	0b1111	NI
+EndEnum
+UnsignedEnum	23:20	NV_frac
+	0b0000	NV_NV2
+	0b0001	NV2_ONLY
+EndEnum
+UnsignedEnum	19:16	FGWTE3
+	0b0000	NI
+	0b0001	IMP
+EndEnum
+UnsignedEnum	15:12	HACDBS
+	0b0000	NI
+	0b0001	IMP
+EndEnum
+UnsignedEnum	11:8	ASID2
+	0b0000	NI
+	0b0001	IMP
+EndEnum
+SignedEnum	7:4	EIESB
+	0b0000	NI
+	0b0001	ToEL3
+	0b0010	ToELx
+	0b1111	ANY
+EndEnum
+Res0	3:0
+EndSysreg
+
 Sysreg	SCTLR_EL1	3	0	1	0	0
 Field	63	TIDCP
 Field	62	SPINTMASK
-- 
2.55.0.rc2.803.g1fd1e6609c-goog

[PATCH 2/5] arm64: Treat HCR_EL2.E2H as RES1 when ID_AA64MMFR4_EL1.E2H0 is negative

[Colton Lewis]

From: Marc Zyngier <maz@kernel.org>

[ Upstream commit 3944382fa6f22b54bc3624c9657b98ec34b5ba59 ]

For CPUs that have ID_AA64MMFR4_EL1.E2H0 as negative, it is important
to avoid the boot path that sets HCR_EL2.E2H=0. Fortunately, we
already have this path to cope with fruity CPUs.

Tweak init_el2 to look at ID_AA64MMFR4_EL1.E2H0 first.

Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Link: https://lore.kernel.org/r/20240122181344.258974-8-maz@kernel.org
Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
---
 arch/arm64/kernel/head.S | 23 +++++++++++++++--------
 1 file changed, 15 insertions(+), 8 deletions(-)

diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 6517bf2644a08..e32c8dd0b17a7 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -589,25 +589,32 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL)
 	mov_q	x1, INIT_SCTLR_EL1_MMU_OFF
 
 	/*
-	 * Fruity CPUs seem to have HCR_EL2.E2H set to RES1,
-	 * making it impossible to start in nVHE mode. Is that
-	 * compliant with the architecture? Absolutely not!
+	 * Compliant CPUs advertise their VHE-onlyness with
+	 * ID_AA64MMFR4_EL1.E2H0 < 0. HCR_EL2.E2H can be
+	 * RES1 in that case.
+	 *
+	 * Fruity CPUs seem to have HCR_EL2.E2H set to RES1, but
+	 * don't advertise it (they predate this relaxation).
 	 */
+	mrs_s	x0, SYS_ID_AA64MMFR4_EL1
+	ubfx	x0, x0, #ID_AA64MMFR4_EL1_E2H0_SHIFT, #ID_AA64MMFR4_EL1_E2H0_WIDTH
+	tbnz	x0, #(ID_AA64MMFR4_EL1_E2H0_SHIFT + ID_AA64MMFR4_EL1_E2H0_WIDTH - 1), 1f
+
 	mrs	x0, hcr_el2
 	and	x0, x0, #HCR_E2H
-	cbz	x0, 1f
-
+	cbz	x0, 2f
+1:
 	/* Set a sane SCTLR_EL1, the VHE way */
 	pre_disable_mmu_workaround
 	msr_s	SYS_SCTLR_EL12, x1
 	mov	x2, #BOOT_CPU_FLAG_E2H
-	b	2f
+	b	3f
 
-1:
+2:
 	pre_disable_mmu_workaround
 	msr	sctlr_el1, x1
 	mov	x2, xzr
-2:
+3:
 	__init_el2_nvhe_prepare_eret
 
 	mov	w0, #BOOT_CPU_MODE_EL2
-- 
2.55.0.rc2.803.g1fd1e6609c-goog

[PATCH 3/5] arm64: Fix early handling of FEAT_E2H0 not being implemented

[Colton Lewis]

From: Marc Zyngier <maz@kernel.org>

[ Upstream commit b3320142f3db9b3f36a59bd9769ba249f06155b4 ]

Commit 3944382fa6f2 introduced checks for the FEAT_E2H0 not being
implemented. However, the check is absolutely wrong and makes a
point it testing a bit that is guaranteed to be zero.

On top of that, the detection happens way too late, after the
init_el2_state has done its job.

This went undetected because the HW this was tested on has E2H being
RAO/WI, and not RES1. However, the bug shows up when run as a nested
guest, where HCR_EL2.E2H is not necessarily set to 1. As a result,
booting the kernel in hVHE mode fails with timer accesses being
cought in a trap loop (which was fun to debug).

Fix the check for ID_AA64MMFR4_EL1.E2H0, and set the HCR_EL2.E2H bit
early so that it can be checked by the rest of the init sequence.

With this, hVHE works again in a NV environment that doesn't have
FEAT_E2H0.

Fixes: 3944382fa6f2 ("arm64: Treat HCR_EL2.E2H as RES1 when ID_AA64MMFR4_EL1.E2H0 is negative")
Signed-off-by: Marc Zyngier <maz@kernel.org>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Link: https://lore.kernel.org/r/20240321115414.3169115-1-maz@kernel.org
Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
---
 arch/arm64/kernel/head.S | 29 ++++++++++++++++-------------
 1 file changed, 16 insertions(+), 13 deletions(-)

diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index e32c8dd0b17a7..e0e710b36da37 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -576,6 +576,21 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL)
 	isb
 0:
 	mov_q	x0, HCR_HOST_NVHE_FLAGS
+
+	/*
+	 * Compliant CPUs advertise their VHE-onlyness with
+	 * ID_AA64MMFR4_EL1.E2H0 < 0. HCR_EL2.E2H can be
+	 * RES1 in that case. Publish the E2H bit early so that
+	 * it can be picked up by the init_el2_state macro.
+	 *
+	 * Fruity CPUs seem to have HCR_EL2.E2H set to RAO/WI, but
+	 * don't advertise it (they predate this relaxation).
+	 */
+	mrs_s	x1, SYS_ID_AA64MMFR4_EL1
+	tbz	x1, #(ID_AA64MMFR4_EL1_E2H0_SHIFT + ID_AA64MMFR4_EL1_E2H0_WIDTH - 1), 1f
+
+	orr	x0, x0, #HCR_E2H
+1:
 	msr	hcr_el2, x0
 	isb
 
@@ -588,22 +603,10 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL)
 
 	mov_q	x1, INIT_SCTLR_EL1_MMU_OFF
 
-	/*
-	 * Compliant CPUs advertise their VHE-onlyness with
-	 * ID_AA64MMFR4_EL1.E2H0 < 0. HCR_EL2.E2H can be
-	 * RES1 in that case.
-	 *
-	 * Fruity CPUs seem to have HCR_EL2.E2H set to RES1, but
-	 * don't advertise it (they predate this relaxation).
-	 */
-	mrs_s	x0, SYS_ID_AA64MMFR4_EL1
-	ubfx	x0, x0, #ID_AA64MMFR4_EL1_E2H0_SHIFT, #ID_AA64MMFR4_EL1_E2H0_WIDTH
-	tbnz	x0, #(ID_AA64MMFR4_EL1_E2H0_SHIFT + ID_AA64MMFR4_EL1_E2H0_WIDTH - 1), 1f
-
 	mrs	x0, hcr_el2
 	and	x0, x0, #HCR_E2H
 	cbz	x0, 2f
-1:
+
 	/* Set a sane SCTLR_EL1, the VHE way */
 	pre_disable_mmu_workaround
 	msr_s	SYS_SCTLR_EL12, x1
-- 
2.55.0.rc2.803.g1fd1e6609c-goog

[PATCH 4/5] KVM: arm64: Initialize HCR_EL2.E2H early

[Colton Lewis]

From: Mark Rutland <mark.rutland@arm.com>

[ Upstream commit 7a68b55ff39b0d2dcd92ee241b12b23a7e03c621 ]

On CPUs without FEAT_E2H0, HCR_EL2.E2H is RES1, but may reset to an
UNKNOWN value out of reset and consequently may not read as 1 unless it
has been explicitly initialized.

We handled this for the head.S boot code in commits:

  3944382fa6f22b54 ("arm64: Treat HCR_EL2.E2H as RES1 when ID_AA64MMFR4_EL1.E2H0 is negative")
  b3320142f3db9b3f ("arm64: Fix early handling of FEAT_E2H0 not being implemented")

Unfortunately, we forgot to apply a similar fix to the KVM PSCI entry
points used when relaying CPU_ON, CPU_SUSPEND, and SYSTEM SUSPEND. When
KVM is entered via these entry points, the value of HCR_EL2.E2H may be
consumed before it has been initialized (e.g. by the 'init_el2_state'
macro).

Initialize HCR_EL2.E2H early in these paths such that it can be consumed
reliably. The existing code in head.S is factored out into a new
'init_el2_hcr' macro, and this is used in the __kvm_hyp_init_cpu()
function common to all the relevant PSCI entry points.

For clarity, I've tweaked the assembly used to check whether
ID_AA64MMFR4_EL1.E2H0 is negative. The bitfield is extracted as a signed
value, and this is checked with a signed-greater-or-equal (GE) comparison.

As the hyp code will reconfigure HCR_EL2 later in ___kvm_hyp_init(), all
bits other than E2H are initialized to zero in __kvm_hyp_init_cpu().

Fixes: 3944382fa6f22b54 ("arm64: Treat HCR_EL2.E2H as RES1 when ID_AA64MMFR4_EL1.E2H0 is negative")
Fixes: b3320142f3db9b3f ("arm64: Fix early handling of FEAT_E2H0 not being implemented")
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Ahmed Genidi <ahmed.genidi@arm.com>
Cc: Ben Horgan <ben.horgan@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Leo Yan <leo.yan@arm.com>
Cc: Marc Zyngier <maz@kernel.org>
Cc: Oliver Upton <oliver.upton@linux.dev>
Cc: Will Deacon <will@kernel.org>
Link: https://lore.kernel.org/r/20250227180526.1204723-2-mark.rutland@arm.com
[maz: fixed LT->GE thinko]
Signed-off-by: Marc Zyngier <maz@kernel.org>

[ Backport: Resolved conflict in arch/arm64/kvm/hyp/nvhe/hyp-init.S
  by extracting EL2 state initialization into __kvm_init_el2_state
  and calling it after HCR setup. ]
---
 arch/arm64/include/asm/el2_setup.h | 26 ++++++++++++++++++++++++++
 arch/arm64/kernel/head.S           | 19 +------------------
 arch/arm64/kvm/hyp/nvhe/hyp-init.S | 16 +++++++++++++---
 3 files changed, 40 insertions(+), 21 deletions(-)

diff --git a/arch/arm64/include/asm/el2_setup.h b/arch/arm64/include/asm/el2_setup.h
index b7afaa026842b..3498dc5d02c18 100644
--- a/arch/arm64/include/asm/el2_setup.h
+++ b/arch/arm64/include/asm/el2_setup.h
@@ -16,6 +16,32 @@
 #include <asm/sysreg.h>
 #include <linux/irqchip/arm-gic-v3.h>
 
+.macro init_el2_hcr	val
+	mov_q	x0, \val
+
+	/*
+	 * Compliant CPUs advertise their VHE-onlyness with
+	 * ID_AA64MMFR4_EL1.E2H0 < 0. On such CPUs HCR_EL2.E2H is RES1, but it
+	 * can reset into an UNKNOWN state and might not read as 1 until it has
+	 * been initialized explicitly.
+	 *
+	 * Fruity CPUs seem to have HCR_EL2.E2H set to RAO/WI, but
+	 * don't advertise it (they predate this relaxation).
+	 *
+	 * Initalize HCR_EL2.E2H so that later code can rely upon HCR_EL2.E2H
+	 * indicating whether the CPU is running in E2H mode.
+	 */
+	mrs_s	x1, SYS_ID_AA64MMFR4_EL1
+	sbfx	x1, x1, #ID_AA64MMFR4_EL1_E2H0_SHIFT, #ID_AA64MMFR4_EL1_E2H0_WIDTH
+	cmp	x1, #0
+	b.ge	.LnVHE_\@
+
+	orr	x0, x0, #HCR_E2H
+.LnVHE_\@:
+	msr	hcr_el2, x0
+	isb
+.endm
+
 .macro __init_el2_sctlr
 	mov_q	x0, INIT_SCTLR_EL2_MMU_OFF
 	msr	sctlr_el2, x0
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index e0e710b36da37..ff7769821166a 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -575,25 +575,8 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL)
 	msr	sctlr_el2, x0
 	isb
 0:
-	mov_q	x0, HCR_HOST_NVHE_FLAGS
-
-	/*
-	 * Compliant CPUs advertise their VHE-onlyness with
-	 * ID_AA64MMFR4_EL1.E2H0 < 0. HCR_EL2.E2H can be
-	 * RES1 in that case. Publish the E2H bit early so that
-	 * it can be picked up by the init_el2_state macro.
-	 *
-	 * Fruity CPUs seem to have HCR_EL2.E2H set to RAO/WI, but
-	 * don't advertise it (they predate this relaxation).
-	 */
-	mrs_s	x1, SYS_ID_AA64MMFR4_EL1
-	tbz	x1, #(ID_AA64MMFR4_EL1_E2H0_SHIFT + ID_AA64MMFR4_EL1_E2H0_WIDTH - 1), 1f
-
-	orr	x0, x0, #HCR_E2H
-1:
-	msr	hcr_el2, x0
-	isb
 
+	init_el2_hcr	HCR_HOST_NVHE_FLAGS
 	init_el2_state
 
 	/* Hypervisor stub */
diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S
index 1cc06e6797bda..a08363b9b10fd 100644
--- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S
+++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S
@@ -75,6 +75,16 @@ __do_hyp_init:
 	eret
 SYM_CODE_END(__kvm_hyp_init)
 
+/*
+ * Initialize EL2 CPU state to sane values.
+ *
+ * HCR_EL2.E2H must have been initialized already.
+ */
+SYM_CODE_START_LOCAL(__kvm_init_el2_state)
+	init_el2_state				// Clobbers x0..x2
+	finalise_el2_state
+	ret
+SYM_CODE_END(__kvm_init_el2_state)
 /*
  * Initialize the hypervisor in EL2.
  *
@@ -202,9 +212,9 @@ SYM_CODE_START_LOCAL(__kvm_hyp_init_cpu)
 
 2:	msr	SPsel, #1			// We want to use SP_EL{1,2}
 
-	/* Initialize EL2 CPU state to sane values. */
-	init_el2_state				// Clobbers x0..x2
-	finalise_el2_state
+	init_el2_hcr	0
+
+	bl	__kvm_init_el2_state
 	__init_el2_nvhe_prepare_eret
 
 	/* Enable MMU, set vectors and stack. */
-- 
2.55.0.rc2.803.g1fd1e6609c-goog

[PATCH 5/5] arm64: Revamp HCR_EL2.E2H RES1 detection

[Colton Lewis]

From: Marc Zyngier <maz@kernel.org>

[ Upstream commit ca88ecdce5f5127ef2ee241b12b23a7e03c6210f ]

We currently have two ways to identify CPUs that only implement FEAT_VHE
and not FEAT_E2H0:

- either they advertise it via ID_AA64MMFR4_EL1.E2H0,
- or the HCR_EL2.E2H bit is RAO/WI

However, there is a third category of "cpus" that fall between these
two cases: on CPUs that do not implement FEAT_FGT, it is IMPDEF whether
an access to ID_AA64MMFR4_EL1 can trap to EL2 when the register value
is zero.

A consequence of this is that on systems such as Neoverse V2, a NV
guest cannot reliably detect that it is in a VHE-only configuration
(E2H is writable, and ID_AA64MMFR0_EL1 is 0), despite the hypervisor's
best effort to repaint the id register.

Replace the RAO/WI test by a sequence that makes use of the VHE
register remnapping between EL1 and EL2 to detect this situation,
and work out whether we get the VHE behaviour even after having
set HCR_EL2.E2H to 0.

This solves the NV problem, and provides a more reliable acid test
for CPUs that do not completely follow the letter of the architecture
while providing a RES1 behaviour for HCR_EL2.E2H.

Suggested-by: Mark Rutland <mark.rutland@arm.com>
Acked-by: Mark Rutland <mark.rutland@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Oliver Upton <oliver.upton@linux.dev>
Tested-by: Jan Kotas <jank@cadence.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/15A85F2B-1A0C-4FA7-9FE4-EEC2203CC09E@global.cadence.com

[ Backport: Resolved conflict in arch/arm64/include/asm/el2_setup.h
  by replacing msr_hcr_el2 macro usages with raw msr hcr_el2 (since
  the macro is missing in 6.6.y). ]
---
 arch/arm64/include/asm/el2_setup.h | 38 +++++++++++++++++++++++++-----
 1 file changed, 32 insertions(+), 6 deletions(-)

diff --git a/arch/arm64/include/asm/el2_setup.h b/arch/arm64/include/asm/el2_setup.h
index 3498dc5d02c18..38d32116a23eb 100644
--- a/arch/arm64/include/asm/el2_setup.h
+++ b/arch/arm64/include/asm/el2_setup.h
@@ -24,22 +24,48 @@
 	 * ID_AA64MMFR4_EL1.E2H0 < 0. On such CPUs HCR_EL2.E2H is RES1, but it
 	 * can reset into an UNKNOWN state and might not read as 1 until it has
 	 * been initialized explicitly.
-	 *
-	 * Fruity CPUs seem to have HCR_EL2.E2H set to RAO/WI, but
-	 * don't advertise it (they predate this relaxation).
-	 *
 	 * Initalize HCR_EL2.E2H so that later code can rely upon HCR_EL2.E2H
 	 * indicating whether the CPU is running in E2H mode.
 	 */
 	mrs_s	x1, SYS_ID_AA64MMFR4_EL1
 	sbfx	x1, x1, #ID_AA64MMFR4_EL1_E2H0_SHIFT, #ID_AA64MMFR4_EL1_E2H0_WIDTH
 	cmp	x1, #0
-	b.ge	.LnVHE_\@
+	b.lt	.LnE2H0_\@
 
+	/*
+	 * Unfortunately, HCR_EL2.E2H can be RES1 even if not advertised
+	 * as such via ID_AA64MMFR4_EL1.E2H0:
+	 *
+	 * - Fruity CPUs predate the !FEAT_E2H0 relaxation, and seem to
+	 *   have HCR_EL2.E2H implemented as RAO/WI.
+	 *
+	 * - On CPUs that lack FEAT_FGT, a hypervisor can't trap guest
+	 *   reads of ID_AA64MMFR4_EL1 to advertise !FEAT_E2H0. NV
+	 *   guests on these hosts can write to HCR_EL2.E2H without
+	 *   trapping to the hypervisor, but these writes have no
+	 *   functional effect.
+	 *
+	 * Handle both cases by checking for an essential VHE property
+	 * (system register remapping) to decide whether we're
+	 * effectively VHE-only or not.
+	 */
+	msr	hcr_el2, x0		// Setup HCR_EL2 as nVHE
+	isb
+	mov	x1, #1		// Write something to FAR_EL1
+	msr	far_el1, x1
+	isb
+	mov	x1, #2		// Try to overwrite it via FAR_EL2
+	msr	far_el2, x1
+	isb
+	mrs	x1, far_el1	// If we see the latest write in FAR_EL1,
+	cmp	x1, #2		// we can safely assume we are VHE only.
+	b.ne	.LnVHE_\@	// Otherwise, we know that nVHE works.
+
+.LnE2H0_\@:
 	orr	x0, x0, #HCR_E2H
-.LnVHE_\@:
 	msr	hcr_el2, x0
 	isb
+.LnVHE_\@:
 .endm
 
 .macro __init_el2_sctlr
-- 
2.55.0.rc2.803.g1fd1e6609c-goog

Re: [PATCH 0/5] Backport ARM64 VHE boot fixes to 6.6.y

[Oliver Upton]

The subject prefix should be "[PATCH 6.6 0/5]" so people know right up
front where this is going.

On Wed, Jul 01, 2026 at 08:43:37PM +0000, Colton Lewis wrote:
> This series backports VHE CPU boot fixes to the 6.6.y stable branch.
> 
> These fixes are already present in the 6.12.y stable branch (and
> newer), but are missing in 6.6.y. They are required to enable booting
> L1 guests with nested virtualization enabled (kvm-arm.mode=nested).

It's a bit worse than this. The architecture retroactively made
FEAT_E2H0 an optional feature, there are now implementations in the wild
that do not support the feature.

> Without these patches, a 6.6.y guest boots with HCR_EL2.E2H
> incorrectly configured (because it misses VHE-only detection or early
> initialization), causing early boot hangs/trap loops.
> 
> Conflict resolutions:
> - Patch 4 (KVM: arm64: Initialize HCR_EL2.E2H early) had conflicts in
>   arch/arm64/kvm/hyp/nvhe/hyp-init.S due to differences in state
>   initialization. Resolved by extracting EL2 state initialization into
>   __kvm_init_el2_state.
> - Patch 5 (arm64: Revamp HCR_EL2.E2H RES1 detection) had conflicts in
>   arch/arm64/include/asm/el2_setup.h. Resolved by using raw msr hcr_el2
>   instead of the missing msr_hcr_el2 macro.
> 
> 
> Marc Zyngier (4):
>   arm64: sysreg: Add layout for ID_AA64MMFR4_EL1
>   arm64: Treat HCR_EL2.E2H as RES1 when ID_AA64MMFR4_EL1.E2H0 is
>     negative
>   arm64: Fix early handling of FEAT_E2H0 not being implemented
>   arm64: Revamp HCR_EL2.E2H RES1 detection
> 
> Mark Rutland (1):
>   KVM: arm64: Initialize HCR_EL2.E2H early
> 
>  arch/arm64/include/asm/el2_setup.h | 52 ++++++++++++++++++++++++++++++
>  arch/arm64/kernel/head.S           | 17 +++-------
>  arch/arm64/kvm/hyp/nvhe/hyp-init.S | 16 +++++++--
>  arch/arm64/tools/sysreg            | 37 +++++++++++++++++++++
>  4 files changed, 107 insertions(+), 15 deletions(-)
> 
> 
> base-commit: d1cfde2d5d15be14123bdd1689162bd27f995a90
> --
> 2.55.0.rc2.803.g1fd1e6609c-goog
> 

Re: [PATCH 2/5] arm64: Treat HCR_EL2.E2H as RES1 when ID_AA64MMFR4_EL1.E2H0 is negative

[Oliver Upton]

On Wed, Jul 01, 2026 at 08:43:39PM +0000, Colton Lewis wrote:
> From: Marc Zyngier <maz@kernel.org>
> 
> [ Upstream commit 3944382fa6f22b54bc3624c9657b98ec34b5ba59 ]

What is this? I have the commit in question as

3944382fa6f22b54fd399632b1af92c28123979b

> For CPUs that have ID_AA64MMFR4_EL1.E2H0 as negative, it is important
> to avoid the boot path that sets HCR_EL2.E2H=0. Fortunately, we
> already have this path to cope with fruity CPUs.
> 
> Tweak init_el2 to look at ID_AA64MMFR4_EL1.E2H0 first.
> 
> Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> Signed-off-by: Marc Zyngier <maz@kernel.org>
> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
> Link: https://lore.kernel.org/r/20240122181344.258974-8-maz@kernel.org
> Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
> ---
>  arch/arm64/kernel/head.S | 23 +++++++++++++++--------
>  1 file changed, 15 insertions(+), 8 deletions(-)
> 
> diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
> index 6517bf2644a08..e32c8dd0b17a7 100644
> --- a/arch/arm64/kernel/head.S
> +++ b/arch/arm64/kernel/head.S
> @@ -589,25 +589,32 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL)
>  	mov_q	x1, INIT_SCTLR_EL1_MMU_OFF
>  
>  	/*
> -	 * Fruity CPUs seem to have HCR_EL2.E2H set to RES1,
> -	 * making it impossible to start in nVHE mode. Is that
> -	 * compliant with the architecture? Absolutely not!
> +	 * Compliant CPUs advertise their VHE-onlyness with
> +	 * ID_AA64MMFR4_EL1.E2H0 < 0. HCR_EL2.E2H can be
> +	 * RES1 in that case.
> +	 *
> +	 * Fruity CPUs seem to have HCR_EL2.E2H set to RES1, but
> +	 * don't advertise it (they predate this relaxation).
>  	 */
> +	mrs_s	x0, SYS_ID_AA64MMFR4_EL1
> +	ubfx	x0, x0, #ID_AA64MMFR4_EL1_E2H0_SHIFT, #ID_AA64MMFR4_EL1_E2H0_WIDTH
> +	tbnz	x0, #(ID_AA64MMFR4_EL1_E2H0_SHIFT + ID_AA64MMFR4_EL1_E2H0_WIDTH - 1), 1f
> +
>  	mrs	x0, hcr_el2
>  	and	x0, x0, #HCR_E2H
> -	cbz	x0, 1f
> -
> +	cbz	x0, 2f
> +1:
>  	/* Set a sane SCTLR_EL1, the VHE way */
>  	pre_disable_mmu_workaround
>  	msr_s	SYS_SCTLR_EL12, x1
>  	mov	x2, #BOOT_CPU_FLAG_E2H
> -	b	2f
> +	b	3f
>  
> -1:
> +2:
>  	pre_disable_mmu_workaround
>  	msr	sctlr_el1, x1
>  	mov	x2, xzr
> -2:
> +3:
>  	__init_el2_nvhe_prepare_eret
>  
>  	mov	w0, #BOOT_CPU_MODE_EL2
> -- 
> 2.55.0.rc2.803.g1fd1e6609c-goog
> 
> 

Re: [PATCH 4/5] KVM: arm64: Initialize HCR_EL2.E2H early

[Oliver Upton]

On Wed, Jul 01, 2026 at 08:43:41PM +0000, Colton Lewis wrote:
> From: Mark Rutland <mark.rutland@arm.com>
> 
> [ Upstream commit 7a68b55ff39b0d2dcd92ee241b12b23a7e03c621 ]
> 
> On CPUs without FEAT_E2H0, HCR_EL2.E2H is RES1, but may reset to an
> UNKNOWN value out of reset and consequently may not read as 1 unless it
> has been explicitly initialized.
> 
> We handled this for the head.S boot code in commits:
> 
>   3944382fa6f22b54 ("arm64: Treat HCR_EL2.E2H as RES1 when ID_AA64MMFR4_EL1.E2H0 is negative")
>   b3320142f3db9b3f ("arm64: Fix early handling of FEAT_E2H0 not being implemented")
> 
> Unfortunately, we forgot to apply a similar fix to the KVM PSCI entry
> points used when relaying CPU_ON, CPU_SUSPEND, and SYSTEM SUSPEND. When
> KVM is entered via these entry points, the value of HCR_EL2.E2H may be
> consumed before it has been initialized (e.g. by the 'init_el2_state'
> macro).
> 
> Initialize HCR_EL2.E2H early in these paths such that it can be consumed
> reliably. The existing code in head.S is factored out into a new
> 'init_el2_hcr' macro, and this is used in the __kvm_hyp_init_cpu()
> function common to all the relevant PSCI entry points.
> 
> For clarity, I've tweaked the assembly used to check whether
> ID_AA64MMFR4_EL1.E2H0 is negative. The bitfield is extracted as a signed
> value, and this is checked with a signed-greater-or-equal (GE) comparison.
> 
> As the hyp code will reconfigure HCR_EL2 later in ___kvm_hyp_init(), all
> bits other than E2H are initialized to zero in __kvm_hyp_init_cpu().
> 
> Fixes: 3944382fa6f22b54 ("arm64: Treat HCR_EL2.E2H as RES1 when ID_AA64MMFR4_EL1.E2H0 is negative")
> Fixes: b3320142f3db9b3f ("arm64: Fix early handling of FEAT_E2H0 not being implemented")
> Signed-off-by: Mark Rutland <mark.rutland@arm.com>
> Cc: Ahmed Genidi <ahmed.genidi@arm.com>
> Cc: Ben Horgan <ben.horgan@arm.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Leo Yan <leo.yan@arm.com>
> Cc: Marc Zyngier <maz@kernel.org>
> Cc: Oliver Upton <oliver.upton@linux.dev>
> Cc: Will Deacon <will@kernel.org>
> Link: https://lore.kernel.org/r/20250227180526.1204723-2-mark.rutland@arm.com
> [maz: fixed LT->GE thinko]
> Signed-off-by: Marc Zyngier <maz@kernel.org>
> 
> [ Backport: Resolved conflict in arch/arm64/kvm/hyp/nvhe/hyp-init.S
>   by extracting EL2 state initialization into __kvm_init_el2_state
>   and calling it after HCR setup. ]
> ---
>  arch/arm64/include/asm/el2_setup.h | 26 ++++++++++++++++++++++++++
>  arch/arm64/kernel/head.S           | 19 +------------------
>  arch/arm64/kvm/hyp/nvhe/hyp-init.S | 16 +++++++++++++---
>  3 files changed, 40 insertions(+), 21 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/el2_setup.h b/arch/arm64/include/asm/el2_setup.h
> index b7afaa026842b..3498dc5d02c18 100644
> --- a/arch/arm64/include/asm/el2_setup.h
> +++ b/arch/arm64/include/asm/el2_setup.h
> @@ -16,6 +16,32 @@
>  #include <asm/sysreg.h>
>  #include <linux/irqchip/arm-gic-v3.h>
>  
> +.macro init_el2_hcr	val
> +	mov_q	x0, \val
> +
> +	/*
> +	 * Compliant CPUs advertise their VHE-onlyness with
> +	 * ID_AA64MMFR4_EL1.E2H0 < 0. On such CPUs HCR_EL2.E2H is RES1, but it
> +	 * can reset into an UNKNOWN state and might not read as 1 until it has
> +	 * been initialized explicitly.
> +	 *
> +	 * Fruity CPUs seem to have HCR_EL2.E2H set to RAO/WI, but
> +	 * don't advertise it (they predate this relaxation).
> +	 *
> +	 * Initalize HCR_EL2.E2H so that later code can rely upon HCR_EL2.E2H
> +	 * indicating whether the CPU is running in E2H mode.
> +	 */
> +	mrs_s	x1, SYS_ID_AA64MMFR4_EL1
> +	sbfx	x1, x1, #ID_AA64MMFR4_EL1_E2H0_SHIFT, #ID_AA64MMFR4_EL1_E2H0_WIDTH
> +	cmp	x1, #0
> +	b.ge	.LnVHE_\@
> +
> +	orr	x0, x0, #HCR_E2H
> +.LnVHE_\@:
> +	msr	hcr_el2, x0
> +	isb
> +.endm
> +
>  .macro __init_el2_sctlr
>  	mov_q	x0, INIT_SCTLR_EL2_MMU_OFF
>  	msr	sctlr_el2, x0
> diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
> index e0e710b36da37..ff7769821166a 100644
> --- a/arch/arm64/kernel/head.S
> +++ b/arch/arm64/kernel/head.S
> @@ -575,25 +575,8 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL)
>  	msr	sctlr_el2, x0
>  	isb
>  0:
> -	mov_q	x0, HCR_HOST_NVHE_FLAGS
> -
> -	/*
> -	 * Compliant CPUs advertise their VHE-onlyness with
> -	 * ID_AA64MMFR4_EL1.E2H0 < 0. HCR_EL2.E2H can be
> -	 * RES1 in that case. Publish the E2H bit early so that
> -	 * it can be picked up by the init_el2_state macro.
> -	 *
> -	 * Fruity CPUs seem to have HCR_EL2.E2H set to RAO/WI, but
> -	 * don't advertise it (they predate this relaxation).
> -	 */
> -	mrs_s	x1, SYS_ID_AA64MMFR4_EL1
> -	tbz	x1, #(ID_AA64MMFR4_EL1_E2H0_SHIFT + ID_AA64MMFR4_EL1_E2H0_WIDTH - 1), 1f
> -
> -	orr	x0, x0, #HCR_E2H
> -1:
> -	msr	hcr_el2, x0
> -	isb
>  
> +	init_el2_hcr	HCR_HOST_NVHE_FLAGS
>  	init_el2_state
>  
>  	/* Hypervisor stub */
> diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S
> index 1cc06e6797bda..a08363b9b10fd 100644
> --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S
> +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S
> @@ -75,6 +75,16 @@ __do_hyp_init:
>  	eret
>  SYM_CODE_END(__kvm_hyp_init)
>  
> +/*
> + * Initialize EL2 CPU state to sane values.
> + *
> + * HCR_EL2.E2H must have been initialized already.
> + */
> +SYM_CODE_START_LOCAL(__kvm_init_el2_state)
> +	init_el2_state				// Clobbers x0..x2
> +	finalise_el2_state
> +	ret
> +SYM_CODE_END(__kvm_init_el2_state)
>  /*
>   * Initialize the hypervisor in EL2.
>   *
> @@ -202,9 +212,9 @@ SYM_CODE_START_LOCAL(__kvm_hyp_init_cpu)
>  
>  2:	msr	SPsel, #1			// We want to use SP_EL{1,2}
>  
> -	/* Initialize EL2 CPU state to sane values. */
> -	init_el2_state				// Clobbers x0..x2
> -	finalise_el2_state
> +	init_el2_hcr	0
> +
> +	bl	__kvm_init_el2_state

Please don't churn unrelated code. Leave everything where it is and make
sure init_el2_hcr is done before the others.

Thanks,
Oliver

Re: [PATCH 0/5] Backport ARM64 VHE boot fixes to 6.6.y

[Oliver Upton]

Circling back around...

On Wed, Jul 01, 2026 at 04:23:21PM -0700, Oliver Upton wrote:
> The subject prefix should be "[PATCH 6.6 0/5]" so people know right up
> front where this is going.
> 
> On Wed, Jul 01, 2026 at 08:43:37PM +0000, Colton Lewis wrote:
> > This series backports VHE CPU boot fixes to the 6.6.y stable branch.
> > 
> > These fixes are already present in the 6.12.y stable branch (and
> > newer), but are missing in 6.6.y. They are required to enable booting
> > L1 guests with nested virtualization enabled (kvm-arm.mode=nested).
> 
> It's a bit worse than this. The architecture retroactively made
> FEAT_E2H0 an optional feature, there are now implementations in the wild
> that do not support the feature.
> 
> > Without these patches, a 6.6.y guest boots with HCR_EL2.E2H
> > incorrectly configured (because it misses VHE-only detection or early
> > initialization), causing early boot hangs/trap loops.
> > 
> > Conflict resolutions:
> > - Patch 4 (KVM: arm64: Initialize HCR_EL2.E2H early) had conflicts in
> >   arch/arm64/kvm/hyp/nvhe/hyp-init.S due to differences in state
> >   initialization. Resolved by extracting EL2 state initialization into
> >   __kvm_init_el2_state.
> > - Patch 5 (arm64: Revamp HCR_EL2.E2H RES1 detection) had conflicts in
> >   arch/arm64/include/asm/el2_setup.h. Resolved by using raw msr hcr_el2
> >   instead of the missing msr_hcr_el2 macro.
> > 
> > 
> > Marc Zyngier (4):
> >   arm64: sysreg: Add layout for ID_AA64MMFR4_EL1
> >   arm64: Treat HCR_EL2.E2H as RES1 when ID_AA64MMFR4_EL1.E2H0 is
> >     negative
> >   arm64: Fix early handling of FEAT_E2H0 not being implemented
> >   arm64: Revamp HCR_EL2.E2H RES1 detection
> > 
> > Mark Rutland (1):
> >   KVM: arm64: Initialize HCR_EL2.E2H early

Please go through and correct all of the SHA1s for the cherry-picks,
smells like some LLM just hallucinated some bits given how close they
are to the real deal.

I also want to see that all KVM modes have been tested (nVHE, hVHE, VHE,
protected) before this gets picked up. Overall though taking this to
stable seems like the right thing to do.

Any reason why you've only done 6.6? The kernel was first aware of
E2H=RES1 as far back as 5.13.

Thanks,
Oliver

Re: [PATCH 0/5] Backport ARM64 VHE boot fixes to 6.6.y

[Sasha Levin]

> This series backports VHE CPU boot fixes to the 6.6.y stable branch.
>
> These fixes are already present in the 6.12.y stable branch (and
> newer), but are missing in 6.6.y. They are required to enable booting
> L1 guests with nested virtualization enabled (kvm-arm.mode=nested).

Ugh, yes, the sha1s don't make sense.

While respinning, please also consider whether upstream 3855a7b91d42
("KVM: arm64: Initialize SCTLR_EL1 in __kvm_hyp_init_cpu()") should
join the series.

-- 
Thanks,
Sasha

Re: [PATCH 0/5] Backport ARM64 VHE boot fixes to 6.6.y

[Greg KH]

On Wed, Jul 01, 2026 at 08:43:37PM +0000, Colton Lewis wrote:
> This series backports VHE CPU boot fixes to the 6.6.y stable branch.
> 
> These fixes are already present in the 6.12.y stable branch (and
> newer), but are missing in 6.6.y. They are required to enable booting
> L1 guests with nested virtualization enabled (kvm-arm.mode=nested).
> 
> Without these patches, a 6.6.y guest boots with HCR_EL2.E2H
> incorrectly configured (because it misses VHE-only detection or early
> initialization), causing early boot hangs/trap loops.

Why is this needed for 6.6.y?  Why not just use 6.12.y and newer for
systems that require this new feature?  What is preventing that from
happening?

thanks,

greg k-h