[PATCH 0/2] KVM: arm64: Enable S1PIE for nVHE and hVHE

[PATCH 0/2] KVM: arm64: Enable S1PIE for nVHE and hVHE

[Mark Brown]

When Stage 1 Permission Indirection (FEAT_S1PIE) support was merged KVM
was updated to enable it in the VHE hypervisor but we do not currently
use it in the nVHE hypervisor.  Update the nVHE hypervisor to use it as
well, as with the initial merge of the S1PIE this should have no
immediate practical impact but it makes the nVHE and VHE hypervisors
more consistent and will be required for use of features like D128 and
GCS which depend on permission indirection.

Signed-off-by: Mark Brown <broonie@kernel.org>
---
Mark Brown (2):
      KVM: arm64: Initialise TCR2_EL2 for nVHE mode
      KVM: arm64: Enable S1PIE for nVHE and hVHE

 arch/arm64/include/asm/kvm_asm.h     |  2 ++
 arch/arm64/include/asm/kvm_pgtable.h | 26 ++++++++++++++++++++++++++
 arch/arm64/kernel/asm-offsets.c      |  2 ++
 arch/arm64/kvm/arm.c                 | 13 ++++++++++++-
 arch/arm64/kvm/hyp/nvhe/hyp-init.S   | 24 ++++++++++++++++++++++--
 arch/arm64/kvm/hyp/pgtable.c         |  7 +++++++
 6 files changed, 71 insertions(+), 3 deletions(-)
---
base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
change-id: 20260828-kvm-arm64-nvhe-pie-cb86c417f9ea

Best regards,
--  
Mark Brown <broonie@kernel.org>

[PATCH 1/2] KVM: arm64: Initialise TCR2_EL2 for nVHE mode

[Mark Brown]

We currently only configure TCR2_EL2 in VHE mode, this is done in
__finalise_el2 which only runs for VHE. While all systems with TCR2_EL2
should have VHE support users may wish to run them in nVHE mode, for
example in order to use protected VMs.

Determine the value to load for TCR2_EL2 in C code in a similar manner to
TCR_EL2, further patches will configure some bits in the register. When
resetting back to the hypervisor stub clear all bits in the register in
case something without support for TCR2_EL2 runs later.

The only practical impact should be if we are started with a misconfigured
TCR2_EL2.

Signed-off-by: Mark Brown <broonie@kernel.org>
---
 arch/arm64/include/asm/kvm_asm.h   |  1 +
 arch/arm64/kernel/asm-offsets.c    |  1 +
 arch/arm64/kvm/arm.c               |  5 ++++-
 arch/arm64/kvm/hyp/nvhe/hyp-init.S | 18 ++++++++++++++++--
 4 files changed, 22 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
index e5b92ac09e69..eb796436d6eb 100644
--- a/arch/arm64/include/asm/kvm_asm.h
+++ b/arch/arm64/include/asm/kvm_asm.h
@@ -208,6 +208,7 @@ extern void *__vhe_undefined_symbol;
 struct kvm_nvhe_init_params {
 	unsigned long mair_el2;
 	unsigned long tcr_el2;
+	unsigned long tcr2_el2;
 	unsigned long tpidr_el2;
 	unsigned long stack_hyp_va;
 	unsigned long stack_pa;
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 9c853ed3ceab..baffe58015d6 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -118,6 +118,7 @@ int main(void)
   DEFINE(HOST_DATA_CONTEXT,	offsetof(struct kvm_host_data, host_ctxt));
   DEFINE(NVHE_INIT_MAIR_EL2,	offsetof(struct kvm_nvhe_init_params, mair_el2));
   DEFINE(NVHE_INIT_TCR_EL2,	offsetof(struct kvm_nvhe_init_params, tcr_el2));
+  DEFINE(NVHE_INIT_TCR2_EL2,	offsetof(struct kvm_nvhe_init_params, tcr2_el2));
   DEFINE(NVHE_INIT_TPIDR_EL2,	offsetof(struct kvm_nvhe_init_params, tpidr_el2));
   DEFINE(NVHE_INIT_STACK_HYP_VA,	offsetof(struct kvm_nvhe_init_params, stack_hyp_va));
   DEFINE(NVHE_INIT_PGD_PA,	offsetof(struct kvm_nvhe_init_params, pgd_pa));
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 8b080804bc90..88eb0459ad4b 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -2158,7 +2158,7 @@ static int kvm_init_vector_slots(void)
 static void __init cpu_prepare_hyp_mode(int cpu, u32 hyp_va_bits)
 {
 	struct kvm_nvhe_init_params *params = per_cpu_ptr_nvhe_sym(kvm_init_params, cpu);
-	unsigned long tcr;
+	unsigned long tcr, tcr2;
 
 	/*
 	 * Calculate the raw per-cpu offset without a translation from the
@@ -2186,6 +2186,9 @@ static void __init cpu_prepare_hyp_mode(int cpu, u32 hyp_va_bits)
 	tcr |= TCR_T0SZ(hyp_va_bits);
 	params->tcr_el2 = tcr;
 
+	tcr2 = 0;
+	params->tcr2_el2 = tcr2;
+
 	params->pgd_pa = kvm_mmu_get_httbr();
 	if (is_protected_kvm_enabled())
 		params->hcr_el2 = HCR_HOST_NVHE_PROTECTED_FLAGS;
diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S
index 0b3e0b28dfc7..a39de9c20d27 100644
--- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S
+++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S
@@ -137,8 +137,13 @@ alternative_if ARM64_HAS_CNP
 alternative_else_nop_endif
 	msr	ttbr0_el2, x2
 
-	ldr	x0, [x0, #NVHE_INIT_TCR_EL2]
-	msr	tcr_el2, x0
+	ldr	x1, [x0, #NVHE_INIT_TCR_EL2]
+	msr	tcr_el2, x1
+
+alternative_if ARM64_HAS_TCR2
+	ldr	x1, [x0, #NVHE_INIT_TCR2_EL2]
+	msr	REG_TCR2_EL2, x1
+alternative_else_nop_endif
 
 	isb
 
@@ -250,6 +255,15 @@ reset:
 	mov_q	x5, INIT_SCTLR_EL2_MMU_OFF
 	pre_disable_mmu_workaround
 	msr	sctlr_el2, x5
+
+alternative_if ARM64_HAS_TCR2
+	/*
+	 * Disable any features we enabled in case the next user doesn't
+	 * have TCR2_EL2 support.
+	 */
+	msr	REG_TCR2_EL2, xzr
+alternative_else_nop_endif
+
 	isb
 
 	/* Install stub vectors */

-- 
2.47.3

[PATCH 2/2] KVM: arm64: Enable S1PIE for nVHE and hVHE

[Mark Brown]

When FEAT_S1PIE (stage 1 permission indirection) is supported we
currently enable and use it in the hypervisor when running in VHE mode
but not when running in nVHE or hVHE mode.  While systems with
FEAT_S1PIE would normally use VHE users can configure them for nVHE or
hVHE, for example in order to run protected guests.  Enable FEAT_S1PIE
with nVHE and hVHE.

AP[1] is one of the bits used to encode the indirected permissions.
Since for hVHE this is always 0 and for nVHE it is always 1 we only
configure the subset of indirected permissions that the system is
expected to use.

With permission indirection read only permissions must be encoded in the
bits used by PIE, set DBM for read only mappings.  Only do this when
using S1PIE, the hypervisor does not otherwise use DBM and if we were
actually using DBM it would be for writable mappings.

This should have no practical impact other than causing any unexpected
encodings to map to no permissions instead of their default meanings.
It will mean that the configuration is closer to that in VHE mode, and
will be required for future work enabling features like D128 and GCS
which are only available via indirection.

Signed-off-by: Mark Brown <broonie@kernel.org>
---
 arch/arm64/include/asm/kvm_asm.h     |  1 +
 arch/arm64/include/asm/kvm_pgtable.h | 26 ++++++++++++++++++++++++++
 arch/arm64/kernel/asm-offsets.c      |  1 +
 arch/arm64/kvm/arm.c                 |  8 ++++++++
 arch/arm64/kvm/hyp/nvhe/hyp-init.S   |  6 ++++++
 arch/arm64/kvm/hyp/pgtable.c         |  7 +++++++
 6 files changed, 49 insertions(+)

diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
index eb796436d6eb..bc587be33703 100644
--- a/arch/arm64/include/asm/kvm_asm.h
+++ b/arch/arm64/include/asm/kvm_asm.h
@@ -207,6 +207,7 @@ extern void *__vhe_undefined_symbol;
 
 struct kvm_nvhe_init_params {
 	unsigned long mair_el2;
+	unsigned long pir_el2;
 	unsigned long tcr_el2;
 	unsigned long tcr2_el2;
 	unsigned long tpidr_el2;
diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h
index 41a8687938eb..7c5c87c6745e 100644
--- a/arch/arm64/include/asm/kvm_pgtable.h
+++ b/arch/arm64/include/asm/kvm_pgtable.h
@@ -93,6 +93,7 @@ typedef u64 kvm_pte_t;
 
 #define KVM_PTE_LEAF_ATTR_HI_S2_XN	GENMASK(54, 53)
 
+#define KVM_PTE_LEAF_ATTR_HI_S1_DBM	BIT(51)
 #define KVM_PTE_LEAF_ATTR_HI_S1_GP	BIT(50)
 
 #define KVM_PTE_LEAF_ATTR_S2_PERMS	(KVM_PTE_LEAF_ATTR_LO_S2_S2AP_R | \
@@ -297,6 +298,31 @@ enum kvm_pgtable_prot {
 #define PAGE_HYP_RO		(KVM_PGTABLE_PROT_R)
 #define PAGE_HYP_DEVICE		(PAGE_HYP | KVM_PGTABLE_PROT_DEVICE)
 
+/*
+ * Permission indirection configuration for the nVHE hypervisor when we
+ * have FEAT_S1PIE. Like the host kernel we configure a mapping
+ * equivalent to the non-PIE meanings of the bits so the page table
+ * manipulation code does not need to account for PIE.
+ *
+ * Since nVHE and hVHE fix AP[1] as 1 or 0 respectively we define
+ * separate PIE mappings for each. These mappings are minimal with
+ * only things used from the hypervisor. Write permission is
+ * controlled via DBM.
+ */
+
+#define KVM_HYP_PIR_IDX(uxn, pxn, dbm, ap1) (((uxn) << 3) | ((pxn) << 2) | \
+					     ((dbm) << 1) | (ap1))
+
+#define KVM_NVHE_PIR_EL2        (					\
+       PIRx_ELx_PERM_PREP(KVM_HYP_PIR_IDX(0, 0, 0, 1), PIE_RX)	|	\
+       PIRx_ELx_PERM_PREP(KVM_HYP_PIR_IDX(1, 0, 0, 1), PIE_RW)	|	\
+       PIRx_ELx_PERM_PREP(KVM_HYP_PIR_IDX(1, 0, 1, 1), PIE_R))
+
+#define KVM_HVHE_PIR_EL2        (					\
+       PIRx_ELx_PERM_PREP(KVM_HYP_PIR_IDX(0, 0, 0, 0), PIE_RX)	|	\
+       PIRx_ELx_PERM_PREP(KVM_HYP_PIR_IDX(1, 1, 1, 0), PIE_R)	|	\
+       PIRx_ELx_PERM_PREP(KVM_HYP_PIR_IDX(1, 1, 0, 0), PIE_RW))
+
 typedef bool (*kvm_pgtable_force_pte_cb_t)(u64 addr, u64 end,
 					   enum kvm_pgtable_prot prot);
 
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index baffe58015d6..fab949435aae 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -117,6 +117,7 @@ int main(void)
   DEFINE(HOST_CONTEXT_VCPU,	offsetof(struct kvm_cpu_context, __hyp_running_vcpu));
   DEFINE(HOST_DATA_CONTEXT,	offsetof(struct kvm_host_data, host_ctxt));
   DEFINE(NVHE_INIT_MAIR_EL2,	offsetof(struct kvm_nvhe_init_params, mair_el2));
+  DEFINE(NVHE_INIT_PIR_EL2,	offsetof(struct kvm_nvhe_init_params, pir_el2));
   DEFINE(NVHE_INIT_TCR_EL2,	offsetof(struct kvm_nvhe_init_params, tcr_el2));
   DEFINE(NVHE_INIT_TCR2_EL2,	offsetof(struct kvm_nvhe_init_params, tcr2_el2));
   DEFINE(NVHE_INIT_TPIDR_EL2,	offsetof(struct kvm_nvhe_init_params, tpidr_el2));
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 88eb0459ad4b..9232b4581723 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -2187,6 +2187,14 @@ static void __init cpu_prepare_hyp_mode(int cpu, u32 hyp_va_bits)
 	params->tcr_el2 = tcr;
 
 	tcr2 = 0;
+	if (cpus_have_final_cap(ARM64_HAS_S1PIE)) {
+		if (cpus_have_final_cap(ARM64_KVM_HVHE))
+			params->pir_el2 = KVM_HVHE_PIR_EL2;
+		else
+			params->pir_el2 = KVM_NVHE_PIR_EL2;
+
+		tcr2 |= TCR2_EL2_PIE;
+	}
 	params->tcr2_el2 = tcr2;
 
 	params->pgd_pa = kvm_mmu_get_httbr();
diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S
index a39de9c20d27..20f926276688 100644
--- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S
+++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S
@@ -140,6 +140,12 @@ alternative_else_nop_endif
 	ldr	x1, [x0, #NVHE_INIT_TCR_EL2]
 	msr	tcr_el2, x1
 
+alternative_if ARM64_HAS_S1PIE
+	ldr	x1, [x0, #NVHE_INIT_PIR_EL2]
+	msr	REG_PIR_EL2, x1
+	msr	REG_PIRE0_EL2, xzr
+alternative_else_nop_endif
+
 alternative_if ARM64_HAS_TCR2
 	ldr	x1, [x0, #NVHE_INIT_TCR2_EL2]
 	msr	REG_TCR2_EL2, x1
diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c
index b74dd5ce1efd..fd6854913549 100644
--- a/arch/arm64/kvm/hyp/pgtable.c
+++ b/arch/arm64/kvm/hyp/pgtable.c
@@ -349,6 +349,13 @@ static int hyp_set_prot_attr(enum kvm_pgtable_prot prot, kvm_pte_t *ptep)
 
 		if (system_supports_bti_kernel())
 			attr |= KVM_PTE_LEAF_ATTR_HI_S1_GP;
+	} else if (cpus_have_final_cap(ARM64_HAS_S1PIE) &&
+		   !(prot & KVM_PGTABLE_PROT_W)) {
+		/*
+		 * When using S1PIE for nVHE set DBM for read only
+		 * mappings since AP[2] is ineffective.
+		 */
+		attr |= KVM_PTE_LEAF_ATTR_HI_S1_DBM;
 	}
 
 	if (cpus_have_final_cap(ARM64_KVM_HVHE)) {

-- 
2.47.3

Re: [PATCH 2/2] KVM: arm64: Enable S1PIE for nVHE and hVHE

[Marc Zyngier]

On Fri, 04 Sep 2026 00:14:42 +0100,
Mark Brown <broonie@kernel.org> wrote:
> 
> When FEAT_S1PIE (stage 1 permission indirection) is supported we
> currently enable and use it in the hypervisor when running in VHE mode
> but not when running in nVHE or hVHE mode.  While systems with
> FEAT_S1PIE would normally use VHE users can configure them for nVHE or
> hVHE, for example in order to run protected guests.  Enable FEAT_S1PIE
> with nVHE and hVHE.

No. nVHE is dead, and I'm not adding support for random stuff that
appeared over 10 years after v8.0. If you have S1PIE, you have VHE.

If you force the kernel to run with E2H==0, that's your own problem.

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCH 1/2] KVM: arm64: Initialise TCR2_EL2 for nVHE mode

[Marc Zyngier]

On Fri, 04 Sep 2026 00:14:41 +0100,
Mark Brown <broonie@kernel.org> wrote:
> 
> We currently only configure TCR2_EL2 in VHE mode, this is done in
> __finalise_el2 which only runs for VHE. While all systems with TCR2_EL2
> should have VHE support users may wish to run them in nVHE mode, for
> example in order to use protected VMs.
> 
> Determine the value to load for TCR2_EL2 in C code in a similar manner to
> TCR_EL2, further patches will configure some bits in the register. When
> resetting back to the hypervisor stub clear all bits in the register in
> case something without support for TCR2_EL2 runs later.
> 
> The only practical impact should be if we are started with a misconfigured
> TCR2_EL2.
> 
> Signed-off-by: Mark Brown <broonie@kernel.org>
> ---
>  arch/arm64/include/asm/kvm_asm.h   |  1 +
>  arch/arm64/kernel/asm-offsets.c    |  1 +
>  arch/arm64/kvm/arm.c               |  5 ++++-
>  arch/arm64/kvm/hyp/nvhe/hyp-init.S | 18 ++++++++++++++++--
>  4 files changed, 22 insertions(+), 3 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
> index e5b92ac09e69..eb796436d6eb 100644
> --- a/arch/arm64/include/asm/kvm_asm.h
> +++ b/arch/arm64/include/asm/kvm_asm.h
> @@ -208,6 +208,7 @@ extern void *__vhe_undefined_symbol;
>  struct kvm_nvhe_init_params {
>  	unsigned long mair_el2;
>  	unsigned long tcr_el2;
> +	unsigned long tcr2_el2;
>  	unsigned long tpidr_el2;
>  	unsigned long stack_hyp_va;
>  	unsigned long stack_pa;
> diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
> index 9c853ed3ceab..baffe58015d6 100644
> --- a/arch/arm64/kernel/asm-offsets.c
> +++ b/arch/arm64/kernel/asm-offsets.c
> @@ -118,6 +118,7 @@ int main(void)
>    DEFINE(HOST_DATA_CONTEXT,	offsetof(struct kvm_host_data, host_ctxt));
>    DEFINE(NVHE_INIT_MAIR_EL2,	offsetof(struct kvm_nvhe_init_params, mair_el2));
>    DEFINE(NVHE_INIT_TCR_EL2,	offsetof(struct kvm_nvhe_init_params, tcr_el2));
> +  DEFINE(NVHE_INIT_TCR2_EL2,	offsetof(struct kvm_nvhe_init_params, tcr2_el2));
>    DEFINE(NVHE_INIT_TPIDR_EL2,	offsetof(struct kvm_nvhe_init_params, tpidr_el2));
>    DEFINE(NVHE_INIT_STACK_HYP_VA,	offsetof(struct kvm_nvhe_init_params, stack_hyp_va));
>    DEFINE(NVHE_INIT_PGD_PA,	offsetof(struct kvm_nvhe_init_params, pgd_pa));
> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> index 8b080804bc90..88eb0459ad4b 100644
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
> @@ -2158,7 +2158,7 @@ static int kvm_init_vector_slots(void)
>  static void __init cpu_prepare_hyp_mode(int cpu, u32 hyp_va_bits)
>  {
>  	struct kvm_nvhe_init_params *params = per_cpu_ptr_nvhe_sym(kvm_init_params, cpu);
> -	unsigned long tcr;
> +	unsigned long tcr, tcr2;
>  
>  	/*
>  	 * Calculate the raw per-cpu offset without a translation from the
> @@ -2186,6 +2186,9 @@ static void __init cpu_prepare_hyp_mode(int cpu, u32 hyp_va_bits)
>  	tcr |= TCR_T0SZ(hyp_va_bits);
>  	params->tcr_el2 = tcr;
>  
> +	tcr2 = 0;
> +	params->tcr2_el2 = tcr2;
> +
>  	params->pgd_pa = kvm_mmu_get_httbr();
>  	if (is_protected_kvm_enabled())
>  		params->hcr_el2 = HCR_HOST_NVHE_PROTECTED_FLAGS;
> diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S
> index 0b3e0b28dfc7..a39de9c20d27 100644
> --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S
> +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S
> @@ -137,8 +137,13 @@ alternative_if ARM64_HAS_CNP
>  alternative_else_nop_endif
>  	msr	ttbr0_el2, x2
>  
> -	ldr	x0, [x0, #NVHE_INIT_TCR_EL2]
> -	msr	tcr_el2, x0
> +	ldr	x1, [x0, #NVHE_INIT_TCR_EL2]
> +	msr	tcr_el2, x1
> +
> +alternative_if ARM64_HAS_TCR2
> +	ldr	x1, [x0, #NVHE_INIT_TCR2_EL2]
> +	msr	REG_TCR2_EL2, x1
> +alternative_else_nop_endif
>  
>  	isb
>  
> @@ -250,6 +255,15 @@ reset:
>  	mov_q	x5, INIT_SCTLR_EL2_MMU_OFF
>  	pre_disable_mmu_workaround
>  	msr	sctlr_el2, x5
> +
> +alternative_if ARM64_HAS_TCR2
> +	/*
> +	 * Disable any features we enabled in case the next user doesn't
> +	 * have TCR2_EL2 support.
> +	 */
> +	msr	REG_TCR2_EL2, xzr
> +alternative_else_nop_endif
> +

I don't see the point of this. The MMU is off, and if the next piece
of SW can't correctly initialise the HW, that's its problem.

And with this hunk gone, the rest should be moved to the following
patch.

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCH 2/2] KVM: arm64: Enable S1PIE for nVHE and hVHE

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 992 bytes --]

On Fri, Sep 04, 2026 at 08:25:58AM +0100, Marc Zyngier wrote:
> Mark Brown <broonie@kernel.org> wrote:

> > When FEAT_S1PIE (stage 1 permission indirection) is supported we
> > currently enable and use it in the hypervisor when running in VHE mode
> > but not when running in nVHE or hVHE mode.  While systems with
> > FEAT_S1PIE would normally use VHE users can configure them for nVHE or
> > hVHE, for example in order to run protected guests.  Enable FEAT_S1PIE
> > with nVHE and hVHE.

> No. nVHE is dead, and I'm not adding support for random stuff that
> appeared over 10 years after v8.0. If you have S1PIE, you have VHE.

> If you force the kernel to run with E2H==0, that's your own problem.

What about pKVM, my understanding was that that is very much an ongoing
concern and uses the nVHE hypervisor?  My expectation was that this
would only ever get used in hVHE configurations when people enable pKVM
on modern hardware, actual nVHE mode was mostly just carried along with
that.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 2/2] KVM: arm64: Enable S1PIE for nVHE and hVHE

[Will Deacon]

On Fri, Sep 04, 2026 at 11:33:24AM +0100, Mark Brown wrote:
> On Fri, Sep 04, 2026 at 08:25:58AM +0100, Marc Zyngier wrote:
> > Mark Brown <broonie@kernel.org> wrote:
> 
> > > When FEAT_S1PIE (stage 1 permission indirection) is supported we
> > > currently enable and use it in the hypervisor when running in VHE mode
> > > but not when running in nVHE or hVHE mode.  While systems with
> > > FEAT_S1PIE would normally use VHE users can configure them for nVHE or
> > > hVHE, for example in order to run protected guests.  Enable FEAT_S1PIE
> > > with nVHE and hVHE.
> 
> > No. nVHE is dead, and I'm not adding support for random stuff that
> > appeared over 10 years after v8.0. If you have S1PIE, you have VHE.
> 
> > If you force the kernel to run with E2H==0, that's your own problem.
> 
> What about pKVM, my understanding was that that is very much an ongoing
> concern and uses the nVHE hypervisor?  My expectation was that this
> would only ever get used in hVHE configurations when people enable pKVM
> on modern hardware, actual nVHE mode was mostly just carried along with
> that.

pKVM uses hVHE wherever it can nowadays, so I'm fine with only supporting
S1PIE on hVHE and VHE configurations if that's easier to maintain. We
just need to make sure that the failure mode for nVHE results in the
feature being gracefully disabled, rather than e.g. crashing or quietly
doing the wrong thing.

Will

Re: [PATCH 1/2] KVM: arm64: Initialise TCR2_EL2 for nVHE mode

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 928 bytes --]

On Fri, Sep 04, 2026 at 08:32:04AM +0100, Marc Zyngier wrote:
> Mark Brown <broonie@kernel.org> wrote:

> > +alternative_if ARM64_HAS_TCR2
> > +	/*
> > +	 * Disable any features we enabled in case the next user doesn't
> > +	 * have TCR2_EL2 support.
> > +	 */
> > +	msr	REG_TCR2_EL2, xzr
> > +alternative_else_nop_endif

> I don't see the point of this. The MMU is off, and if the next piece
> of SW can't correctly initialise the HW, that's its problem.

So we don't care about reverting back to users who are just plain
unaware of the register?  In practice for Linux that's fairly old
kernels, but people might feel adverturous and try or try some non-Linux
thing.  This stuff is all a bit edge casey with what we support.

> And with this hunk gone, the rest should be moved to the following
> patch.

I can do that, I kept it separate since TCR2_EL2 has some other controls
in it and people might want this independently.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 2/2] KVM: arm64: Enable S1PIE for nVHE and hVHE

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1041 bytes --]

On Fri, Sep 04, 2026 at 12:29:23PM +0100, Will Deacon wrote:
> On Fri, Sep 04, 2026 at 11:33:24AM +0100, Mark Brown wrote:

> > What about pKVM, my understanding was that that is very much an ongoing
> > concern and uses the nVHE hypervisor?  My expectation was that this
> > would only ever get used in hVHE configurations when people enable pKVM
> > on modern hardware, actual nVHE mode was mostly just carried along with
> > that.

> pKVM uses hVHE wherever it can nowadays, so I'm fine with only supporting
> S1PIE on hVHE and VHE configurations if that's easier to maintain. We
> just need to make sure that the failure mode for nVHE results in the
> feature being gracefully disabled, rather than e.g. crashing or quietly
> doing the wrong thing.

I figure that if we're supporting hVHE it will end up less complicated
to also enable nVHE, it reduces the potential for having bad or missing
fallback paths for features downstream of S1POE.  It's not like it's a
huge extra bit of code and it seems likely to save hassle down the line.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 2/2] KVM: arm64: Enable S1PIE for nVHE and hVHE

[Fuad Tabba]

On Fri, 4 Sept 2026 at 12:48, Mark Brown <broonie@kernel.org> wrote:
>
> On Fri, Sep 04, 2026 at 12:29:23PM +0100, Will Deacon wrote:
> > On Fri, Sep 04, 2026 at 11:33:24AM +0100, Mark Brown wrote:
>
> > > What about pKVM, my understanding was that that is very much an ongoing
> > > concern and uses the nVHE hypervisor?  My expectation was that this
> > > would only ever get used in hVHE configurations when people enable pKVM
> > > on modern hardware, actual nVHE mode was mostly just carried along with
> > > that.
>
> > pKVM uses hVHE wherever it can nowadays, so I'm fine with only supporting
> > S1PIE on hVHE and VHE configurations if that's easier to maintain. We
> > just need to make sure that the failure mode for nVHE results in the
> > feature being gracefully disabled, rather than e.g. crashing or quietly
> > doing the wrong thing.
>
> I figure that if we're supporting hVHE it will end up less complicated
> to also enable nVHE, it reduces the potential for having bad or missing
> fallback paths for features downstream of S1POE.  It's not like it's a
> huge extra bit of code and it seems likely to save hassle down the line.

For pKVM there's no fallback path to get wrong: any CPU with S1PIE has
VHE, so pKVM runs hVHE on it, and a CPU without VHE has no S1PIE to
disable. I couldn't construct the protected nVHE plus S1PIE case under
QEMU at all.

Where I think it applies is kvm-arm.mode=nvhe on S1PIE hardware. That
boots, and it's the one configuration where the nVHE side of the patch
would run.

Cheers,
/fuad

Re: [PATCH 1/2] KVM: arm64: Initialise TCR2_EL2 for nVHE mode

[Marc Zyngier]

On Fri, 04 Sep 2026 12:35:22 +0100,
Mark Brown <broonie@kernel.org> wrote:
> 
> On Fri, Sep 04, 2026 at 08:32:04AM +0100, Marc Zyngier wrote:
> > Mark Brown <broonie@kernel.org> wrote:
> 
> > > +alternative_if ARM64_HAS_TCR2
> > > +	/*
> > > +	 * Disable any features we enabled in case the next user doesn't
> > > +	 * have TCR2_EL2 support.
> > > +	 */
> > > +	msr	REG_TCR2_EL2, xzr
> > > +alternative_else_nop_endif
> 
> > I don't see the point of this. The MMU is off, and if the next piece
> > of SW can't correctly initialise the HW, that's its problem.
> 
> So we don't care about reverting back to users who are just plain
> unaware of the register?  In practice for Linux that's fairly old
> kernels, but people might feel adverturous and try or try some non-Linux
> thing.  This stuff is all a bit edge casey with what we support.

I certainly do not care. If you kexec an ancient kernel on new HW and
can't correctly initialise it, you lose. Don't do that.

	M.

-- 
Jazz isn't dead. It just smells funny.

Re: [PATCH 2/2] KVM: arm64: Enable S1PIE for nVHE and hVHE

[Marc Zyngier]

On Fri, 04 Sep 2026 12:48:14 +0100,
Mark Brown <broonie@kernel.org> wrote:
> 
> On Fri, Sep 04, 2026 at 12:29:23PM +0100, Will Deacon wrote:
> > On Fri, Sep 04, 2026 at 11:33:24AM +0100, Mark Brown wrote:
> 
> > > What about pKVM, my understanding was that that is very much an ongoing
> > > concern and uses the nVHE hypervisor?  My expectation was that this
> > > would only ever get used in hVHE configurations when people enable pKVM
> > > on modern hardware, actual nVHE mode was mostly just carried along with
> > > that.
> 
> > pKVM uses hVHE wherever it can nowadays, so I'm fine with only supporting
> > S1PIE on hVHE and VHE configurations if that's easier to maintain. We
> > just need to make sure that the failure mode for nVHE results in the
> > feature being gracefully disabled, rather than e.g. crashing or quietly
> > doing the wrong thing.
> 
> I figure that if we're supporting hVHE it will end up less complicated
> to also enable nVHE, it reduces the potential for having bad or missing
> fallback paths for features downstream of S1POE.  It's not like it's a
> huge extra bit of code and it seems likely to save hassle down the line.

It's not about the code size. it's about the code being *dead*. Nobody
is going to maintain this configuration, because this is a silly one.

KVM with E2H==0 has reached its architectural peak years ago. We're
not adding anything to it anymore, and all new architectural bits
require running with E2H==1.

	M.

-- 
Jazz isn't dead. It just smells funny.

Re: [PATCH 2/2] KVM: arm64: Enable S1PIE for nVHE and hVHE

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1558 bytes --]

On Fri, Sep 04, 2026 at 01:20:44PM +0100, Fuad Tabba wrote:
> On Fri, 4 Sept 2026 at 12:48, Mark Brown <broonie@kernel.org> wrote:

> > I figure that if we're supporting hVHE it will end up less complicated
> > to also enable nVHE, it reduces the potential for having bad or missing
> > fallback paths for features downstream of S1POE.  It's not like it's a
> > huge extra bit of code and it seems likely to save hassle down the line.

> For pKVM there's no fallback path to get wrong: any CPU with S1PIE has
> VHE, so pKVM runs hVHE on it, and a CPU without VHE has no S1PIE to
> disable. I couldn't construct the protected nVHE plus S1PIE case under
> QEMU at all.

> Where I think it applies is kvm-arm.mode=nvhe on S1PIE hardware. That
> boots, and it's the one configuration where the nVHE side of the patch
> would run.

Yes, exactly.  As you say any system with S1PIE would only run actual
nNVHE mode if it was requested on the command line.  Realistically this
is something that people end up doing relatively often in development
even though it is not useful for production, so as Will said it should
run.  It seemed explicitly excluding supporting the nVHE case was more
likely to trip people up one way or another.

Marc also started out with a flat no and didn't mention anything
positive about the hVHE part so it appeared that he was objecting to the
whole concept of the patch, including the hVHE aspect.  It seems you and
Will are both OK with the hVHE bit and indifferent to the nVHE case, and
I can certainly rework to unsupport nVHE mode.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 2/2] KVM: arm64: Enable S1PIE for nVHE and hVHE

[Will Deacon]

On Fri, Sep 04, 2026 at 01:20:44PM +0100, Fuad Tabba wrote:
> On Fri, 4 Sept 2026 at 12:48, Mark Brown <broonie@kernel.org> wrote:
> >
> > On Fri, Sep 04, 2026 at 12:29:23PM +0100, Will Deacon wrote:
> > > On Fri, Sep 04, 2026 at 11:33:24AM +0100, Mark Brown wrote:
> >
> > > > What about pKVM, my understanding was that that is very much an ongoing
> > > > concern and uses the nVHE hypervisor?  My expectation was that this
> > > > would only ever get used in hVHE configurations when people enable pKVM
> > > > on modern hardware, actual nVHE mode was mostly just carried along with
> > > > that.
> >
> > > pKVM uses hVHE wherever it can nowadays, so I'm fine with only supporting
> > > S1PIE on hVHE and VHE configurations if that's easier to maintain. We
> > > just need to make sure that the failure mode for nVHE results in the
> > > feature being gracefully disabled, rather than e.g. crashing or quietly
> > > doing the wrong thing.
> >
> > I figure that if we're supporting hVHE it will end up less complicated
> > to also enable nVHE, it reduces the potential for having bad or missing
> > fallback paths for features downstream of S1POE.  It's not like it's a
> > huge extra bit of code and it seems likely to save hassle down the line.
> 
> For pKVM there's no fallback path to get wrong: any CPU with S1PIE has
> VHE, so pKVM runs hVHE on it, and a CPU without VHE has no S1PIE to
> disable. I couldn't construct the protected nVHE plus S1PIE case under
> QEMU at all.

I can't remember the exact incantation, but there's definitely a way to
force pKVM + nVHE on VHE-capable hardware on the cmdline. So we should
just make sure that doesn't go horribly wrong!

Will

Re: [PATCH 2/2] KVM: arm64: Enable S1PIE for nVHE and hVHE

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 289 bytes --]

On Fri, Sep 04, 2026 at 02:23:32PM +0100, Marc Zyngier wrote:

> KVM with E2H==0 has reached its architectural peak years ago. We're
> not adding anything to it anymore, and all new architectural bits
> require running with E2H==1.

OK, so you're OK with a hVHE only version of this then?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 2/2] KVM: arm64: Enable S1PIE for nVHE and hVHE

[Marc Zyngier]

On Fri, 04 Sep 2026 14:38:07 +0100,
Mark Brown <broonie@kernel.org> wrote:
> 
> Marc also started out with a flat no and didn't mention anything
> positive about the hVHE part so it appeared that he was objecting to the
> whole concept of the patch, including the hVHE aspect.

Then you should read my email again.

	M.

-- 
Jazz isn't dead. It just smells funny.

Re: [PATCH 2/2] KVM: arm64: Enable S1PIE for nVHE and hVHE

[Alexandru Elisei]

Hi Will,

On Fri, Sep 04, 2026 at 02:41:28PM +0100, Will Deacon wrote:
> On Fri, Sep 04, 2026 at 01:20:44PM +0100, Fuad Tabba wrote:
> > On Fri, 4 Sept 2026 at 12:48, Mark Brown <broonie@kernel.org> wrote:
> > >
> > > On Fri, Sep 04, 2026 at 12:29:23PM +0100, Will Deacon wrote:
> > > > On Fri, Sep 04, 2026 at 11:33:24AM +0100, Mark Brown wrote:
> > >
> > > > > What about pKVM, my understanding was that that is very much an ongoing
> > > > > concern and uses the nVHE hypervisor?  My expectation was that this
> > > > > would only ever get used in hVHE configurations when people enable pKVM
> > > > > on modern hardware, actual nVHE mode was mostly just carried along with
> > > > > that.
> > >
> > > > pKVM uses hVHE wherever it can nowadays, so I'm fine with only supporting
> > > > S1PIE on hVHE and VHE configurations if that's easier to maintain. We
> > > > just need to make sure that the failure mode for nVHE results in the
> > > > feature being gracefully disabled, rather than e.g. crashing or quietly
> > > > doing the wrong thing.
> > >
> > > I figure that if we're supporting hVHE it will end up less complicated
> > > to also enable nVHE, it reduces the potential for having bad or missing
> > > fallback paths for features downstream of S1POE.  It's not like it's a
> > > huge extra bit of code and it seems likely to save hassle down the line.
> > 
> > For pKVM there's no fallback path to get wrong: any CPU with S1PIE has
> > VHE, so pKVM runs hVHE on it, and a CPU without VHE has no S1PIE to
> > disable. I couldn't construct the protected nVHE plus S1PIE case under
> > QEMU at all.
> 
> I can't remember the exact incantation, but there's definitely a way to
> force pKVM + nVHE on VHE-capable hardware on the cmdline. So we should
> just make sure that doesn't go horribly wrong!

arm64_sw.hvhe=0 id_aa64mmfr1.vh=0

From kernel-paramters.txt, and last time I checked (two or three kernel versions
ago) it was working.

Thanks,
Ale

Re: [PATCH 2/2] KVM: arm64: Enable S1PIE for nVHE and hVHE

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 519 bytes --]

On Fri, Sep 04, 2026 at 02:57:23PM +0100, Marc Zyngier wrote:
> Mark Brown <broonie@kernel.org> wrote:

> > Marc also started out with a flat no and didn't mention anything
> > positive about the hVHE part so it appeared that he was objecting to the
> > whole concept of the patch, including the hVHE aspect.

> Then you should read my email again.

It is true that you only explicitly mentioned nVHE which doesn't exclude
hVHE, but the silence on hVHE made the position there unclear and the
tone was rather negative.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]