[PATCH 00/28] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3

[PATCH 00/28] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3

[Marc Zyngier]

This series adds support for two extensions targeting Nested Virt on
arm64:

- FEAT_NV2p1 is effectively a bug fix for two registers (CNTHCTL_EL2
  and CPTR_EL2) that are missing stateful bits when accessed from EL1
  in a NV configuration. When this is present, the hypervisor can
  avoid a bunch of traps.

- FEAT_NV3 is much more ambitious, and changes the way ERET behaves in
  a NV environment. By moving EL1 accesses to HCR_EL2 from memory (via
  VNCR) to a dedicated register (NVHCR_EL2), the HW can detect whether
  the guest is performing an ERET for itself (NVHCR_EL2.TGE==1) or to
  its own guest (NVHCR_EL2.TGE==0). In the former case, ERET is done
  directly, and no trap occurs. Similar optimisations are available
  for a class of TLBI instructions.

The whole thing has been tested on an FVP model, and shown measurable
improvements for an L1 guest (about 1.5% fewer instructions).

Given that this isn't very convincing on its own, I have built an
approximate emulation of FEAT_NV3 that L1 (and deeper levels) can use
on actual production hardware. For these deeper levels, the numbers
are in the double digit of percentage point reduction (those
interested can look at the patches in the kvm-arm64/nv3 branch in my
tree).

Does it make NV better? Yes!
Does it make NV good? Get real!

Anyway, patches on top of -rc1 plus the current state of kvmarm/fixes.

Marc Zyngier (28):
  arm64: sysreg: Emit RESx/UNKN values for Mapping definitions
  arm64: Update ID_AA64MMFR4_EL1 description to 2026-03 JSON release
  KVM: arm64: Merge guest's HCRX_EL2 using NV_HCRX_GUEST_EXCLUDE
  KVM: arm64: Drop __HCRX_EL2_* masks
  KVM: arm64: Plumb HCRX_EL2.SRMASKEn in HCRX_EL2 sanitisation
  KVM: arm64: Classify CPTR_EL2 as a SR_LOC_SPECIAL register
  KVM: arm64: Don't evaluate HCR_EL2.NV on ERET fast path
  arm64: Add ARM64_HAS_NV2P1 capability
  KVM: arm64: Relax CPTR_EL2 handling when FEAT_NV2p1 is present
  KVM: arm64: Relax CNTHCTL_EL2 handling when FEAT_NV2p1 is present
  KVM: arm64: Expose FEAT_NV2p1 to NV guests
  arm64: Add FEAT_NV2p1 detection
  arm64: sysreg: Add NVHCR_EL2 description as a mirror of HCR_EL2
  arm64: sysreg: Add HCRX_EL2 bits related to FEAT_NV3
  arm64: Add ARM64_HAS_NV3 capability
  KVM: arm64: Split NV-specific exit fixups from the non-NV handling
  KVM: arm64: Add NV3 control bits to HCRX_EL2 sanitisation
  KVM: arm64: Add kvm_has_nv{2,3}() predicates
  KVM: arm64: Make HCR_EL2 a non-VNCR register
  KVM: arm64: Add sanitisation for NVHCR_EL2
  KVM: arm64: Add NVHCR_EL2 handling to the sysreg array
  KVM: arm64: Add routing for NVHCR_EL2 trap
  KVM: arm64: Add NVHCR_EL2 context switching
  KVM: arm64: Engage NV3 ERET trap elision
  KVM: arm64: Engage NV3 TLBI trap elision
  KVM: arm64: Add FEAT_NV3 detection
  KVM: arm64: Expose FEAT_NV3 to guests
  arm64: Add override for ID_AA64MMFR4_EL1.NV_frac

 arch/arm64/include/asm/cpufeature.h        |  1 +
 arch/arm64/include/asm/kvm_arm.h           | 15 ------
 arch/arm64/include/asm/kvm_emulate.h       | 41 +++++++++++++++-
 arch/arm64/include/asm/kvm_host.h          |  3 +-
 arch/arm64/include/asm/vncr_mapping.h      |  2 +-
 arch/arm64/kernel/cpufeature.c             | 18 ++++++-
 arch/arm64/kernel/image-vars.h             |  1 +
 arch/arm64/kernel/pi/idreg-override.c      | 10 ++++
 arch/arm64/kvm/arch_timer.c                | 10 +++-
 arch/arm64/kvm/config.c                    | 25 +++++++++-
 arch/arm64/kvm/emulate-nested.c            | 16 ++++--
 arch/arm64/kvm/hyp/include/hyp/switch.h    | 27 ++++++++--
 arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 11 +++++
 arch/arm64/kvm/hyp/vhe/switch.c            | 44 +++++++++++++----
 arch/arm64/kvm/hyp/vhe/sysreg-sr.c         | 21 +++++---
 arch/arm64/kvm/nested.c                    | 14 +++++-
 arch/arm64/kvm/sys_regs.c                  | 57 ++++++++++++++++++++--
 arch/arm64/tools/cpucaps                   |  2 +
 arch/arm64/tools/gen-sysreg.awk            | 14 ++++--
 arch/arm64/tools/sysreg                    | 42 ++++++++++++++--
 20 files changed, 313 insertions(+), 61 deletions(-)

-- 
2.47.3

[PATCH 01/28] arm64: sysreg: Emit RESx/UNKN values for Mapping definitions

[Marc Zyngier]

The sysreg file is using the Mapping qualifier to indicate that
a given encoding is only a mapping to a particular register.
As a result, we don't output any definition, and instead expect
the canonical definitions to be used.

This works rather well for individual fields, but creates problems
for macros that refer to more generic classes of bits such as RESx.

Relax the above rule by emitting the RESx and UNKN values for Mapping
qualifiers as well.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/tools/gen-sysreg.awk | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/tools/gen-sysreg.awk b/arch/arm64/tools/gen-sysreg.awk
index 86860ab672dc7..d7f7ceb768fe5 100755
--- a/arch/arm64/tools/gen-sysreg.awk
+++ b/arch/arm64/tools/gen-sysreg.awk
@@ -228,7 +228,7 @@ $1 == "EndSysreg" && block_current() == "Sysreg" {
 }
 
 # Currently this is effectivey a comment, in future we may want to emit
-# defines for the fields.
+# defines for the fields. "Mapping" does emit the RESx/UNKN definitions.
 ($1 == "Fields" || $1 == "Mapping") && block_current() == "Sysreg" {
 	expect_fields(2)
 
@@ -239,9 +239,15 @@ $1 == "EndSysreg" && block_current() == "Sysreg" {
 	print ""
 
 	next_bit = -1
-	res0 = null
-	res1 = null
-	unkn = null
+	if ($1 == "Mapping") {
+		res0 = $2 "_RES0"
+		res1 = $2 "_RES1"
+		unkn = $2 "_UNKN"
+	} else {
+		res0 = null
+		res1 = null
+		unkn = null
+	}
 
 	next
 }
-- 
2.47.3

[PATCH 02/28] arm64: Update ID_AA64MMFR4_EL1 description to 2026-03 JSON release

[Marc Zyngier]

ID_AA64MMFR4_EL1 has gained a few fields and enum values in the past
few months, so resync its definition with the 2026-03 JSON release.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/tools/sysreg | 30 +++++++++++++++++++++++++++---
 1 file changed, 27 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/tools/sysreg b/arch/arm64/tools/sysreg
index bc1788b1662b7..32e2f9856768b 100644
--- a/arch/arm64/tools/sysreg
+++ b/arch/arm64/tools/sysreg
@@ -2386,17 +2386,40 @@ EndEnum
 EndSysreg
 
 Sysreg	ID_AA64MMFR4_EL1	3	0	0	7	4
-Res0	63:48
+UnsignedEnum	63:60	MTEFGT
+	0b0000	NI
+	0b0001	IMP
+EndEnum
+UnsignedEnum	59:56	SCRX
+	0b0000	NI
+	0b0001	IMP
+EndEnum
+UnsignedEnum	55:52	TEV
+	0b0000	NI
+	0b0001	IMP
+EndEnum
+UnsignedEnum	51:48	TPS
+	0b0000	VAL_0000
+	0b0001	VAL_0001
+	0b0010	VAL_0010
+EndEnum
 UnsignedEnum	47:44	SRMASK
 	0b0000	NI
 	0b0001	IMP
+	0b0010	SRMASK2
+EndEnum
+UnsignedEnum	43:40	TLBID
+	0b0000	NI
+	0b0001	IMP
 EndEnum
-Res0	43:40
 UnsignedEnum	39:36	E3DSE
 	0b0000	NI
 	0b0001	IMP
 EndEnum
-Res0	35:32
+UnsignedEnum	35:32	EAESR
+	0b0000	NI
+	0b0001	IMP
+EndEnum
 UnsignedEnum	31:28	RMEGDI
 	0b0000	NI
 	0b0001	IMP
@@ -2410,6 +2433,7 @@ UnsignedEnum	23:20	NV_frac
 	0b0000	NV_NV2
 	0b0001	NV2_ONLY
 	0b0010	NV2P1
+	0b0011	NV3
 EndEnum
 UnsignedEnum	19:16	FGWTE3
 	0b0000	NI
-- 
2.47.3

[PATCH 03/28] KVM: arm64: Merge guest's HCRX_EL2 using NV_HCRX_GUEST_EXCLUDE

[Marc Zyngier]

The way we merge the guest-provided HCRX_EL2 value with the host's
is bonkers. We try to make it look like the FGT registers by using
positive and negative polarities for traps, but most of these bits
are not strictly about trapping, as they actively change the way
some architectural state is managed.

It would be far better to deal with these bits like we do for
HCR_EL2, by enumerating the list of bits we don't allow the guest
to override. This is simplified by the fact that HCRX_EL2 only
affects EL1, and not EL2.

Re-jig the HCRX_EL2 handling with a macro that list the bits excluded
from the merge (TMEA, PTTWI, EnIDCP128).

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/kvm/hyp/include/hyp/switch.h | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index 4bf624a49591d..8e5f492f39086 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -325,6 +325,24 @@ static inline void __deactivate_traps_mpam(void)
 		write_sysreg_s(MPAMHCR_HOST_FLAGS, SYS_MPAMHCR_EL2);
 }
 
+/*
+ * Just like for HCR_EL2, we can't let the guest mess with some of the
+ * basics we rely on in HCRX_EL2. However, the major difference is that
+ * HCRX_EL2 only affects EL1, and never EL2 (sudden outburst of sanity, I
+ * guess). So it is always the guest inflicting it on its own guestx.
+ *
+ * Things we don't want to let the guest control are:
+ *
+ * - TMEA: That's for us to decide how an SEA is routed, not the guest.
+ *
+ * - PTTWI: Similarly, it is for us to decide whether Reduced Coherency for
+ *   the PTW is a thing. It really isn't.
+ *
+ * - EnIDCP128: We don't allow IMPDEF sysregs -- full stop.
+ */
+#define NV_HCRX_GUEST_EXCLUDE	(HCRX_EL2_TMEA	    | HCRX_EL2_PTTWI | \
+				 HCRX_EL2_EnIDCP128)
+
 static inline void __activate_traps_common(struct kvm_vcpu *vcpu)
 {
 	struct kvm_cpu_context *hctxt = host_data_ptr(host_ctxt);
@@ -350,8 +368,8 @@ static inline void __activate_traps_common(struct kvm_vcpu *vcpu)
 		u64 hcrx = vcpu->arch.hcrx_el2;
 		if (is_nested_ctxt(vcpu)) {
 			u64 val = __vcpu_sys_reg(vcpu, HCRX_EL2);
-			hcrx |= val & __HCRX_EL2_MASK;
-			hcrx &= ~(~val & __HCRX_EL2_nMASK);
+			hcrx |= (val & ~NV_HCRX_GUEST_EXCLUDE);
+			hcrx &= ~(~val & ~NV_HCRX_GUEST_EXCLUDE);
 		}
 
 		ctxt_sys_reg(hctxt, HCRX_EL2) = read_sysreg_s(SYS_HCRX_EL2);
-- 
2.47.3

[PATCH 04/28] KVM: arm64: Drop __HCRX_EL2_* masks

[Marc Zyngier]

The __HCRX_EL2_* masks are a leftover from a time where we didn't
have much sanitisation for the system registers. Since we are now
in a better place, rely on the existing checks to detect unhandled
bits in HCRX_EL2.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/include/asm/kvm_arm.h | 15 ---------------
 arch/arm64/kvm/config.c          |  3 +--
 arch/arm64/kvm/emulate-nested.c  |  5 -----
 3 files changed, 1 insertion(+), 22 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 3f9233b5a1308..f6cd851047947 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -287,21 +287,6 @@
 				 GENMASK(19, 18) |	\
 				 GENMASK(15, 0))
 
-/*
- * Polarity masks for HCRX_EL2, limited to the bits that we know about
- * at this point in time. It doesn't mean that we actually *handle*
- * them, but that at least those that are not advertised to a guest
- * will be RES0 for that guest.
- */
-#define __HCRX_EL2_MASK		(BIT_ULL(6))
-#define __HCRX_EL2_nMASK	(GENMASK_ULL(24, 14) | \
-				 GENMASK_ULL(11, 7)  | \
-				 GENMASK_ULL(5, 0))
-#define __HCRX_EL2_RES0		~(__HCRX_EL2_nMASK | __HCRX_EL2_MASK)
-#define __HCRX_EL2_RES1		~(__HCRX_EL2_nMASK | \
-				  __HCRX_EL2_MASK  | \
-				  __HCRX_EL2_RES0)
-
 /* Hyp Prefetch Fault Address Register (HPFAR/HDFAR) */
 #define HPFAR_MASK	(~UL(0xf))
 /*
diff --git a/arch/arm64/kvm/config.c b/arch/arm64/kvm/config.c
index 0622162b089e5..16d8148dc3f12 100644
--- a/arch/arm64/kvm/config.c
+++ b/arch/arm64/kvm/config.c
@@ -933,7 +933,7 @@ static const struct reg_bits_to_feat_map hcrx_feat_map[] = {
 };
 
 
-static const DECLARE_FEAT_MAP(hcrx_desc, __HCRX_EL2,
+static const DECLARE_FEAT_MAP(hcrx_desc, HCRX_EL2,
 			      hcrx_feat_map, FEAT_HCX);
 
 static const struct reg_bits_to_feat_map hcr_feat_map[] = {
@@ -1579,7 +1579,6 @@ struct resx get_reg_fixed_bits(struct kvm *kvm, enum vcpu_sysreg reg)
 		break;
 	case HCRX_EL2:
 		resx = compute_reg_resx_bits(kvm, &hcrx_desc, 0, 0);
-		resx.res1 |= __HCRX_EL2_RES1;
 		break;
 	case HCR_EL2:
 		resx = compute_reg_resx_bits(kvm, &hcr_desc, 0, 0);
diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c
index 3c82f392845d1..b7f3d86a94031 100644
--- a/arch/arm64/kvm/emulate-nested.c
+++ b/arch/arm64/kvm/emulate-nested.c
@@ -2320,7 +2320,6 @@ int __init populate_nv_trap_config(void)
 	BUILD_BUG_ON(__NR_CGT_GROUP_IDS__ > BIT(TC_CGT_BITS));
 	BUILD_BUG_ON(__NR_FGT_GROUP_IDS__ > BIT(TC_FGT_BITS));
 	BUILD_BUG_ON(__NR_FG_FILTER_IDS__ > BIT(TC_FGF_BITS));
-	BUILD_BUG_ON(__HCRX_EL2_MASK & __HCRX_EL2_nMASK);
 
 	for (int i = 0; i < ARRAY_SIZE(encoding_to_cgt); i++) {
 		const struct encoding_to_trap_config *cgt = &encoding_to_cgt[i];
@@ -2346,10 +2345,6 @@ int __init populate_nv_trap_config(void)
 		}
 	}
 
-	if (__HCRX_EL2_RES0 != HCRX_EL2_RES0)
-		kvm_info("Sanitised HCR_EL2_RES0 = %016llx, expecting %016llx\n",
-			 __HCRX_EL2_RES0, HCRX_EL2_RES0);
-
 	kvm_info("nv: %ld coarse grained trap handlers\n",
 		 ARRAY_SIZE(encoding_to_cgt));
 
-- 
2.47.3

[PATCH 05/28] KVM: arm64: Plumb HCRX_EL2.SRMASKEn in HCRX_EL2 sanitisation

[Marc Zyngier]

HCRX_EL2.SRMASKEn is a new bit enabling FEAT_SRMASK for a guest.
We don't plan to support it any time soon, but it doesn't hurt to
actively document it, specially as we are going to add more bits
we actually care about.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/kvm/config.c | 1 +
 arch/arm64/tools/sysreg | 4 +++-
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/kvm/config.c b/arch/arm64/kvm/config.c
index 16d8148dc3f12..8d5e4aacf49c4 100644
--- a/arch/arm64/kvm/config.c
+++ b/arch/arm64/kvm/config.c
@@ -904,6 +904,7 @@ static const DECLARE_FEAT_MAP_FGT(hdfgwtr2_desc, hdfgwtr2_masks,
 
 
 static const struct reg_bits_to_feat_map hcrx_feat_map[] = {
+	NEEDS_FEAT(HCRX_EL2_SRMASKEn, FEAT_SRMASK),
 	NEEDS_FEAT(HCRX_EL2_PACMEn, feat_pauth_lr),
 	NEEDS_FEAT(HCRX_EL2_EnFPM, FEAT_FPMR),
 	NEEDS_FEAT(HCRX_EL2_GCSEn, FEAT_GCS),
diff --git a/arch/arm64/tools/sysreg b/arch/arm64/tools/sysreg
index 32e2f9856768b..c6e8117a6f9cd 100644
--- a/arch/arm64/tools/sysreg
+++ b/arch/arm64/tools/sysreg
@@ -4545,7 +4545,9 @@ Fields	ZCR_ELx
 EndSysreg
 
 Sysreg	HCRX_EL2	3	4	1	2	2
-Res0	63:25
+Res0	63:27
+Field	26	SRMASKEn
+Res0	25
 Field	24	PACMEn
 Field	23	EnFPM
 Field	22	GCSEn
-- 
2.47.3

[PATCH 06/28] KVM: arm64: Classify CPTR_EL2 as a SR_LOC_SPECIAL register

[Marc Zyngier]

It may not be obvious unless you look at it closely, but CPTR_EL2
is treated very differently from other registers. It is one the
registers that, despite looking very similar between EL1 and EL2
when E2H==1, have RES0 bits that get in the way.

Make it clear that CPTR_EL2 is odd by classifying it as SR_LOC_SPECIAL,
just like CNTHCTL_EL2 (and for the same reasons). This makes it
possible to use vcpu_read_sys_reg() with it, and will be necessary
once we support FEAT_NV2P1.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/include/asm/kvm_emulate.h |  2 +-
 arch/arm64/kvm/sys_regs.c            | 20 ++++++++++++++++++--
 2 files changed, 19 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index 5bf3d7e1d92c7..9831166695186 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -617,7 +617,7 @@ static __always_inline void kvm_incr_pc(struct kvm_vcpu *vcpu)
  */
 static inline u64 vcpu_sanitised_cptr_el2(const struct kvm_vcpu *vcpu)
 {
-	u64 cptr = __vcpu_sys_reg(vcpu, CPTR_EL2);
+	u64 cptr = vcpu_read_sys_reg(vcpu, CPTR_EL2);
 
 	if (!vcpu_el2_e2h_is_set(vcpu))
 		cptr = translate_cptr_el2_to_cpacr_el1(cptr);
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 5d5c579d45790..6b47d936efb32 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -183,8 +183,6 @@ static void locate_register(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg,
 	switch (reg) {
 		MAPPED_EL2_SYSREG(SCTLR_EL2,   SCTLR_EL1,
 				  translate_sctlr_el2_to_sctlr_el1	     );
-		MAPPED_EL2_SYSREG(CPTR_EL2,    CPACR_EL1,
-				  translate_cptr_el2_to_cpacr_el1	     );
 		MAPPED_EL2_SYSREG(TTBR0_EL2,   TTBR0_EL1,
 				  translate_ttbr0_el2_to_ttbr0_el1	     );
 		MAPPED_EL2_SYSREG(TTBR1_EL2,   TTBR1_EL1,   NULL	     );
@@ -210,6 +208,19 @@ static void locate_register(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg,
 		loc->loc = ((is_hyp_ctxt(vcpu) && vcpu_el2_e2h_is_set(vcpu)) ?
 			    SR_LOC_SPECIAL : SR_LOC_MEMORY);
 		break;
+	case CPTR_EL2:
+		/*
+		 * CPTR_EL2 is just as special, and needs a certain amount
+		 * of handholding. It always lives in memory, due to being
+		 * heavily trapped thanks to CPACR_EL1.TCPAC being RES0.
+		 * FEAT_NV2p1 fixes this.
+		 */
+		locate_mapped_el2_register(vcpu, CPTR_EL2, CPACR_EL1,
+					   translate_cptr_el2_to_cpacr_el1,
+					   loc);
+		if (is_hyp_ctxt(vcpu) && vcpu_el2_e2h_is_set(vcpu))
+			loc->loc = SR_LOC_SPECIAL;
+		break;
 	default:
 		loc->loc = locate_direct_register(vcpu, reg);
 	}
@@ -314,6 +325,8 @@ u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg)
 			val &= CNTKCTL_VALID_BITS;
 			val |= __vcpu_sys_reg(vcpu, reg) & ~CNTKCTL_VALID_BITS;
 			return val;
+		case CPTR_EL2:
+			return __vcpu_sys_reg(vcpu, reg);
 		default:
 			WARN_ON_ONCE(1);
 		}
@@ -359,6 +372,9 @@ void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, enum vcpu_sysreg reg)
 			 */
 			write_sysreg_el1(val, SYS_CNTKCTL);
 			break;
+		case CPTR_EL2:
+			write_sysreg_el1(val, SYS_CPACR);
+			break;
 		default:
 			WARN_ON_ONCE(1);
 		}
-- 
2.47.3

[PATCH 07/28] KVM: arm64: Don't evaluate HCR_EL2.NV on ERET fast path

[Marc Zyngier]

We currently avoid using the ERET fast path if the guest has HCR_EL2.NV
set. This is an odd check, as  NV doesn't mean much if HCR_EL2.TGE==1.

Replace this bizarre check with is_nested_ctxt() which makes a lot
more sense: if we are running an L2, the ERET trap must go to L1.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/kvm/hyp/vhe/switch.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index bbe9cebd3d9d5..3b76e0468317b 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -349,8 +349,8 @@ static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code)
 	 * Unless the trap has to be forwarded further down the line,
 	 * of course...
 	 */
-	if ((__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_NV) ||
-	    (__vcpu_sys_reg(vcpu, HFGITR_EL2) & HFGITR_EL2_ERET))
+	if (is_nested_ctxt(vcpu) ||
+	    __vcpu_sys_reg(vcpu, HFGITR_EL2) & HFGITR_EL2_ERET)
 		return false;
 
 	spsr = read_sysreg_el1(SYS_SPSR);
-- 
2.47.3

[PATCH 08/28] arm64: Add ARM64_HAS_NV2P1 capability

[Marc Zyngier]

As we're about to deal with FEAT_NV2P1, add a new capability that
will be used to key any support for it.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/tools/cpucaps | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps
index 9b85a84f6fd49..242dc211d8efa 100644
--- a/arch/arm64/tools/cpucaps
+++ b/arch/arm64/tools/cpucaps
@@ -51,6 +51,7 @@ HAS_LS64_V
 HAS_LSUI
 HAS_MOPS
 HAS_NESTED_VIRT
+HAS_NV2P1
 HAS_BBML2_NOABORT
 HAS_PAN
 HAS_PMUV3
-- 
2.47.3

[PATCH 09/28] KVM: arm64: Relax CPTR_EL2 handling when FEAT_NV2p1 is present

[Marc Zyngier]

With FEAT_NV2P1, it is no longer necessary to trap CPTR_EL2 accesses
via CPACR_EL1, as CPACR_EL1.TCPAC is guaranteed to be stateful.

Prevent such trapping and context switch CPACTR_EL1 in NV contexts
when NV2P1 is present.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/kvm/hyp/include/hyp/switch.h | 5 +++--
 arch/arm64/kvm/hyp/vhe/switch.c         | 3 +++
 arch/arm64/kvm/hyp/vhe/sysreg-sr.c      | 8 +++++---
 arch/arm64/kvm/sys_regs.c               | 5 ++++-
 4 files changed, 15 insertions(+), 6 deletions(-)

diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index 8e5f492f39086..7b27296c94607 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -108,9 +108,10 @@ static inline void __activate_cptr_traps_vhe(struct kvm_vcpu *vcpu)
 	 * The architecture is a bit crap (what a surprise): an EL2 guest
 	 * writing to CPTR_EL2 via CPACR_EL1 can't set any of TCPAC or TTA,
 	 * as they are RES0 in the guest's view. To work around it, trap the
-	 * sucker using the very same bit it can't set...
+	 * sucker using the very same bit it can't set. FEAT_NV2p1 fixes it.
 	 */
-	if (vcpu_el2_e2h_is_set(vcpu) && is_hyp_ctxt(vcpu))
+	if (!cpus_have_final_cap(ARM64_HAS_NV2P1) &&
+	    vcpu_el2_e2h_is_set(vcpu) && is_hyp_ctxt(vcpu))
 		val |= CPTR_EL2_TCPAC;
 
 	/*
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 3b76e0468317b..361d3f8344dd8 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -441,6 +441,9 @@ static bool kvm_hyp_handle_cpacr_el1(struct kvm_vcpu *vcpu, u64 *exit_code)
 	u64 esr = kvm_vcpu_get_esr(vcpu);
 	int rt;
 
+	if (cpus_have_final_cap(ARM64_HAS_NV2P1))
+		return false;
+
 	if (!is_hyp_ctxt(vcpu) || esr_sys64_to_sysreg(esr) != SYS_CPACR_EL1)
 		return false;
 
diff --git a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c
index be685b63e8cf2..6f0f046e4ca4e 100644
--- a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c
@@ -42,10 +42,12 @@ static void __sysreg_save_vel2_state(struct kvm_vcpu *vcpu)
 		u64 val;
 
 		/*
-		 * We don't save CPTR_EL2, as accesses to CPACR_EL1
-		 * are always trapped, ensuring that the in-memory
-		 * copy is always up-to-date. A small blessing...
+		 * Without FEAT_NV2p1, we don't save CPTR_EL2, as accesses
+		 * to CPACR_EL1 are always trapped, ensuring that the
+		 * in-memory copy is always up-to-date. A small blessing...
 		 */
+		if (cpus_have_final_cap(ARM64_HAS_NV2P1))
+			__vcpu_assign_sys_reg(vcpu, CPTR_EL2, read_sysreg_el1(SYS_CPACR));
 		__vcpu_assign_sys_reg(vcpu, SCTLR_EL2,	 read_sysreg_el1(SYS_SCTLR));
 		__vcpu_assign_sys_reg(vcpu, TTBR0_EL2,	 read_sysreg_el1(SYS_TTBR0));
 		__vcpu_assign_sys_reg(vcpu, TTBR1_EL2,	 read_sysreg_el1(SYS_TTBR1));
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 6b47d936efb32..1dfc1f88bec82 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -326,7 +326,10 @@ u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg)
 			val |= __vcpu_sys_reg(vcpu, reg) & ~CNTKCTL_VALID_BITS;
 			return val;
 		case CPTR_EL2:
-			return __vcpu_sys_reg(vcpu, reg);
+			if (cpus_have_final_cap(ARM64_HAS_NV2P1))
+				return read_sysreg_el1(SYS_CPACR);
+			else
+				return __vcpu_sys_reg(vcpu, reg);
 		default:
 			WARN_ON_ONCE(1);
 		}
-- 
2.47.3

[PATCH 10/28] KVM: arm64: Relax CNTHCTL_EL2 handling when FEAT_NV2p1 is present

[Marc Zyngier]

With NV2p1, it is no longer necessary to use the split approach
where bits of CNTHCTL_EL2 cannot be accessed via CNTKCTL_EL1,
and we can treat the CNTKCTL_EL1 accessor as if it was "normal".

Key the special casing on FEAT_NV2P1 not being implemented.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/kvm/arch_timer.c        | 10 ++++++++--
 arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 13 ++++++++++---
 arch/arm64/kvm/sys_regs.c          |  6 ++++--
 3 files changed, 22 insertions(+), 7 deletions(-)

diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
index 4155fe89b58a1..db60facad9f3c 100644
--- a/arch/arm64/kvm/arch_timer.c
+++ b/arch/arm64/kvm/arch_timer.c
@@ -876,8 +876,14 @@ static void timer_set_traps(struct kvm_vcpu *vcpu, struct timer_map *map)
 	assign_clear_set_bit(tvt02, CNTHCTL_EL1NVVCT, clr, set);
 	assign_clear_set_bit(tpt02, CNTHCTL_EL1NVPCT, clr, set);
 
-	/* This only happens on VHE, so use the CNTHCTL_EL2 accessor. */
-	sysreg_clear_set(cnthctl_el2, clr, set);
+	/*
+	 * This only happens on VHE, so use the CNTHCTL_EL2 accessor, unless
+	 * we are sure CNTKCTL_EL1 is completely stateful with FEAT_NV2p1.
+	 */
+	if (!cpus_have_final_cap(ARM64_HAS_NV2P1))
+		sysreg_clear_set(cnthctl_el2, clr, set);
+	else
+		sysreg_clear_set(cntkctl_el1, clr, set);
 }
 
 void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu)
diff --git a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c
index 6f0f046e4ca4e..0c4ef1ce32ae7 100644
--- a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c
@@ -69,11 +69,18 @@ static void __sysreg_save_vel2_state(struct kvm_vcpu *vcpu)
 		 * The EL1 view of CNTKCTL_EL1 has a bunch of RES0 bits where
 		 * the interesting CNTHCTL_EL2 bits live. So preserve these
 		 * bits when reading back the guest-visible value.
+		 *
+		 * While NV2p1 fixes some of that, it makes CNTHCTL_EL2.ECV
+		 * even more broken than it already was with NV2.
 		 */
 		val = read_sysreg_el1(SYS_CNTKCTL);
-		val &= CNTKCTL_VALID_BITS;
-		__vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, &=, ~CNTKCTL_VALID_BITS);
-		__vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, |=, val);
+		if (!cpus_have_final_cap(ARM64_HAS_NV2P1)) {
+			val &= CNTKCTL_VALID_BITS;
+			__vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, &=, ~CNTKCTL_VALID_BITS);
+			__vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, |=, val);
+		} else {
+			__vcpu_assign_sys_reg(vcpu, CNTHCTL_EL2, val);
+		}
 	}
 
 	__vcpu_assign_sys_reg(vcpu, SP_EL2,	 read_sysreg(sp_el1));
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 1dfc1f88bec82..9439c5b2b1fe8 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -322,8 +322,10 @@ u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg)
 		switch (reg) {
 		case CNTHCTL_EL2:
 			val = read_sysreg_el1(SYS_CNTKCTL);
-			val &= CNTKCTL_VALID_BITS;
-			val |= __vcpu_sys_reg(vcpu, reg) & ~CNTKCTL_VALID_BITS;
+			if (!cpus_have_final_cap(ARM64_HAS_NV2P1)) {
+				val &= CNTKCTL_VALID_BITS;
+				val |= __vcpu_sys_reg(vcpu, reg) & ~CNTKCTL_VALID_BITS;
+			}
 			return val;
 		case CPTR_EL2:
 			if (cpus_have_final_cap(ARM64_HAS_NV2P1))
-- 
2.47.3

[PATCH 11/28] KVM: arm64: Expose FEAT_NV2p1 to NV guests

[Marc Zyngier]

Since NV2p1 is reducing the number of traps, it is valuable to expose
it to NV guests. Do so.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/kvm/nested.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index dfb96edbdc43c..9972dea42d12a 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -1728,7 +1728,7 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)
 		 * You get EITHER
 		 *
 		 * - FEAT_VHE without FEAT_E2H0
-		 * - FEAT_NV limited to FEAT_NV2
+		 * - FEAT_NV limited to FEAT_NV2(p1)
 		 * - HCR_EL2.NV1 being RES0
 		 *
 		 * OR
@@ -1740,7 +1740,11 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)
 		if (test_bit(KVM_ARM_VCPU_HAS_EL2_E2H0, kvm->arch.vcpu_features)) {
 			val = 0;
 		} else {
-			val = SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY);
+			if (cpus_have_final_cap(ARM64_HAS_NV2P1))
+				val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64MMFR4_EL1, NV_frac, NV2P1);
+			else
+				val = SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY);
+			val &= ~ID_AA64MMFR4_EL1_E2H0;
 			val |= SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, E2H0, NI_NV1);
 		}
 		break;
-- 
2.47.3

[PATCH 12/28] arm64: Add FEAT_NV2p1 detection

[Marc Zyngier]

Add the necessary NV2p1 probing to the cpufeature infrastructure.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/kernel/cpufeature.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 9a22df0c5120f..c9c124b0ccc8e 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -2620,6 +2620,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
 			{ /* Sentinel */ }
 		},
 	},
+	{
+		.desc = "FEAT_NV2p1",
+		.capability = ARM64_HAS_NV2P1,
+		.type = ARM64_CPUCAP_SYSTEM_FEATURE,
+		.matches = has_cpuid_feature,
+		ARM64_CPUID_FIELDS(ID_AA64MMFR4_EL1, NV_frac, NV2P1)
+	},
 	{
 		.capability = ARM64_HAS_32BIT_EL0_DO_NOT_USE,
 		.type = ARM64_CPUCAP_SYSTEM_FEATURE,
-- 
2.47.3

[PATCH 13/28] arm64: sysreg: Add NVHCR_EL2 description as a mirror of HCR_EL2

[Marc Zyngier]

FEAT_NV3 introduces a new register that contains the HCR_EL2 value
exposed to a NV guest. As such, it has the exact same layout as
HCR_EL2.

Describe NVHCR_EL2 as a mapping to HCR_EL2.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/tools/sysreg | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/arch/arm64/tools/sysreg b/arch/arm64/tools/sysreg
index c6e8117a6f9cd..31e4ea455a9ce 100644
--- a/arch/arm64/tools/sysreg
+++ b/arch/arm64/tools/sysreg
@@ -4266,6 +4266,9 @@ Field	1	E2TRE
 Field	0	E0HTRE
 EndSysreg
 
+Sysreg	NVHCR_EL2	3	4	1	5	0
+Mapping	HCR_EL2
+EndSysreg
 
 Sysreg HDFGRTR2_EL2	3	4	3	1	0
 Res0	63:25
-- 
2.47.3

[PATCH 14/28] arm64: sysreg: Add HCRX_EL2 bits related to FEAT_NV3

[Marc Zyngier]

FEAT_NV3 introduces 4 new HCRX_EL2 control bits. Describe them
in the sysreg file.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/tools/sysreg | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/tools/sysreg b/arch/arm64/tools/sysreg
index 31e4ea455a9ce..afe9337851a2e 100644
--- a/arch/arm64/tools/sysreg
+++ b/arch/arm64/tools/sysreg
@@ -4548,7 +4548,12 @@ Fields	ZCR_ELx
 EndSysreg
 
 Sysreg	HCRX_EL2	3	4	1	2	2
-Res0	63:27
+Res0	63:35
+Field	34	NVnTTLBOS
+Field	33	NVnTTLBIS
+Field	32	NVnTTLB
+Res0	31:28
+Field	27	NVTGE
 Field	26	SRMASKEn
 Res0	25
 Field	24	PACMEn
-- 
2.47.3

[PATCH 15/28] arm64: Add ARM64_HAS_NV3 capability

[Marc Zyngier]

As a bunch of KVM code is going to depend on FEAT_NV3 being detected
on the host, add a new capability that will describe it.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/tools/cpucaps | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps
index 242dc211d8efa..7e0414509e89a 100644
--- a/arch/arm64/tools/cpucaps
+++ b/arch/arm64/tools/cpucaps
@@ -52,6 +52,7 @@ HAS_LSUI
 HAS_MOPS
 HAS_NESTED_VIRT
 HAS_NV2P1
+HAS_NV3
 HAS_BBML2_NOABORT
 HAS_PAN
 HAS_PMUV3
-- 
2.47.3

[PATCH 16/28] KVM: arm64: Split NV-specific exit fixups from the non-NV handling

[Marc Zyngier]

In order to facilitate further changes, move the NV handling of
early fixups in its own helper. This also makes the code slightly
simpler to parse.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/kvm/hyp/vhe/switch.c | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 361d3f8344dd8..8268779df4fa9 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -537,18 +537,15 @@ static const exit_handler_fn hyp_exit_handlers[] = {
 	[0x3F]				= kvm_hyp_handle_impdef,
 };
 
-static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
+static void fixup_nv_guest_exit(struct kvm_vcpu *vcpu)
 {
-	synchronize_vcpu_pstate(vcpu);
-
 	/*
 	 * If we were in HYP context on entry, adjust the PSTATE view
 	 * so that the usual helpers work correctly. This enforces our
 	 * invariant that the guest's HYP context status is preserved
 	 * across a run.
 	 */
-	if (vcpu_has_nv(vcpu) &&
-	    unlikely(host_data_test_flag(VCPU_IN_HYP_CONTEXT))) {
+	if (unlikely(host_data_test_flag(VCPU_IN_HYP_CONTEXT))) {
 		u64 mode = *vcpu_cpsr(vcpu) & (PSR_MODE_MASK | PSR_MODE32_BIT);
 
 		switch (mode) {
@@ -565,8 +562,15 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
 	}
 
 	/* Apply extreme paranoia! */
-	BUG_ON(vcpu_has_nv(vcpu) &&
-	       !!host_data_test_flag(VCPU_IN_HYP_CONTEXT) != is_hyp_ctxt(vcpu));
+	BUG_ON(!!host_data_test_flag(VCPU_IN_HYP_CONTEXT) != is_hyp_ctxt(vcpu));
+}
+
+static bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
+{
+	synchronize_vcpu_pstate(vcpu);
+
+	if (vcpu_has_nv(vcpu))
+		fixup_nv_guest_exit(vcpu);
 
 	return __fixup_guest_exit(vcpu, exit_code, hyp_exit_handlers);
 }
-- 
2.47.3

[PATCH 17/28] KVM: arm64: Add NV3 control bits to HCRX_EL2 sanitisation

[Marc Zyngier]

Expose the FEAT_NV3 control bits to the sanitisation code so that
KVM stops moaning about the unattributed bits.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/kvm/config.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/arch/arm64/kvm/config.c b/arch/arm64/kvm/config.c
index 8d5e4aacf49c4..b9a9d65b973e6 100644
--- a/arch/arm64/kvm/config.c
+++ b/arch/arm64/kvm/config.c
@@ -225,6 +225,7 @@ struct reg_feat_map_desc {
 #define FEAT_HCX		ID_AA64MMFR1_EL1, HCX, IMP
 #define FEAT_S2PIE		ID_AA64MMFR3_EL1, S2PIE, IMP
 #define FEAT_GCIE		ID_AA64PFR2_EL1, GCIE, IMP
+#define FEAT_NV3		ID_AA64MMFR4_EL1, NV_frac, NV3
 
 static bool not_feat_aa64el3(struct kvm *kvm)
 {
@@ -904,6 +905,11 @@ static const DECLARE_FEAT_MAP_FGT(hdfgwtr2_desc, hdfgwtr2_masks,
 
 
 static const struct reg_bits_to_feat_map hcrx_feat_map[] = {
+	NEEDS_FEAT(HCRX_EL2_NVTGE		|
+		   HCRX_EL2_NVnTTLB		|
+		   HCRX_EL2_NVnTTLBIS		|
+		   HCRX_EL2_NVnTTLBOS,
+		   FEAT_NV3),
 	NEEDS_FEAT(HCRX_EL2_SRMASKEn, FEAT_SRMASK),
 	NEEDS_FEAT(HCRX_EL2_PACMEn, feat_pauth_lr),
 	NEEDS_FEAT(HCRX_EL2_EnFPM, FEAT_FPMR),
-- 
2.47.3

[PATCH 18/28] KVM: arm64: Add kvm_has_nv{2,3}() predicates

[Marc Zyngier]

Add a new set of predicates indicating whether VM is capable of
NV2, NV3, and is in a nested NV3 context.

This is going to become useful as we start dealing with a mix of
behaviours (NV2, NV3, NV2 on NV3...).

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/include/asm/kvm_emulate.h | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index 9831166695186..c562d8171d5e1 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -266,6 +266,26 @@ static inline bool vserror_state_is_nested(struct kvm_vcpu *vcpu)
 	       (__vcpu_sys_reg(vcpu, HCRX_EL2) & HCRX_EL2_TMEA);
 }
 
+static inline bool kvm_has_nv2(struct kvm *kvm)
+{
+	return (cpus_have_final_cap(ARM64_HAS_NESTED_VIRT) &&
+		kvm_has_feat(kvm, ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY));
+}
+
+static inline bool kvm_has_nv3(struct kvm *kvm)
+{
+	return (cpus_have_final_cap(ARM64_HAS_NESTED_VIRT) &&
+		cpus_have_final_cap(ARM64_HAS_NV3) &&
+		kvm_has_feat(kvm, ID_AA64MMFR4_EL1, NV_frac, NV3));
+}
+
+static inline bool is_nested_nv3_ctxt(struct kvm_vcpu *vcpu)
+{
+	return (has_vhe() && kvm_has_nv3(vcpu->kvm) && is_nested_ctxt(vcpu) &&
+		(__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_EL2_NV) &&
+		(__vcpu_sys_reg(vcpu, HCRX_EL2) & HCRX_EL2_NVTGE));
+}
+
 /*
  * The layout of SPSR for an AArch32 state is different when observed from an
  * AArch64 SPSR_ELx or an AArch32 SPSR_*. This function generates the AArch32
-- 
2.47.3

[PATCH 19/28] KVM: arm64: Make HCR_EL2 a non-VNCR register

[Marc Zyngier]

FEAT_NV3 makes a fundamental change to the architecture, by moving
guest-initiated HCR_EL2 accesses to the NVHCR_EL2 register. As the
names suggests, this is HCR_EL2 for a NV guest.

But where do NVHCR_EL2 accesses from a guest go? The are redirected
to the VNCR page, right where HCR_EL2 is stored in the NV2 case.
Does it hurt? Good. There's more coming.

The challenge here is to make KVM work seamlessly, without rewriting
everything. Which implies that things such as __vcpu_sys_reg(HCR_EL2)
must work, no matter the underlying NV implementation.

A simple way to deal with it is to move HCR_EL2's canonical storage
outside of VNCR for the vast majority of the KVM code, and only have
a copy at entry/exit times. Given that we don't really support NV3
yet, this is pretty simple.

In the process, advertise NVHCR_EL2 as the register that now holds
offset 0x78 in the VNCR page.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/include/asm/kvm_host.h     | 3 ++-
 arch/arm64/include/asm/vncr_mapping.h | 2 +-
 arch/arm64/kvm/hyp/vhe/switch.c       | 9 +++++++++
 3 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index bae2c4f92ef5c..2648c8a717ba0 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -543,6 +543,7 @@ enum vcpu_sysreg {
 	MDCR_EL2,	/* Monitor Debug Configuration Register (EL2) */
 	CNTHCTL_EL2,	/* Counter-timer Hypervisor Control register */
 	ZCR_EL2,	/* SVE Control Register (EL2) */
+	HCR_EL2,	/* Hypervisor Control Register */
 
 	/* Any VNCR-capable reg goes after this point */
 	MARKER(__VNCR_START__),
@@ -571,7 +572,7 @@ enum vcpu_sysreg {
 	VNCR(TFSR_EL1),	/* Tag Fault Status Register (EL1) */
 	VNCR(VPIDR_EL2),/* Virtualization Processor ID Register */
 	VNCR(VMPIDR_EL2),/* Virtualization Multiprocessor ID Register */
-	VNCR(HCR_EL2),	/* Hypervisor Configuration Register */
+	VNCR(NVHCR_EL2),/* NV Hypervisor Configuration Register */
 	VNCR(HSTR_EL2),	/* Hypervisor System Trap Register */
 	VNCR(VTTBR_EL2),/* Virtualization Translation Table Base Register */
 	VNCR(VTCR_EL2),	/* Virtualization Translation Control Register */
diff --git a/arch/arm64/include/asm/vncr_mapping.h b/arch/arm64/include/asm/vncr_mapping.h
index 14366d35ce82f..9e8a49fa8b638 100644
--- a/arch/arm64/include/asm/vncr_mapping.h
+++ b/arch/arm64/include/asm/vncr_mapping.h
@@ -11,7 +11,7 @@
 #define VNCR_VTCR_EL2           0x040
 #define VNCR_VMPIDR_EL2         0x050
 #define VNCR_CNTVOFF_EL2        0x060
-#define VNCR_HCR_EL2            0x078
+#define VNCR_NVHCR_EL2          0x078
 #define VNCR_HSTR_EL2           0x080
 #define VNCR_VPIDR_EL2          0x088
 #define VNCR_TPIDR_EL2          0x090
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 8268779df4fa9..05bcf8bf7f978 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -70,6 +70,9 @@ static u64 __compute_hcr(struct kvm_vcpu *vcpu)
 		if (!vcpu_el2_e2h_is_set(vcpu))
 			hcr |= HCR_NV1;
 
+		/* Publish the guest's view of HCR_EL2 to the HW */
+		__vcpu_assign_sys_reg(vcpu, NVHCR_EL2, __vcpu_sys_reg(vcpu, HCR_EL2));
+
 		/*
 		 * Nothing in HCR_EL2 should impact running in hypervisor
 		 * context, apart from bits we have defined as RESx (E2H,
@@ -547,6 +550,7 @@ static void fixup_nv_guest_exit(struct kvm_vcpu *vcpu)
 	 */
 	if (unlikely(host_data_test_flag(VCPU_IN_HYP_CONTEXT))) {
 		u64 mode = *vcpu_cpsr(vcpu) & (PSR_MODE_MASK | PSR_MODE32_BIT);
+		u64 hcr;
 
 		switch (mode) {
 		case PSR_MODE_EL1t:
@@ -559,6 +563,11 @@ static void fixup_nv_guest_exit(struct kvm_vcpu *vcpu)
 
 		*vcpu_cpsr(vcpu) &= ~(PSR_MODE_MASK | PSR_MODE32_BIT);
 		*vcpu_cpsr(vcpu) |= mode;
+
+		/* Publish the latest HCR_EL2 to the emulation */
+		hcr = __vcpu_sys_reg(vcpu, NVHCR_EL2);
+
+		__vcpu_assign_sys_reg(vcpu, HCR_EL2, hcr);
 	}
 
 	/* Apply extreme paranoia! */
-- 
2.47.3

[PATCH 20/28] KVM: arm64: Add sanitisation for NVHCR_EL2

[Marc Zyngier]

Just like any other VNCR-based register, NVHCR_EL2 requires some
level of sanitisation. Being specified as a live copy of HCR_EL2,
it adopts the exact same format, but depends on FEAT_NV3 instead.

A subtle aspect is that we only want to apply the sanitisation if
FEAT_NV3 is actually present, as the VNCR location is otherwise
used to back accesses to HCR_EL2.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/kvm/config.c | 15 +++++++++++++++
 arch/arm64/kvm/nested.c |  4 ++++
 2 files changed, 19 insertions(+)

diff --git a/arch/arm64/kvm/config.c b/arch/arm64/kvm/config.c
index b9a9d65b973e6..7e86479142723 100644
--- a/arch/arm64/kvm/config.c
+++ b/arch/arm64/kvm/config.c
@@ -1017,6 +1017,9 @@ static const struct reg_bits_to_feat_map hcr_feat_map[] = {
 static const DECLARE_FEAT_MAP(hcr_desc, HCR_EL2,
 			      hcr_feat_map, FEAT_AA64EL2);
 
+static const DECLARE_FEAT_MAP(nvhcr_desc, NVHCR_EL2,
+			      hcr_feat_map, FEAT_NV3);
+
 static const struct reg_bits_to_feat_map sctlr2_feat_map[] = {
 	NEEDS_FEAT(SCTLR2_EL1_NMEA	|
 		   SCTLR2_EL1_EASE,
@@ -1391,6 +1394,7 @@ void __init check_feature_map(void)
 	check_reg_desc(&hdfgwtr2_desc);
 	check_reg_desc(&hcrx_desc);
 	check_reg_desc(&hcr_desc);
+	check_reg_desc(&nvhcr_desc);
 	check_reg_desc(&sctlr2_desc);
 	check_reg_desc(&tcr2_el2_desc);
 	check_reg_desc(&sctlr_el1_desc);
@@ -1590,6 +1594,17 @@ struct resx get_reg_fixed_bits(struct kvm *kvm, enum vcpu_sysreg reg)
 	case HCR_EL2:
 		resx = compute_reg_resx_bits(kvm, &hcr_desc, 0, 0);
 		break;
+	case NVHCR_EL2:
+		/*
+		 * Only apply sanitisation if we do have FEAT_NV3.
+		 * Otherwise, the register aliases with HCR_EL2 in VNCR,
+		 * and we're better off relying on data transfers between
+		 * NVHCR_EL2 and HCR_EL2 to sanitise things.
+		 */
+		resx = (kvm_has_nv3(kvm) ?
+			compute_reg_resx_bits(kvm, &nvhcr_desc, 0, 0) :
+			(typeof(resx)){});
+		break;
 	case SCTLR2_EL1:
 	case SCTLR2_EL2:
 		resx = compute_reg_resx_bits(kvm, &sctlr2_desc, 0, 0);
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 9972dea42d12a..c9bf04944f9cb 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -1830,6 +1830,10 @@ int kvm_init_nv_sysregs(struct kvm_vcpu *vcpu)
 	resx = get_reg_fixed_bits(kvm, HCR_EL2);
 	set_sysreg_masks(kvm, HCR_EL2, resx);
 
+	/* NVHCR_EL2 */
+	resx = get_reg_fixed_bits(kvm, NVHCR_EL2);
+	set_sysreg_masks(kvm, NVHCR_EL2, resx);
+
 	/* HCRX_EL2 */
 	resx = get_reg_fixed_bits(kvm, HCRX_EL2);
 	set_sysreg_masks(kvm, HCRX_EL2, resx);
-- 
2.47.3

[PATCH 21/28] KVM: arm64: Add NVHCR_EL2 handling to the sysreg array

[Marc Zyngier]

Expose NVHCR_EL2 to userspace, and treat the direct access as UNDEF,
as that would only outline a bug in our exception routing.

The generic accessors are also updated to deal with the relatively
uncommon location of that register.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/kvm/sys_regs.c | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 9439c5b2b1fe8..0aeb2e736fde3 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -221,6 +221,20 @@ static void locate_register(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg,
 		if (is_hyp_ctxt(vcpu) && vcpu_el2_e2h_is_set(vcpu))
 			loc->loc = SR_LOC_SPECIAL;
 		break;
+	case NVHCR_EL2:
+		/*
+		 * Yes, NVHCR_EL2 maps to itself when loaded in nested
+		 * context. If you feel like the architecture is double
+		 * backing on itself upside down, you're not alone.
+		 */
+		WARN_ON_ONCE(!kvm_has_nv3(vcpu->kvm));
+		if (is_hyp_ctxt(vcpu)) {
+			loc->loc = SR_LOC_MEMORY;
+		} else {
+			loc->loc = SR_LOC_LOADED | SR_LOC_MAPPED;
+			loc->map_reg = NVHCR_EL2;
+		}
+		break;
 	default:
 		loc->loc = locate_direct_register(vcpu, reg);
 	}
@@ -260,6 +274,7 @@ static u64 read_sr_from_cpu(enum vcpu_sysreg reg)
 	case DACR32_EL2:	val = read_sysreg_s(SYS_DACR32_EL2);	break;
 	case IFSR32_EL2:	val = read_sysreg_s(SYS_IFSR32_EL2);	break;
 	case DBGVCR32_EL2:	val = read_sysreg_s(SYS_DBGVCR32_EL2);	break;
+	case NVHCR_EL2:		val = read_sysreg_s(SYS_NVHCR_EL2);	break;
 	default:		WARN_ON_ONCE(1);
 	}
 
@@ -298,6 +313,7 @@ static void write_sr_to_cpu(enum vcpu_sysreg reg, u64 val)
 	case DACR32_EL2:	write_sysreg_s(val, SYS_DACR32_EL2);	break;
 	case IFSR32_EL2:	write_sysreg_s(val, SYS_IFSR32_EL2);	break;
 	case DBGVCR32_EL2:	write_sysreg_s(val, SYS_DBGVCR32_EL2);	break;
+	case NVHCR_EL2:		write_sysreg_s(val, SYS_NVHCR_EL2);	break;
 	default:		WARN_ON_ONCE(1);
 	}
 }
@@ -2861,6 +2877,16 @@ static unsigned int vncr_el2_visibility(const struct kvm_vcpu *vcpu,
 	return REG_HIDDEN;
 }
 
+static unsigned int nvhcr_el2_visibility(const struct kvm_vcpu *vcpu,
+					const struct sys_reg_desc *rd)
+{
+	if (el2_visibility(vcpu, rd) == 0 &&
+	    kvm_has_feat(vcpu->kvm, ID_AA64MMFR4_EL1, NV_frac, NV3))
+		return 0;
+
+	return REG_HIDDEN;
+}
+
 static unsigned int sctlr2_visibility(const struct kvm_vcpu *vcpu,
 				      const struct sys_reg_desc *rd)
 {
@@ -3774,6 +3800,8 @@ static const struct sys_reg_desc sys_reg_descs[] = {
 			 sve_el2_visibility),
 
 	EL2_REG_VNCR(HCRX_EL2, reset_val, 0),
+	EL2_REG_FILTERED(NVHCR_EL2, undef_access, reset_val, 0,
+			 nvhcr_el2_visibility),
 
 	EL2_REG(TTBR0_EL2, access_rw, reset_val, 0),
 	EL2_REG(TTBR1_EL2, access_rw, reset_val, 0),
-- 
2.47.3

[PATCH 22/28] KVM: arm64: Add routing for NVHCR_EL2 trap

[Marc Zyngier]

NVHCR_EL2 accesses from EL1 are taken to EL2 when HCRX_EL2.NVTGE==0
and HCR_EL2.NV==1. Describe this in the exception routing tables.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/kvm/emulate-nested.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c
index b7f3d86a94031..f5dc578d8c985 100644
--- a/arch/arm64/kvm/emulate-nested.c
+++ b/arch/arm64/kvm/emulate-nested.c
@@ -90,6 +90,7 @@ enum cgt_group_id {
 	CGT_HCRX_EnFPM,
 	CGT_HCRX_TCR2En,
 	CGT_HCRX_SCTLR2En,
+	CGT_HCRX_nNVTGE,
 
 	CGT_CNTHCTL_EL1TVT,
 	CGT_CNTHCTL_EL1TVCT,
@@ -121,6 +122,8 @@ enum cgt_group_id {
 	CGT_MDCR_TDE_TDRA,
 	CGT_MDCR_TDCC_TDE_TDA,
 
+	CGT_HCR_NV_HCRX_nNVTGE,
+
 	CGT_ICH_HCR_TC_TDIR,
 
 	/*
@@ -413,6 +416,12 @@ static const struct trap_bits coarse_trap_bits[] = {
 		.mask		= HCRX_EL2_SCTLR2En,
 		.behaviour	= BEHAVE_FORWARD_RW,
 	},
+	[CGT_HCRX_nNVTGE] = {
+		.index		= HCRX_EL2,
+		.value		= 0,
+		.mask		= HCRX_EL2_NVTGE,
+		.behaviour	= BEHAVE_FORWARD_RW,
+	},
 	[CGT_CNTHCTL_EL1TVT] = {
 		.index		= CNTHCTL_EL2,
 		.value		= CNTHCTL_EL1TVT,
@@ -468,6 +477,7 @@ static const enum cgt_group_id *coarse_control_combo[] = {
 					CGT_HCR_TVM, CGT_HCR_TRVM, CGT_HCRX_SCTLR2En),
 	MCB(CGT_HCR_TPU_TICAB,		CGT_HCR_TPU, CGT_HCR_TICAB),
 	MCB(CGT_HCR_TPU_TOCU,		CGT_HCR_TPU, CGT_HCR_TOCU),
+	MCB(CGT_HCR_NV_HCRX_nNVTGE,	CGT_HCR_NV, CGT_HCRX_nNVTGE),
 	MCB(CGT_HCR_NV1_nNV2_ENSCXT,	CGT_HCR_NV1_nNV2, CGT_HCR_ENSCXT),
 	MCB(CGT_MDCR_TPM_TPMCR,		CGT_MDCR_TPM, CGT_MDCR_TPMCR),
 	MCB(CGT_MDCR_TPM_HPMN,		CGT_MDCR_TPM, CGT_MDCR_HPMN),
@@ -853,6 +863,7 @@ static const struct encoding_to_trap_config encoding_to_cgt[] __initconst = {
 	SR_TRAP(SYS_SCTLR2_EL2,		CGT_HCR_NV),
 	SR_RANGE_TRAP(SYS_HCR_EL2,
 		      SYS_HCRX_EL2,	CGT_HCR_NV),
+	SR_TRAP(SYS_NVHCR_EL2,		CGT_HCR_NV_HCRX_nNVTGE),
 	SR_TRAP(SYS_SMPRIMAP_EL2,	CGT_HCR_NV),
 	SR_TRAP(SYS_SMCR_EL2,		CGT_HCR_NV),
 	SR_RANGE_TRAP(SYS_TTBR0_EL2,
-- 
2.47.3

[PATCH 23/28] KVM: arm64: Add NVHCR_EL2 context switching

[Marc Zyngier]

Since NVHCR_EL2 represents the HCR_EL2 state of the EL1 guest, it
must be dealt with in some particular way:

- for a guest in hyp context (an L1 by definition), NVHCR_EL2 directly
  reflects HCR_EL2 as read and written by the guest itself. It must
  therefore be eagerly synced back with the emulation code which only
  knows about HCR_EL2. This is unconditional if NV3 is available on
  the host.

- For an L2 guest, NVHCR_EL2 is controlled by the L1 guest, and we
  just context switch it like any other EL1 register. Yes, EL1, as
  that's where this thing runs from the PoV of L1. This is conditioned
  on the guest using NV3.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 11 +++++++++++
 arch/arm64/kvm/hyp/vhe/switch.c            | 10 ++++++++--
 2 files changed, 19 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
index a17cbe7582de9..c382848d31947 100644
--- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
+++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
@@ -172,6 +172,10 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
 
 	if (ctxt_has_sctlr2(ctxt))
 		ctxt_sys_reg(ctxt, SCTLR2_EL1) = read_sysreg_el1(SYS_SCTLR2);
+
+	/* Retrieve L2's HCR_EL2, and save it for future use */
+	if (is_nested_nv3_ctxt(ctxt_to_vcpu(ctxt)))
+		ctxt_sys_reg(ctxt, NVHCR_EL2) = read_sysreg_s(SYS_NVHCR_EL2);
 }
 
 static inline void __sysreg_save_el2_return_state(struct kvm_cpu_context *ctxt)
@@ -285,6 +289,13 @@ static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt,
 
 	if (ctxt_has_sctlr2(ctxt))
 		write_sysreg_el1(ctxt_sys_reg(ctxt, SCTLR2_EL1), SYS_SCTLR2);
+
+	/*
+	 * Publish the L2 view of HCR_EL2 to the HW if L1 is using NV3.
+	 * Otherwise, the data is already in place in the L1's own VNCR.
+	 */
+	if (is_nested_nv3_ctxt(ctxt_to_vcpu(ctxt)))
+		write_sysreg_s(ctxt_sys_reg(ctxt, NVHCR_EL2), SYS_NVHCR_EL2);
 }
 
 /* Read the VCPU state's PSTATE, but translate (v)EL2 to EL1. */
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 05bcf8bf7f978..c5c06ae41b229 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -71,7 +71,10 @@ static u64 __compute_hcr(struct kvm_vcpu *vcpu)
 			hcr |= HCR_NV1;
 
 		/* Publish the guest's view of HCR_EL2 to the HW */
-		__vcpu_assign_sys_reg(vcpu, NVHCR_EL2, __vcpu_sys_reg(vcpu, HCR_EL2));
+		if (cpus_have_final_cap(ARM64_HAS_NV3) && vcpu_el2_e2h_is_set(vcpu))
+			write_sysreg_s(__vcpu_sys_reg(vcpu, HCR_EL2), SYS_NVHCR_EL2);
+		else
+			__vcpu_assign_sys_reg(vcpu, NVHCR_EL2, __vcpu_sys_reg(vcpu, HCR_EL2));
 
 		/*
 		 * Nothing in HCR_EL2 should impact running in hypervisor
@@ -565,7 +568,10 @@ static void fixup_nv_guest_exit(struct kvm_vcpu *vcpu)
 		*vcpu_cpsr(vcpu) |= mode;
 
 		/* Publish the latest HCR_EL2 to the emulation */
-		hcr = __vcpu_sys_reg(vcpu, NVHCR_EL2);
+		hcr = (cpus_have_final_cap(ARM64_HAS_NV3) &&
+		       vcpu_el2_e2h_is_set(vcpu)) ?
+			read_sysreg_s(SYS_NVHCR_EL2) :
+			__vcpu_sys_reg(vcpu, NVHCR_EL2);
 
 		__vcpu_assign_sys_reg(vcpu, HCR_EL2, hcr);
 	}
-- 
2.47.3

[PATCH 24/28] KVM: arm64: Engage NV3 ERET trap elision

[Marc Zyngier]

When running on NV3 HW, always engage ERET trap elision when running
the L1 context, as there is no benefit in not doing so.

An L1 can itself engage trap elision by setting its own view of
HCRX_EL2.NVTGE==1, which will subsequently be honored.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/include/asm/kvm_emulate.h | 10 ++++++++++
 arch/arm64/kvm/hyp/vhe/switch.c      |  4 ++++
 2 files changed, 14 insertions(+)

diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index c562d8171d5e1..b32870a5e1236 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -706,6 +706,16 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu)
 
 		if (kvm_has_feat(kvm, ID_AA64ISAR1_EL1, LS64, LS64_V))
 			vcpu->arch.hcrx_el2 |= HCRX_EL2_EnASR;
+
+		/*
+		 * NV3 is a host-specific extension, and we always use it
+		 * when present and that the guest uses NV. It may be be
+		 * hidden from the guest though.
+		 */
+		if (cpus_have_final_cap(ARM64_HAS_NV3) &&
+		    vcpu_has_nv(vcpu) && vcpu_el2_e2h_is_set(vcpu)) {
+			vcpu->arch.hcrx_el2 |= HCRX_EL2_NVTGE;
+		}
 	}
 }
 #endif /* __ARM64_KVM_EMULATE_H__ */
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index c5c06ae41b229..f129f22f15618 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -345,6 +345,10 @@ static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code)
 	u64 esr = kvm_vcpu_get_esr(vcpu);
 	u64 spsr, elr, mode;
 
+	/* With NV3, the fast path is handled in HW */
+	if (cpus_have_final_cap(ARM64_HAS_NV3) && vcpu_el2_e2h_is_set(vcpu))
+		return false;
+
 	/*
 	 * Going through the whole put/load motions is a waste of time
 	 * if this is a VHE guest hypervisor returning to its own
-- 
2.47.3

[PATCH 25/28] KVM: arm64: Engage NV3 TLBI trap elision

[Marc Zyngier]

Similarly to the ERET elision mechanism, FEAT_NV3 can elide TLBIs
that only affects the guest's S1 translation. Enable this, with the
express condition that the guest isn't NV2 aware, as we otherwise
need to trap these TLBIs to deal with VNCR mappings.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/include/asm/kvm_emulate.h | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index b32870a5e1236..d6f432b1558f5 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -715,6 +715,15 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu)
 		if (cpus_have_final_cap(ARM64_HAS_NV3) &&
 		    vcpu_has_nv(vcpu) && vcpu_el2_e2h_is_set(vcpu)) {
 			vcpu->arch.hcrx_el2 |= HCRX_EL2_NVTGE;
+
+			/*
+			 * If the guest is NV2-capable, then we need to see
+			 * all the TLBIs, as configured in HCR_EL2.
+			 * Otherwise, relax the TLBI traps to only TGE=0.
+			 */
+			if (!kvm_has_nv2(vcpu->kvm))
+				vcpu->arch.hcrx_el2 |= (HCRX_EL2_NVnTTLB  |
+							HCRX_EL2_NVnTTLBIS);
 		}
 	}
 }
-- 
2.47.3

[PATCH 26/28] KVM: arm64: Add FEAT_NV3 detection

[Marc Zyngier]

Now that everything is in place to engage the FEAT_NV3 fast-path,
add the detection code to cpufeature.c.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/kernel/cpufeature.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index c9c124b0ccc8e..6ae1c816e2010 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -2627,6 +2627,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
 		.matches = has_cpuid_feature,
 		ARM64_CPUID_FIELDS(ID_AA64MMFR4_EL1, NV_frac, NV2P1)
 	},
+	{
+		.desc = "FEAT_NV3",
+		.capability = ARM64_HAS_NV3,
+		.type = ARM64_CPUCAP_SYSTEM_FEATURE,
+		.matches = has_cpuid_feature,
+		ARM64_CPUID_FIELDS(ID_AA64MMFR4_EL1, NV_frac, NV3)
+	},
 	{
 		.capability = ARM64_HAS_32BIT_EL0_DO_NOT_USE,
 		.type = ARM64_CPUCAP_SYSTEM_FEATURE,
-- 
2.47.3

[PATCH 27/28] KVM: arm64: Expose FEAT_NV3 to guests

[Marc Zyngier]

Further enable FEAT_NV3 by making it visible to NV guests.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/kvm/nested.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index c9bf04944f9cb..64c8fb82fadf6 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -1728,7 +1728,7 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)
 		 * You get EITHER
 		 *
 		 * - FEAT_VHE without FEAT_E2H0
-		 * - FEAT_NV limited to FEAT_NV2(p1)
+		 * - FEAT_NV limited to FEAT_NV2(p1)/NV3
 		 * - HCR_EL2.NV1 being RES0
 		 *
 		 * OR
@@ -1740,7 +1740,9 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)
 		if (test_bit(KVM_ARM_VCPU_HAS_EL2_E2H0, kvm->arch.vcpu_features)) {
 			val = 0;
 		} else {
-			if (cpus_have_final_cap(ARM64_HAS_NV2P1))
+			if (cpus_have_final_cap(ARM64_HAS_NV3))
+				val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64MMFR4_EL1, NV_frac, NV3);
+			else if (cpus_have_final_cap(ARM64_HAS_NV2P1))
 				val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64MMFR4_EL1, NV_frac, NV2P1);
 			else
 				val = SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY);
-- 
2.47.3

[PATCH 28/28] arm64: Add override for ID_AA64MMFR4_EL1.NV_frac

[Marc Zyngier]

In a very unsurprising turn of events, there is a large class of
firmware that is totally unable to deal with FEAT_NV3, and doesn't
set the required SCR2_EL3.NV3En bit, leading to an UNDEF exception
or an unhandled trap to EL3, depending on the implementation.

Allow the unfortunate user to override ID_AA64MMFR4_EL1.NV_frac
and get a working system. Hopefully firmware will be fixed before
actually HW ships, but I have been there before... :-/

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/include/asm/cpufeature.h   |  1 +
 arch/arm64/kernel/cpufeature.c        |  4 +++-
 arch/arm64/kernel/image-vars.h        |  1 +
 arch/arm64/kernel/pi/idreg-override.c | 10 ++++++++++
 4 files changed, 15 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index a57870fa96db5..a42683af79fb5 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -968,6 +968,7 @@ struct arm64_ftr_reg *get_arm64_ftr_reg(u32 sys_id);
 extern struct arm64_ftr_override id_aa64mmfr0_override;
 extern struct arm64_ftr_override id_aa64mmfr1_override;
 extern struct arm64_ftr_override id_aa64mmfr2_override;
+extern struct arm64_ftr_override id_aa64mmfr4_override;
 extern struct arm64_ftr_override id_aa64pfr0_override;
 extern struct arm64_ftr_override id_aa64pfr1_override;
 extern struct arm64_ftr_override id_aa64zfr0_override;
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 6ae1c816e2010..14fbfa8e6b7b5 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -785,6 +785,7 @@ static const struct arm64_ftr_bits ftr_raz[] = {
 struct arm64_ftr_override __read_mostly id_aa64mmfr0_override;
 struct arm64_ftr_override __read_mostly id_aa64mmfr1_override;
 struct arm64_ftr_override __read_mostly id_aa64mmfr2_override;
+struct arm64_ftr_override __read_mostly id_aa64mmfr4_override;
 struct arm64_ftr_override __read_mostly id_aa64pfr0_override;
 struct arm64_ftr_override __read_mostly id_aa64pfr1_override;
 struct arm64_ftr_override __read_mostly id_aa64zfr0_override;
@@ -858,7 +859,8 @@ static const struct __ftr_reg_entry {
 	ARM64_FTR_REG_OVERRIDE(SYS_ID_AA64MMFR2_EL1, ftr_id_aa64mmfr2,
 			       &id_aa64mmfr2_override),
 	ARM64_FTR_REG(SYS_ID_AA64MMFR3_EL1, ftr_id_aa64mmfr3),
-	ARM64_FTR_REG(SYS_ID_AA64MMFR4_EL1, ftr_id_aa64mmfr4),
+	ARM64_FTR_REG_OVERRIDE(SYS_ID_AA64MMFR4_EL1, ftr_id_aa64mmfr4,
+			       &id_aa64mmfr4_override),
 
 	/* Op1 = 0, CRn = 10, CRm = 4 */
 	ARM64_FTR_REG(SYS_MPAMIDR_EL1, ftr_mpamidr),
diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h
index d4c7d45ae6bc8..d15c2cb1b0f28 100644
--- a/arch/arm64/kernel/image-vars.h
+++ b/arch/arm64/kernel/image-vars.h
@@ -51,6 +51,7 @@ PI_EXPORT_SYM(id_aa64isar2_override);
 PI_EXPORT_SYM(id_aa64mmfr0_override);
 PI_EXPORT_SYM(id_aa64mmfr1_override);
 PI_EXPORT_SYM(id_aa64mmfr2_override);
+PI_EXPORT_SYM(id_aa64mmfr4_override);
 PI_EXPORT_SYM(id_aa64pfr0_override);
 PI_EXPORT_SYM(id_aa64pfr1_override);
 PI_EXPORT_SYM(id_aa64smfr0_override);
diff --git a/arch/arm64/kernel/pi/idreg-override.c b/arch/arm64/kernel/pi/idreg-override.c
index bc57b290e5e7b..4e47616bcac23 100644
--- a/arch/arm64/kernel/pi/idreg-override.c
+++ b/arch/arm64/kernel/pi/idreg-override.c
@@ -106,6 +106,15 @@ static const struct ftr_set_desc mmfr2 __prel64_initconst = {
 	},
 };
 
+static const struct ftr_set_desc mmfr4 __prel64_initconst = {
+	.name		= "id_aa64mmfr4",
+	.override	= &id_aa64mmfr4_override,
+	.fields		= {
+		FIELD("nv_frac", ID_AA64MMFR4_EL1_NV_frac_SHIFT, NULL),
+		{}
+	},
+};
+
 static bool __init pfr0_sve_filter(u64 val)
 {
 	/*
@@ -220,6 +229,7 @@ PREL64(const struct ftr_set_desc, reg) regs[] __prel64_initconst = {
 	{ &mmfr0	},
 	{ &mmfr1	},
 	{ &mmfr2	},
+	{ &mmfr4	},
 	{ &pfr0 	},
 	{ &pfr1 	},
 	{ &isar1	},
-- 
2.47.3

Re: [PATCH 01/28] arm64: sysreg: Emit RESx/UNKN values for Mapping definitions

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Invalid `_RESx` macros generated for `Mapping` aliases when the target uses `Fields`.
--

commit ba830a7cd4d5bdb5ba927754ed4d1c4e3a0b4f1c
Author: Marc Zyngier <maz@kernel.org>

arm64: sysreg: Emit RESx/UNKN values for Mapping definitions

The sysreg script uses the Mapping qualifier to indicate an encoding is a
mapping to a particular register, but macro generation for RESx classes was
previously skipped. This patch relaxes that rule to emit RESx and UNKN
values for Mapping qualifiers.

> diff --git a/arch/arm64/tools/gen-sysreg.awk b/arch/arm64/tools/gen-sysreg.awk
> index 86860ab672dc7..d7f7ceb768fe5 100755
> --- a/arch/arm64/tools/gen-sysreg.awk
> +++ b/arch/arm64/tools/gen-sysreg.awk

[ ... ]

> @@ -239,9 +239,15 @@ $1 == "EndSysreg" && block_current() == "Sysreg" {
>  	print ""
>  
>  	next_bit = -1
> -	res0 = null
> -	res1 = null
> -	unkn = null
> +	if ($1 == "Mapping") {
> +		res0 = $2 "_RES0"
> +		res1 = $2 "_RES1"
> +		unkn = $2 "_UNKN"

[Severity: Low]
Does this logic generate invalid macros if the target register uses the
"Fields" directive?

If the target register itself delegates its fields using the "Fields"
directive (e.g., ZCR_EL1 using "Fields ZCR_ELx"), the script explicitly
suppresses generation of the target register's _RESx macros (setting them to
null).

As a result, would newly emitted alias macros (such as ZCR_EL12_RES0)
reference undeclared identifiers (like ZCR_EL1_RES0)? Could this cause
build failures if these mapping macros are later consumed by C code?

> +	} else {
> +		res0 = null
> +		res1 = null
> +		unkn = null
> +	}
>  
>  	next
>  }

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702160248.1377250-1-maz@kernel.org?part=1

Re: [PATCH 10/28] KVM: arm64: Relax CNTHCTL_EL2 handling when FEAT_NV2p1 is present

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

New issues:
- [High] Unconditional use of the CNTKCTL_EL1 hardware alias under FEAT_NV2p1 clobbers the CNTHCTL_EL2.ECV bit for both the host and the guest.

Pre-existing issues:
- [High] Writes to the control-plane system registers CNTHCTL_EL2 and CNTKCTL_EL1 are missing a mandatory ISB instruction.
--

commit 71cc9698fb316a6dc28bed5d5465cd03b6074925
Author: Marc Zyngier <maz@kernel.org>

KVM: arm64: Relax CNTHCTL_EL2 handling when FEAT_NV2p1 is present

This patch optimizes CNTHCTL_EL2 accesses when FEAT_NV2p1 is present by
treating the CNTKCTL_EL1 accessor as normal. It replaces the split approach
with direct use of the CNTKCTL_EL1 hardware alias.

> diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
> index 4155fe89b58a1..db60facad9f3c 100644
> --- a/arch/arm64/kvm/arch_timer.c
> +++ b/arch/arm64/kvm/arch_timer.c

[ ... ]

> @@ -876,8 +876,14 @@ static void timer_set_traps(struct kvm_vcpu *vcpu, struct timer_map *map)
>  	assign_clear_set_bit(tvt02, CNTHCTL_EL1NVVCT, clr, set);
>  	assign_clear_set_bit(tpt02, CNTHCTL_EL1NVPCT, clr, set);
>  
> -	/* This only happens on VHE, so use the CNTHCTL_EL2 accessor. */
> -	sysreg_clear_set(cnthctl_el2, clr, set);
> +	/*
> +	 * This only happens on VHE, so use the CNTHCTL_EL2 accessor, unless
> +	 * we are sure CNTKCTL_EL1 is completely stateful with FEAT_NV2p1.
> +	 */
> +	if (!cpus_have_final_cap(ARM64_HAS_NV2P1))
> +		sysreg_clear_set(cnthctl_el2, clr, set);
> +	else
> +		sysreg_clear_set(cntkctl_el1, clr, set);

[Severity: High]
Does this read-modify-write on the host's physical state using the
cntkctl_el1 alias inadvertently clear KVM's own physical CNTHCTL_EL2.ECV bit?

Since the alias drops or mishandles the ECV bit, reading it might return ECV
as 0 and write it back as 0. This could silently disable the host's ECV
functionality, such as CNTPOFF initialization from kvm_timer_init_vhe().

[Severity: High]
This is a pre-existing issue, but shouldn't there be an isb() instruction
immediately following these control-plane system register writes?

Writing to CNTHCTL_EL2 or CNTKCTL_EL1 without an explicit context
synchronization event can cause the CPU to operate under a stale timer trap
configuration for several cycles before returning.

>  }
>  
>  void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu)

[ ... ]

> diff --git a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c
> index 6f0f046e4ca4e..0c4ef1ce32ae7 100644
> --- a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c
> +++ b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c

[ ... ]

> @@ -69,11 +69,18 @@ static void __sysreg_save_vel2_state(struct kvm_vcpu *vcpu)
>  		 * The EL1 view of CNTKCTL_EL1 has a bunch of RES0 bits where
>  		 * the interesting CNTHCTL_EL2 bits live. So preserve these
>  		 * bits when reading back the guest-visible value.
> +		 *
> +		 * While NV2p1 fixes some of that, it makes CNTHCTL_EL2.ECV
> +		 * even more broken than it already was with NV2.
>  		 */
>  		val = read_sysreg_el1(SYS_CNTKCTL);
> -		val &= CNTKCTL_VALID_BITS;
> -		__vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, &=, ~CNTKCTL_VALID_BITS);
> -		__vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, |=, val);
> +		if (!cpus_have_final_cap(ARM64_HAS_NV2P1)) {
> +			val &= CNTKCTL_VALID_BITS;
> +			__vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, &=, ~CNTKCTL_VALID_BITS);
> +			__vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, |=, val);
> +		} else {
> +			__vcpu_assign_sys_reg(vcpu, CNTHCTL_EL2, val);
> +		}

[Severity: High]
Will directly assigning the hardware value here clobber the saved in-memory
guest state for the broken ECV bit?

As noted in the comment above, NV2p1 makes CNTHCTL_EL2.ECV broken when
accessed via CNTKCTL_EL1. Bypassing the mask-and-merge step could permanently
destroy the guest's saved ECV configuration.

>  	}
>  
>  	__vcpu_assign_sys_reg(vcpu, SP_EL2,	 read_sysreg(sp_el1));

[ ... ]

> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index 1dfc1f88bec82..9439c5b2b1fe8 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c

[ ... ]

> @@ -322,8 +322,10 @@ u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg)
>  		switch (reg) {
>  		case CNTHCTL_EL2:
>  			val = read_sysreg_el1(SYS_CNTKCTL);
> -			val &= CNTKCTL_VALID_BITS;
> -			val |= __vcpu_sys_reg(vcpu, reg) & ~CNTKCTL_VALID_BITS;
> +			if (!cpus_have_final_cap(ARM64_HAS_NV2P1)) {
> +				val &= CNTKCTL_VALID_BITS;
> +				val |= __vcpu_sys_reg(vcpu, reg) & ~CNTKCTL_VALID_BITS;
> +			}
>  			return val;

[Severity: High]
By skipping this mask-and-merge step for NV2p1, won't this directly return
the broken hardware value to userspace?

This appears to hide the ECV bit from userspace entirely, which might break
live migration.

>  		case CPTR_EL2:
>  			if (cpus_have_final_cap(ARM64_HAS_NV2P1))

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702160248.1377250-1-maz@kernel.org?part=10

Re: [PATCH 07/28] KVM: arm64: Don't evaluate HCR_EL2.NV on ERET fast path

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The `kvm_hyp_handle_eret` fast-path incorrectly evaluates an L1 hypervisor's virtual `HFGITR_EL2` register to determine if L1's own `ERET` should be optimized. Because `is_nested_ctxt(vcpu)` is false for L1, the condition falls through to check `__vcpu_sys_reg(vcpu, HFGITR_EL2) & HFGITR_EL2_ERET`. This virtual register reflects L1's trap configuration for its L2 guest, and has no architectural bearing on L1's own execution. If L1 configures L2's ERETs to trap, KVM will spuriously abort the fast-path for L1's local ERETs. (For L2, the check is dead code since `is_nested_ctxt(vcpu)` short-circuits it).
--

commit b3baa466fe54b91fd60bc2025656ec74bb249ab9
Author: Marc Zyngier <maz@kernel.org>

KVM: arm64: Don't evaluate HCR_EL2.NV on ERET fast path

This commit replaces the check for HCR_EL2.NV in the ERET fast path with
a check for is_nested_ctxt(). This ensures we evaluate if we are actually
running an L2 guest, rather than just checking if HCR_EL2.NV is set.

> diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
> index bbe9cebd3d9d5..3b76e0468317b 100644
> --- a/arch/arm64/kvm/hyp/vhe/switch.c
> +++ b/arch/arm64/kvm/hyp/vhe/switch.c

[ ... ]

> @@ -349,8 +349,8 @@ static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code)
>  	 * Unless the trap has to be forwarded further down the line,
>  	 * of course...
>  	 */
> -	if ((__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_NV) ||
> -	    (__vcpu_sys_reg(vcpu, HFGITR_EL2) & HFGITR_EL2_ERET))
> +	if (is_nested_ctxt(vcpu) ||
> +	    __vcpu_sys_reg(vcpu, HFGITR_EL2) & HFGITR_EL2_ERET)

[Severity: Medium]
Will this apply L1's virtual trap configuration to L1's own execution?

Since is_nested_ctxt(vcpu) is true for L2 guests, this condition will
short-circuit and the HFGITR_EL2 check becomes dead code for L2.

For an L1 guest, is_nested_ctxt(vcpu) is false, causing it to evaluate the
HFGITR_EL2 check. However, this virtual register reflects L1's trap
configuration for its L2 guest and doesn't apply to L1's own execution.

If L1 configures L2's ERETs to trap, won't KVM spuriously abort the
fast-path for L1's local ERETs and incorrectly fall back to the slow path?

>  		return false;
>  
>  	spsr = read_sysreg_el1(SYS_SPSR);

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702160248.1377250-1-maz@kernel.org?part=7

Re: [PATCH 18/28] KVM: arm64: Add kvm_has_nv{2,3}() predicates

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] `is_nested_nv3_ctxt()` incorrectly identifies NV3 context when the guest clears `HCR_EL2.E2H`, diverging from hardware execution semantics.
--

commit c75857355bb73ff60c1d0206705bb863784f8183
Author: Marc Zyngier <maz@kernel.org>

KVM: arm64: Add kvm_has_nv{2,3}() predicates

Add a new set of predicates indicating whether VM is capable of
NV2, NV3, and is in a nested NV3 context. This is going to become
useful as we start dealing with a mix of behaviours.

> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index 9831166695186..c562d8171d5e1 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h

[ ... ]

> +static inline bool kvm_has_nv3(struct kvm *kvm)
> +{
> +	return (cpus_have_final_cap(ARM64_HAS_NESTED_VIRT) &&
> +		cpus_have_final_cap(ARM64_HAS_NV3) &&
> +		kvm_has_feat(kvm, ID_AA64MMFR4_EL1, NV_frac, NV3));
> +}
> +
> +static inline bool is_nested_nv3_ctxt(struct kvm_vcpu *vcpu)
> +{
> +	return (has_vhe() && kvm_has_nv3(vcpu->kvm) && is_nested_ctxt(vcpu) &&
> +		(__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_EL2_NV) &&
> +		(__vcpu_sys_reg(vcpu, HCRX_EL2) & HCRX_EL2_NVTGE));
> +}

[Severity: High]
Does this predicate miss a mandatory check for HCR_EL2.E2H?

Architecturally, NVTGE is RES0 when E2H is 0. If a guest sets HCR_EL2.NV = 1
and HCRX_EL2.NVTGE = 1, but leaves HCR_EL2.E2H = 0, the hardware will not
execute the guest under NV3 semantics.

If is_nested_nv3_ctxt() evaluates to true in this scenario, could it lead to
KVM state desynchronization?

For example, KVM might unconditionally save SYS_NVHCR_EL2 to the software
context during exit handling. Since the hardware wasn't actually using NV3,
SYS_NVHCR_EL2 wouldn't be updated with guest writes and would contain stale
data.

Would this result in KVM later loading that stale value into the guest's
HCR_EL2 when context switching back, clobbering the guest's register? Should
this include a check for vcpu_el2_e2h_is_set(vcpu)?

>  /*
>   * The layout of SPSR for an AArch32 state is different when observed from an
>   * AArch64 SPSR_ELx or an AArch32 SPSR_*. This function generates the AArch32

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702160248.1377250-1-maz@kernel.org?part=18

Re: [PATCH 22/28] KVM: arm64: Add routing for NVHCR_EL2 trap

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The trap routing condition for `NVHCR_EL2` incorrectly uses a logical OR (`MCB`) instead of a logical AND, causing improper trap injection to guest EL2.
--

commit 6d634fa5ff4e9b082f4e46e999fa4990ecf38957
Author: Marc Zyngier <maz@kernel.org>

KVM: arm64: Add routing for NVHCR_EL2 trap
    
NVHCR_EL2 accesses from EL1 are taken to EL2 when HCRX_EL2.NVTGE==0
and HCR_EL2.NV==1. Describe this in the exception routing tables.

> diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c
> index b7f3d86a94031..f5dc578d8c985 100644
> --- a/arch/arm64/kvm/emulate-nested.c
> +++ b/arch/arm64/kvm/emulate-nested.c
[ ... ]
> @@ -468,6 +477,7 @@ static const enum cgt_group_id *coarse_control_combo[] = {
>  					CGT_HCR_TVM, CGT_HCR_TRVM, CGT_HCRX_SCTLR2En),
>  	MCB(CGT_HCR_TPU_TICAB,		CGT_HCR_TPU, CGT_HCR_TICAB),
>  	MCB(CGT_HCR_TPU_TOCU,		CGT_HCR_TPU, CGT_HCR_TOCU),
> +	MCB(CGT_HCR_NV_HCRX_nNVTGE,	CGT_HCR_NV, CGT_HCRX_nNVTGE),

[Severity: High]
Does this correctly evaluate the trap condition as an AND?

The commit message indicates the trap is taken when "HCRX_EL2.NVTGE==0 and
HCR_EL2.NV==1". However, coarse_control_combo[] uses the MCB() macro, which
evaluates multiple conditions using a bitwise OR:

arch/arm64/kvm/emulate-nested.c:__compute_trap_behaviour() {
	...
		for (int i = 0; cgids[i] != __RESERVED__; i++)
			b |= __compute_trap_behaviour(vcpu, cgids[i], b);
	...
}

Because this ORs the behaviors, the trap will be forwarded if HCR_EL2.NV == 1
OR HCRX_EL2.NVTGE == 0 evaluates to forward. This would erroneously send the
trap to EL2 even if normal execution without nested virt is active (NV == 0
and NVTGE == 0).

Should this use a __COMPLEX_CONDITIONS__ callback instead to enforce the
logical AND?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702160248.1377250-1-maz@kernel.org?part=22

Re: [PATCH 05/28] KVM: arm64: Plumb HCRX_EL2.SRMASKEn in HCRX_EL2 sanitisation

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Missing RES0 constraints in system register feature map for `HCRX_EL2` allows nested guests to set reserved hardware bits.
--

commit 5fb0fde52515a7e2e03636fdc18f15c38a091ddd
Author: Marc Zyngier <maz@kernel.org>

KVM: arm64: Plumb HCRX_EL2.SRMASKEn in HCRX_EL2 sanitisation

This patch adds the HCRX_EL2.SRMASKEn bit to the HCRX_EL2 feature map to
enable FEAT_SRMASK for a guest. It also updates the system register
definitions to reflect the new bit field.

> diff --git a/arch/arm64/kvm/config.c b/arch/arm64/kvm/config.c
> --- a/arch/arm64/kvm/config.c
> +++ b/arch/arm64/kvm/config.c
> @@ -904,6 +904,7 @@ static const DECLARE_FEAT_MAP_FGT(hdfgwtr2_desc, hdfgwtr2_masks,
>  
>  
>  static const struct reg_bits_to_feat_map hcrx_feat_map[] = {
> +	NEEDS_FEAT(HCRX_EL2_SRMASKEn, FEAT_SRMASK),

[Severity: High]
This isn't a bug introduced by this patch, but a pre-existing issue in
hcrx_feat_map: does the omission of FORCE_RES0(HCRX_EL2_RES0) and
FORCE_RES1(HCRX_EL2_RES1) allow a nested guest to set architecturally reserved
hardware bits?

Because FORCE_RES0(HCRX_EL2_RES0) is omitted from this array, the architectural
RES0 bits (like 63:27, 25, 21:17) are not included in the virtual RES0 mask.
If a nested guest writes a 1 to these RES0 bits, KVM's dynamic mask application
in __vcpu_sys_reg() will leave those bits unmodified.

When KVM later transitions to the guest, it will write this unsanitized value
directly to the host's hardware SYS_HCRX_EL2 register. Can this violate
architectural invariants and trigger CONSTRAINED UNPREDICTABLE behavior or
silently enable unknown future hardware features?

[ ... ]

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702160248.1377250-1-maz@kernel.org?part=5

Re: [PATCH 11/28] KVM: arm64: Expose FEAT_NV2p1 to NV guests

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Inconsistent preservation of `ID_AA64MMFR4_EL1` feature fields based on host support for `ARM64_HAS_NV2P1` (and later `ARM64_HAS_NV3`), leading to uncontrolled UAPI feature leaks and migration breakages.
--

commit 11eacfaa8962f15232e1ae99c1e9f7d1dfee6846
Author: Marc Zyngier <maz@kernel.org>

KVM: arm64: Expose FEAT_NV2p1 to NV guests

This commit exposes the FEAT_NV2p1 feature to nested virtualization guests.
It does so by updating the ID_AA64MMFR4_EL1 feature register limits in
limit_nv_id_reg() based on host support.

> diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
> --- a/arch/arm64/kvm/nested.c
> +++ b/arch/arm64/kvm/nested.c
> @@ -1740,7 +1740,11 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)
>  		if (test_bit(KVM_ARM_VCPU_HAS_EL2_E2H0, kvm->arch.vcpu_features)) {
>  			val = 0;
>  		} else {
> -			val = SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY);
> +			if (cpus_have_final_cap(ARM64_HAS_NV2P1))
> +				val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64MMFR4_EL1, NV_frac, NV2P1);
> +			else
> +				val = SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY);

[Severity: High]
Does this introduce inconsistent preservation of ID_AA64MMFR4_EL1 feature fields?

When the host supports NV2P1, KVM uses ID_REG_LIMIT_FIELD_ENUM(), which only
modifies the NV_frac field and preserves the rest of the original val. This 
means the guest will inherit other host-populated feature bits from
ID_AA64MMFR4_EL1 (like ASID2, HACDBS, FGWTE3, etc.).

However, if the host is NV2-only, it falls back to the original assignment
using SYS_FIELD_PREP_ENUM(), which effectively wipes out all other fields in
val.

Could this uncontrolled feature leak cause guest-visible state divergence based
on the underlying hardware and break live migration?

>  			val &= ~ID_AA64MMFR4_EL1_E2H0;
>  			val |= SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, E2H0, NI_NV1);
>  		}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702160248.1377250-1-maz@kernel.org?part=11

Re: [PATCH 03/28] KVM: arm64: Merge guest's HCRX_EL2 using NV_HCRX_GUEST_EXCLUDE

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Missing RES0 Sanitization / Unsafe Register Merge for HCRX_EL2
--

commit a8dae7a323ff7e33355d5cb4c144294087a9bbfa
Author: Marc Zyngier <maz@kernel.org>
KVM: arm64: Merge guest's HCRX_EL2 using NV_HCRX_GUEST_EXCLUDE

This commit changes how the guest-provided HCRX_EL2 value is merged with
the host's value. It replaces positive masking with an exclusion list
of bits that the guest is not allowed to override.

> diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
> index 4bf624a49591d..8e5f492f39086 100644
> --- a/arch/arm64/kvm/hyp/include/hyp/switch.h
> +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h

[ ... ]

> @@ -350,8 +368,8 @@ static inline void __activate_traps_common(struct kvm_vcpu *vcpu)
>  		u64 hcrx = vcpu->arch.hcrx_el2;
>  		if (is_nested_ctxt(vcpu)) {
>  			u64 val = __vcpu_sys_reg(vcpu, HCRX_EL2);
> -			hcrx |= val & __HCRX_EL2_MASK;
> -			hcrx &= ~(~val & __HCRX_EL2_nMASK);
> +			hcrx |= (val & ~NV_HCRX_GUEST_EXCLUDE);
> +			hcrx &= ~(~val & ~NV_HCRX_GUEST_EXCLUDE);

[Severity: High]
Does this allow an untrusted L1 guest to write garbage to the
architecturally RES0 bits of the physical HCRX_EL2 register?

By switching from a positive mask to a negative exclusion mask, any bits
not explicitly excluded are now merged into the host value.

If the guest modifies its VNCR page at the VNCR_HCRX_EL2 offset and sets
RES0 bits, these bits would be passed through. Since hcrx_feat_map lacks
FORCE_RES0(HCRX_EL2_RES0), KVM's register mask application in
kvm_vcpu_apply_reg_masks() will fail to clear them before they reach this
merge point in __activate_traps_common().

Could this lead to unpredictable behavior if the physical hardware
register is written with garbage in those RES0 bits?

>  		}
>  
>  		ctxt_sys_reg(hctxt, HCRX_EL2) = read_sysreg_s(SYS_HCRX_EL2);

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702160248.1377250-1-maz@kernel.org?part=3

Re: [PATCH 27/28] KVM: arm64: Expose FEAT_NV3 to guests

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [Critical] A maliciously or incorrectly configured vCPU can trigger a host NULL pointer dereference by requesting an ID register configuration where NV_frac = 0, causing KVM to skip VNCR TLB allocation while still trapping VNCR accesses.
--

commit ca23c995d48d1c5c8370918930aa81e2eeb147ab
Author: Marc Zyngier <maz@kernel.org>

KVM: arm64: Expose FEAT_NV3 to guests

Further enable FEAT_NV3 by making it visible to NV guests.

> diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
> index c9bf04944f9cb..64c8fb82fadf6 100644
> --- a/arch/arm64/kvm/nested.c
> +++ b/arch/arm64/kvm/nested.c
> @@ -1728,7 +1728,7 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)
>  		 * You get EITHER
>  		 *
>  		 * - FEAT_VHE without FEAT_E2H0
> -		 * - FEAT_NV limited to FEAT_NV2(p1)
> +		 * - FEAT_NV limited to FEAT_NV2(p1)/NV3
>  		 * - HCR_EL2.NV1 being RES0
>  		 *
>  		 * OR
> @@ -1740,7 +1740,9 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)
>  		if (test_bit(KVM_ARM_VCPU_HAS_EL2_E2H0, kvm->arch.vcpu_features)) {
>  			val = 0;

[Severity: Critical]
This is a pre-existing issue, but could this cause a host NULL pointer
dereference?

If userspace configures a vCPU with KVM_ARM_VCPU_HAS_EL2_E2H0, this branch
forces the entire ID_AA64MMFR4_EL1 register to 0. This includes the NV_frac
field. Alternatively, userspace could explicitly set ID_AA64MMFR4_EL1.NV_frac
to 0 via KVM's ID register UAPI.

When KVM initializes the nested vCPU, it calls kvm_vcpu_allocate_vncr_tlb()
which checks NV_frac:

arch/arm64/kvm/nested.c:kvm_vcpu_allocate_vncr_tlb() {
        if (!kvm_has_feat(vcpu->kvm, ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY))
                return 0;
        ...
}

Because NV_frac is 0, the check evaluates to false and the function silently
returns success without allocating vcpu->arch.vncr_tlb.

Since vcpu_has_nv() only checks for KVM_ARM_VCPU_HAS_EL2, the vCPU is allowed
to run. If the guest then accesses a VNCR-trapped register or sets HCR_EL2.NV,
the host exit handler calls kvm_translate_vncr() or kvm_map_l1_vncr(). Both
functions unconditionally dereference the unallocated vncr_tlb:

arch/arm64/kvm/nested.c:kvm_translate_vncr() {
        struct vncr_tlb *vt = vcpu->arch.vncr_tlb;
        ...
        vt->valid = false;
        ...
}

Will this lead to a kernel panic when unprivileged userspace configures a
vCPU this way?

>  		} else {
> -			if (cpus_have_final_cap(ARM64_HAS_NV2P1))
> +			if (cpus_have_final_cap(ARM64_HAS_NV3))
> +				val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64MMFR4_EL1, NV_frac, NV3);
> +			else if (cpus_have_final_cap(ARM64_HAS_NV2P1))
>  				val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64MMFR4_EL1, NV_frac, NV2P1);
>  			else
>  				val = SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY);

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702160248.1377250-1-maz@kernel.org?part=27

Re: [PATCH 26/28] KVM: arm64: Add FEAT_NV3 detection

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Writes to the SYS_NVHCR_EL2 control system register are not followed by an immediate isb().
- [High] Writes to CNTKCTL_EL1 and CNTHCTL_EL2 control registers using sysreg_clear_set() lack an immediate isb().
--

commit b72de4ae5666d0aaa35b94915af5eab1152df83b
Author: Marc Zyngier <maz@kernel.org>
KVM: arm64: Add FEAT_NV3 detection

This commit adds the detection code for FEAT_NV3 to the cpufeature
capabilities. This engages the FEAT_NV3 fast-path when the hardware
supports it.

> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index c9c124b0ccc8e..6ae1c816e2010 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -2627,6 +2627,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
>  		.matches = has_cpuid_feature,
>  		ARM64_CPUID_FIELDS(ID_AA64MMFR4_EL1, NV_frac, NV2P1)
>  	},

[Severity: High]
Could the system operate under a stale configuration when updating timer
control registers based on the ARM64_HAS_NV2P1 capability?

In arch/arm64/kvm/arch_timer.c:timer_set_traps(), the control registers
are modified without an immediate isb() following the update:

arch/arm64/kvm/arch_timer.c:timer_set_traps()
	if (!cpus_have_final_cap(ARM64_HAS_NV2P1))
		sysreg_clear_set(cnthctl_el2, clr, set);
	else
		sysreg_clear_set(cntkctl_el1, clr, set);

The sysreg_clear_set() macro expands to a read-modify-write sequence that
does not include an isb() as the very next instruction. Is an explicit
barrier required here to prevent undefined pipeline state?

> +	{
> +		.desc = "FEAT_NV3",
> +		.capability = ARM64_HAS_NV3,
> +		.type = ARM64_CPUCAP_SYSTEM_FEATURE,
> +		.matches = has_cpuid_feature,
> +		ARM64_CPUID_FIELDS(ID_AA64MMFR4_EL1, NV_frac, NV3)
> +	},

[Severity: High]
Does enabling the ARM64_HAS_NV3 capability expose missing synchronization
when writing the SYS_NVHCR_EL2 control register?

Looking at __compute_hcr() in arch/arm64/kvm/hyp/vhe/switch.c, the guest
view is published to SYS_NVHCR_EL2 without an immediate isb():

arch/arm64/kvm/hyp/vhe/switch.c:__compute_hcr()
		if (cpus_have_final_cap(ARM64_HAS_NV3) && vcpu_el2_e2h_is_set(vcpu))
			write_sysreg_s(__vcpu_sys_reg(vcpu, HCR_EL2), SYS_NVHCR_EL2);
		else
			__vcpu_assign_sys_reg(vcpu, NVHCR_EL2, __vcpu_sys_reg(vcpu, HCR_EL2));

Similarly, the L2 view of HCR_EL2 is restored without an immediate isb()
before returning:

arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h:__sysreg_restore_el1_state()
	if (is_nested_nv3_ctxt(ctxt_to_vcpu(ctxt)))
		write_sysreg_s(ctxt_sys_reg(ctxt, NVHCR_EL2), SYS_NVHCR_EL2);
}

Could the CPU operate under a stale configuration for several cycles,
leading to unpredictable behavior or unexpected traps?

>  	{
>  		.capability = ARM64_HAS_32BIT_EL0_DO_NOT_USE,
>  		.type = ARM64_CPUCAP_SYSTEM_FEATURE,

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702160248.1377250-1-maz@kernel.org?part=26

Re: [PATCH 23/28] KVM: arm64: Add NVHCR_EL2 context switching

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Cross-VM Information Leak via unswitched SYS_NVHCR_EL2 physical register
--

commit ce2ffdc8cef6935283480a976d01a70bb788a885
Author: Marc Zyngier <maz@kernel.org>

KVM: arm64: Add NVHCR_EL2 context switching
    
This commit adds context switching for the NVHCR_EL2 register to manage
the HCR_EL2 state of an EL1 guest. The switch is conditional on the guest
using NV3.

> diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
> --- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
> +++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
> @@ -172,6 +172,10 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
>  
>  	if (ctxt_has_sctlr2(ctxt))
>  		ctxt_sys_reg(ctxt, SCTLR2_EL1) = read_sysreg_el1(SYS_SCTLR2);
> +
> +	/* Retrieve L2's HCR_EL2, and save it for future use */
> +	if (is_nested_nv3_ctxt(ctxt_to_vcpu(ctxt)))
> +		ctxt_sys_reg(ctxt, NVHCR_EL2) = read_sysreg_s(SYS_NVHCR_EL2);

[Severity: High]
Could this leak information between VMs when the host hardware supports NV3
but the guest doesn't have it enabled?

If the host hardware implements FEAT_NV3 and runs an L2 guest with physical
HCR_EL2.NV=1, the hardware will redirect EL1 accesses of HCR_EL2 to the
physical SYS_NVHCR_EL2 register.

Since is_nested_nv3_ctxt() evaluates to false when the guest lacks the
virtual NV3 capability, __sysreg_save_el1_state() skips saving this register.
Does this allow an untrusted L2 guest to read the SYS_NVHCR_EL2 state left
behind by a previously running VM, bypassing software isolation?

>  }
>  
[ ... ]
> @@ -285,6 +289,13 @@ static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt,
>  
>  	if (ctxt_has_sctlr2(ctxt))
>  		write_sysreg_el1(ctxt_sys_reg(ctxt, SCTLR2_EL1), SYS_SCTLR2);
> +
> +	/*
> +	 * Publish the L2 view of HCR_EL2 to the HW if L1 is using NV3.
> +	 * Otherwise, the data is already in place in the L1's own VNCR.
> +	 */
> +	if (is_nested_nv3_ctxt(ctxt_to_vcpu(ctxt)))
> +		write_sysreg_s(ctxt_sys_reg(ctxt, NVHCR_EL2), SYS_NVHCR_EL2);

[Severity: High]
Similar to the save path above in __sysreg_save_el1_state(), should this
context switch be based on the host's hardware capability rather than the
guest's software configuration?

If the physical register isn't explicitly overwritten in
__sysreg_restore_el1_state() because is_nested_nv3_ctxt() is false, the
L2 guest could potentially read the stale configuration state of a
completely different VM.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702160248.1377250-1-maz@kernel.org?part=23

Re: [PATCH 25/28] KVM: arm64: Engage NV3 TLBI trap elision

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:

New issues:
- [Critical] Enabling FEAT_NV3 TLBI elision (NVnTTLB) breaks KVM's nested S2 MMU caching, leaving stale L2 guest S1 TLB entries across cached physical VMIDs.
- [High] TLBI trap elision for NV1 guests is nested inside a check requiring the guest to use VHE, rendering the optimization dead code.
- [Medium] NV3 TLBI trap elision is incomplete because HCRX_EL2_NVnTTLBOS is omitted from the TLBI elision mask.

Pre-existing issues:
- [High] vcpu_set_hcrx() statically evaluates the guest's dynamic HCR_EL2.E2H state at boot, permanently disabling HCRX_EL2_NVTGE even if the guest later enables VHE.
--

commit e34c9c8b4a16d0026abc090c888c49e7f1c66d8a
Author: Marc Zyngier <maz@kernel.org>

KVM: arm64: Engage NV3 TLBI trap elision

This commit enables FEAT_NV3 TLBI trap elision for guests that are not
NV2-aware. It relaxes TLBI traps to only TGE=0 using HCRX_EL2 bits.

> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index b32870a5e1236..d6f432b1558f5 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h
> @@ -715,6 +715,15 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu)
>  		if (cpus_have_final_cap(ARM64_HAS_NV3) &&
>  		    vcpu_has_nv(vcpu) && vcpu_el2_e2h_is_set(vcpu)) {

[Severity: High]
This isn't a bug introduced by this patch, but because vcpu_set_hcrx is
called once during kvm_calculate_traps at the first VCPU run, the guest
has not yet executed and its virtual HCR_EL2.E2H is 0. 

Since vcpu_el2_e2h_is_set checks the dynamic HCR_EL2 register, does this
evaluate to false and permanently disable HCRX_EL2_NVTGE even if the guest
later sets HCR_EL2.E2H at runtime?

[Severity: High]
Does checking vcpu_el2_e2h_is_set here prevent the NV1 TLBI elision
optimization from ever being reached?

Since an NV1 guest does not have VHE enabled (E2H=0), this outer condition
will evaluate to false, meaning the inner !kvm_has_nv2(vcpu->kvm) block
added below will be skipped.

>  			vcpu->arch.hcrx_el2 |= HCRX_EL2_NVTGE;
> +
> +			/*
> +			 * If the guest is NV2-capable, then we need to see
> +			 * all the TLBIs, as configured in HCR_EL2.
> +			 * Otherwise, relax the TLBI traps to only TGE=0.
> +			 */
> +			if (!kvm_has_nv2(vcpu->kvm))
> +				vcpu->arch.hcrx_el2 |= (HCRX_EL2_NVnTTLB  |
> +							HCRX_EL2_NVnTTLBIS);

[Severity: Medium]
Does this mask omit the Outer Shareable S1 TLBI elision bit
(HCRX_EL2_NVnTTLBOS)?

Without it, will Outer Shareable S1 TLBI instructions continue to
trap to KVM, missing out on the intended performance optimization?

[Severity: Critical]
Does enabling hardware TLBI elision here break KVM's nested S2 MMU
caching?

KVM caches multiple S2 MMU contexts (with distinct physical VMIDs) for
the same virtual VMID, and defers physical VMID switches until guest entry.

If a guest hypervisor writes to VTTBR_EL2 and executes an S1 TLBI natively,
hardware will only invalidate the currently active physical VMID.

Could this cause the hardware to invalidate the wrong context (the
hypervisor's own S1 TLB instead of the L2 guest's) and miss other cached
physical VMIDs sharing the virtual VMID, potentially leading to stale
TLB entries?

>  		}
>  	}
>  }

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702160248.1377250-1-maz@kernel.org?part=25

Re: [PATCH 01/28] arm64: sysreg: Emit RESx/UNKN values for Mapping definitions

[Marc Zyngier]

On Thu, 02 Jul 2026 17:19:27 +0100,
sashiko-bot@kernel.org wrote:
> 
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [Low] Invalid `_RESx` macros generated for `Mapping` aliases when the target uses `Fields`.
> --
> 
> commit ba830a7cd4d5bdb5ba927754ed4d1c4e3a0b4f1c
> Author: Marc Zyngier <maz@kernel.org>
> 
> arm64: sysreg: Emit RESx/UNKN values for Mapping definitions
> 
> The sysreg script uses the Mapping qualifier to indicate an encoding is a
> mapping to a particular register, but macro generation for RESx classes was
> previously skipped. This patch relaxes that rule to emit RESx and UNKN
> values for Mapping qualifiers.
> 
> > diff --git a/arch/arm64/tools/gen-sysreg.awk b/arch/arm64/tools/gen-sysreg.awk
> > index 86860ab672dc7..d7f7ceb768fe5 100755
> > --- a/arch/arm64/tools/gen-sysreg.awk
> > +++ b/arch/arm64/tools/gen-sysreg.awk
> 
> [ ... ]
> 
> > @@ -239,9 +239,15 @@ $1 == "EndSysreg" && block_current() == "Sysreg" {
> >  	print ""
> >  
> >  	next_bit = -1
> > -	res0 = null
> > -	res1 = null
> > -	unkn = null
> > +	if ($1 == "Mapping") {
> > +		res0 = $2 "_RES0"
> > +		res1 = $2 "_RES1"
> > +		unkn = $2 "_UNKN"
> 
> [Severity: Low]
> Does this logic generate invalid macros if the target register uses the
> "Fields" directive?

Interesting point. This indeed breaks for a Mapping->Fields
indirection.

We don't have anything of this shape just yet, but it shouldn't hurt
to generate the RESx/UNKN symbols for those as well.

	M.

-- 
Jazz isn't dead. It just smells funny.

Re: [PATCH 10/28] KVM: arm64: Relax CNTHCTL_EL2 handling when FEAT_NV2p1 is present

[Marc Zyngier]

On Thu, 02 Jul 2026 17:21:31 +0100,
sashiko-bot@kernel.org wrote:
> 
> Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
> 
> New issues:
> - [High] Unconditional use of the CNTKCTL_EL1 hardware alias under FEAT_NV2p1 clobbers the CNTHCTL_EL2.ECV bit for both the host and the guest.
> 
> Pre-existing issues:
> - [High] Writes to the control-plane system registers CNTHCTL_EL2 and CNTKCTL_EL1 are missing a mandatory ISB instruction.
> --
> 
> commit 71cc9698fb316a6dc28bed5d5465cd03b6074925
> Author: Marc Zyngier <maz@kernel.org>
> 
> KVM: arm64: Relax CNTHCTL_EL2 handling when FEAT_NV2p1 is present
> 
> This patch optimizes CNTHCTL_EL2 accesses when FEAT_NV2p1 is present by
> treating the CNTKCTL_EL1 accessor as normal. It replaces the split approach
> with direct use of the CNTKCTL_EL1 hardware alias.
> 
> > diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
> > index 4155fe89b58a1..db60facad9f3c 100644
> > --- a/arch/arm64/kvm/arch_timer.c
> > +++ b/arch/arm64/kvm/arch_timer.c
> 
> [ ... ]
> 
> > @@ -876,8 +876,14 @@ static void timer_set_traps(struct kvm_vcpu *vcpu, struct timer_map *map)
> >  	assign_clear_set_bit(tvt02, CNTHCTL_EL1NVVCT, clr, set);
> >  	assign_clear_set_bit(tpt02, CNTHCTL_EL1NVPCT, clr, set);
> >  
> > -	/* This only happens on VHE, so use the CNTHCTL_EL2 accessor. */
> > -	sysreg_clear_set(cnthctl_el2, clr, set);
> > +	/*
> > +	 * This only happens on VHE, so use the CNTHCTL_EL2 accessor, unless
> > +	 * we are sure CNTKCTL_EL1 is completely stateful with FEAT_NV2p1.
> > +	 */
> > +	if (!cpus_have_final_cap(ARM64_HAS_NV2P1))
> > +		sysreg_clear_set(cnthctl_el2, clr, set);
> > +	else
> > +		sysreg_clear_set(cntkctl_el1, clr, set);
> 
> [Severity: High]
> Does this read-modify-write on the host's physical state using the
> cntkctl_el1 alias inadvertently clear KVM's own physical CNTHCTL_EL2.ECV bit?
> 
> Since the alias drops or mishandles the ECV bit, reading it might return ECV
> as 0 and write it back as 0. This could silently disable the host's ECV
> functionality, such as CNTPOFF initialization from kvm_timer_init_vhe().

No, and that's the whole point of NV2p1: the CNTKCTL_EL1 bits are
guaranteed to be stateful, and we don't need to do anything else

> [Severity: High]
> This is a pre-existing issue, but shouldn't there be an isb() instruction
> immediately following these control-plane system register writes?

"control-plane"? WTF???

And no. These bits only need to take effect when entering the guest,
for which we already have a context synchronisation event.

> Writing to CNTHCTL_EL2 or CNTKCTL_EL1 without an explicit context
> synchronization event can cause the CPU to operate under a stale timer trap
> configuration for several cycles before returning.

Bollocks.

	M.

-- 
Jazz isn't dead. It just smells funny.

Re: [PATCH 07/28] KVM: arm64: Don't evaluate HCR_EL2.NV on ERET fast path

[Marc Zyngier]

On Thu, 02 Jul 2026 17:24:23 +0100,
sashiko-bot@kernel.org wrote:
> 
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [Medium] The `kvm_hyp_handle_eret` fast-path incorrectly evaluates an L1 hypervisor's virtual `HFGITR_EL2` register to determine if L1's own `ERET` should be optimized. Because `is_nested_ctxt(vcpu)` is false for L1, the condition falls through to check `__vcpu_sys_reg(vcpu, HFGITR_EL2) & HFGITR_EL2_ERET`. This virtual register reflects L1's trap configuration for its L2 guest, and has no architectural bearing on L1's own execution. If L1 configures L2's ERETs to trap, KVM will spuriously abort the fast-path for L1's local ERETs. (For L2, the check is dead code since `is_nested_ctxt(vcpu)` short-circuits it).
> --
> 
> commit b3baa466fe54b91fd60bc2025656ec74bb249ab9
> Author: Marc Zyngier <maz@kernel.org>
> 
> KVM: arm64: Don't evaluate HCR_EL2.NV on ERET fast path
> 
> This commit replaces the check for HCR_EL2.NV in the ERET fast path with
> a check for is_nested_ctxt(). This ensures we evaluate if we are actually
> running an L2 guest, rather than just checking if HCR_EL2.NV is set.
> 
> > diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
> > index bbe9cebd3d9d5..3b76e0468317b 100644
> > --- a/arch/arm64/kvm/hyp/vhe/switch.c
> > +++ b/arch/arm64/kvm/hyp/vhe/switch.c
> 
> [ ... ]
> 
> > @@ -349,8 +349,8 @@ static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code)
> >  	 * Unless the trap has to be forwarded further down the line,
> >  	 * of course...
> >  	 */
> > -	if ((__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_NV) ||
> > -	    (__vcpu_sys_reg(vcpu, HFGITR_EL2) & HFGITR_EL2_ERET))
> > +	if (is_nested_ctxt(vcpu) ||
> > +	    __vcpu_sys_reg(vcpu, HFGITR_EL2) & HFGITR_EL2_ERET)
> 
> [Severity: Medium]
> Will this apply L1's virtual trap configuration to L1's own execution?
> 
> Since is_nested_ctxt(vcpu) is true for L2 guests, this condition will
> short-circuit and the HFGITR_EL2 check becomes dead code for L2.

And that's intentional. If we take an ERET from a nested context, then
the exception must be forwarded to the L1, because we're not the ones
in charge of the L2.

And actually, *any* ERET from a nested context must be deferred to the
L1 until we get NV3. So this FGT check is not only dead code, it is
wrong and must go.

> 
> For an L1 guest, is_nested_ctxt(vcpu) is false, causing it to evaluate the
> HFGITR_EL2 check. However, this virtual register reflects L1's trap
> configuration for its L2 guest and doesn't apply to L1's own execution.
> 
> If L1 configures L2's ERETs to trap, won't KVM spuriously abort the
> fast-path for L1's local ERETs and incorrectly fall back to the slow path?

If L1 configures L2's ERETs to trap, then there *cannot* be a fast
path. It is by definition handled on the slow path, since we need to
switch between L2 and L1.

	M.

-- 
Jazz isn't dead. It just smells funny.

Re: [PATCH 18/28] KVM: arm64: Add kvm_has_nv{2,3}() predicates

[Marc Zyngier]

On Thu, 02 Jul 2026 17:25:17 +0100,
sashiko-bot@kernel.org wrote:
> 
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [High] `is_nested_nv3_ctxt()` incorrectly identifies NV3 context when the guest clears `HCR_EL2.E2H`, diverging from hardware execution semantics.
> --
> 
> commit c75857355bb73ff60c1d0206705bb863784f8183
> Author: Marc Zyngier <maz@kernel.org>
> 
> KVM: arm64: Add kvm_has_nv{2,3}() predicates
> 
> Add a new set of predicates indicating whether VM is capable of
> NV2, NV3, and is in a nested NV3 context. This is going to become
> useful as we start dealing with a mix of behaviours.
> 
> > diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> > index 9831166695186..c562d8171d5e1 100644
> > --- a/arch/arm64/include/asm/kvm_emulate.h
> > +++ b/arch/arm64/include/asm/kvm_emulate.h
> 
> [ ... ]
> 
> > +static inline bool kvm_has_nv3(struct kvm *kvm)
> > +{
> > +	return (cpus_have_final_cap(ARM64_HAS_NESTED_VIRT) &&
> > +		cpus_have_final_cap(ARM64_HAS_NV3) &&
> > +		kvm_has_feat(kvm, ID_AA64MMFR4_EL1, NV_frac, NV3));
> > +}
> > +
> > +static inline bool is_nested_nv3_ctxt(struct kvm_vcpu *vcpu)
> > +{
> > +	return (has_vhe() && kvm_has_nv3(vcpu->kvm) && is_nested_ctxt(vcpu) &&
> > +		(__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_EL2_NV) &&
> > +		(__vcpu_sys_reg(vcpu, HCRX_EL2) & HCRX_EL2_NVTGE));
> > +}
> 
> [Severity: High]
> Does this predicate miss a mandatory check for HCR_EL2.E2H?
> 
> Architecturally, NVTGE is RES0 when E2H is 0. If a guest sets HCR_EL2.NV = 1
> and HCRX_EL2.NVTGE = 1, but leaves HCR_EL2.E2H = 0, the hardware will not
> execute the guest under NV3 semantics.

Stop inventing the architecture. NV3 is only applicable to E2H==1, and
KVM doesn't allow NV with E2H==0.

	M.

-- 
Jazz isn't dead. It just smells funny.

Re: [PATCH 22/28] KVM: arm64: Add routing for NVHCR_EL2 trap

[Marc Zyngier]

On Thu, 02 Jul 2026 17:26:07 +0100,
sashiko-bot@kernel.org wrote:
> 
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [High] The trap routing condition for `NVHCR_EL2` incorrectly uses a logical OR (`MCB`) instead of a logical AND, causing improper trap injection to guest EL2.
> --
> 
> commit 6d634fa5ff4e9b082f4e46e999fa4990ecf38957
> Author: Marc Zyngier <maz@kernel.org>
> 
> KVM: arm64: Add routing for NVHCR_EL2 trap
>     
> NVHCR_EL2 accesses from EL1 are taken to EL2 when HCRX_EL2.NVTGE==0
> and HCR_EL2.NV==1. Describe this in the exception routing tables.
> 
> > diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c
> > index b7f3d86a94031..f5dc578d8c985 100644
> > --- a/arch/arm64/kvm/emulate-nested.c
> > +++ b/arch/arm64/kvm/emulate-nested.c
> [ ... ]
> > @@ -468,6 +477,7 @@ static const enum cgt_group_id *coarse_control_combo[] = {
> >  					CGT_HCR_TVM, CGT_HCR_TRVM, CGT_HCRX_SCTLR2En),
> >  	MCB(CGT_HCR_TPU_TICAB,		CGT_HCR_TPU, CGT_HCR_TICAB),
> >  	MCB(CGT_HCR_TPU_TOCU,		CGT_HCR_TPU, CGT_HCR_TOCU),
> > +	MCB(CGT_HCR_NV_HCRX_nNVTGE,	CGT_HCR_NV, CGT_HCRX_nNVTGE),
> 
> [Severity: High]
> Does this correctly evaluate the trap condition as an AND?
> 
> The commit message indicates the trap is taken when "HCRX_EL2.NVTGE==0 and
> HCR_EL2.NV==1". However, coarse_control_combo[] uses the MCB() macro, which
> evaluates multiple conditions using a bitwise OR:
> 
> arch/arm64/kvm/emulate-nested.c:__compute_trap_behaviour() {
> 	...
> 		for (int i = 0; cgids[i] != __RESERVED__; i++)
> 			b |= __compute_trap_behaviour(vcpu, cgids[i], b);
> 	...
> }
> 
> Because this ORs the behaviors, the trap will be forwarded if HCR_EL2.NV == 1
> OR HCRX_EL2.NVTGE == 0 evaluates to forward. This would erroneously send the
> trap to EL2 even if normal execution without nested virt is active (NV == 0
> and NVTGE == 0).
> 
> Should this use a __COMPLEX_CONDITIONS__ callback instead to enforce the
> logical AND?

Grmbl. Yes, well spotted.

	M.

-- 
Jazz isn't dead. It just smells funny.

Re: [PATCH 05/28] KVM: arm64: Plumb HCRX_EL2.SRMASKEn in HCRX_EL2 sanitisation

[Marc Zyngier]

On Thu, 02 Jul 2026 17:28:15 +0100,
sashiko-bot@kernel.org wrote:
> 
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> 
> Pre-existing issues:
> - [High] Missing RES0 constraints in system register feature map for `HCRX_EL2` allows nested guests to set reserved hardware bits.
> --
> 
> commit 5fb0fde52515a7e2e03636fdc18f15c38a091ddd
> Author: Marc Zyngier <maz@kernel.org>
> 
> KVM: arm64: Plumb HCRX_EL2.SRMASKEn in HCRX_EL2 sanitisation
> 
> This patch adds the HCRX_EL2.SRMASKEn bit to the HCRX_EL2 feature map to
> enable FEAT_SRMASK for a guest. It also updates the system register
> definitions to reflect the new bit field.
> 
> > diff --git a/arch/arm64/kvm/config.c b/arch/arm64/kvm/config.c
> > --- a/arch/arm64/kvm/config.c
> > +++ b/arch/arm64/kvm/config.c
> > @@ -904,6 +904,7 @@ static const DECLARE_FEAT_MAP_FGT(hdfgwtr2_desc, hdfgwtr2_masks,
> >  
> >  
> >  static const struct reg_bits_to_feat_map hcrx_feat_map[] = {
> > +	NEEDS_FEAT(HCRX_EL2_SRMASKEn, FEAT_SRMASK),
> 
> [Severity: High]
> This isn't a bug introduced by this patch, but a pre-existing issue in
> hcrx_feat_map: does the omission of FORCE_RES0(HCRX_EL2_RES0) and
> FORCE_RES1(HCRX_EL2_RES1) allow a nested guest to set architecturally reserved
> hardware bits?

No, because any bit that is not allocated will be treated as RES0, and
the kernel will warn about this. FORCE_RES0() is only an additional
way to detect these things.

Worth doing, but not a real issue.

	M.

-- 
Jazz isn't dead. It just smells funny.

Re: [PATCH 11/28] KVM: arm64: Expose FEAT_NV2p1 to NV guests

[Marc Zyngier]

On Thu, 02 Jul 2026 17:28:30 +0100,
sashiko-bot@kernel.org wrote:
> 
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [High] Inconsistent preservation of `ID_AA64MMFR4_EL1` feature fields based on host support for `ARM64_HAS_NV2P1` (and later `ARM64_HAS_NV3`), leading to uncontrolled UAPI feature leaks and migration breakages.
> --
> 
> commit 11eacfaa8962f15232e1ae99c1e9f7d1dfee6846
> Author: Marc Zyngier <maz@kernel.org>
> 
> KVM: arm64: Expose FEAT_NV2p1 to NV guests
> 
> This commit exposes the FEAT_NV2p1 feature to nested virtualization guests.
> It does so by updating the ID_AA64MMFR4_EL1 feature register limits in
> limit_nv_id_reg() based on host support.
> 
> > diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
> > --- a/arch/arm64/kvm/nested.c
> > +++ b/arch/arm64/kvm/nested.c
> > @@ -1740,7 +1740,11 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)
> >  		if (test_bit(KVM_ARM_VCPU_HAS_EL2_E2H0, kvm->arch.vcpu_features)) {
> >  			val = 0;
> >  		} else {
> > -			val = SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY);
> > +			if (cpus_have_final_cap(ARM64_HAS_NV2P1))
> > +				val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64MMFR4_EL1, NV_frac, NV2P1);
> > +			else
> > +				val = SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY);
> 
> [Severity: High]
> Does this introduce inconsistent preservation of ID_AA64MMFR4_EL1 feature fields?
> 
> When the host supports NV2P1, KVM uses ID_REG_LIMIT_FIELD_ENUM(), which only
> modifies the NV_frac field and preserves the rest of the original val. This 
> means the guest will inherit other host-populated feature bits from
> ID_AA64MMFR4_EL1 (like ASID2, HACDBS, FGWTE3, etc.).
> 
> However, if the host is NV2-only, it falls back to the original assignment
> using SYS_FIELD_PREP_ENUM(), which effectively wipes out all other fields in
> val.
> 
> Could this uncontrolled feature leak cause guest-visible state divergence based
> on the underlying hardware and break live migration?
> 
> >  			val &= ~ID_AA64MMFR4_EL1_E2H0;
> >  			val |= SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, E2H0, NI_NV1);

Yeah, that's indeed wrong. Another one to fix.

	M.

-- 
Jazz isn't dead. It just smells funny.

Re: [PATCH 03/28] KVM: arm64: Merge guest's HCRX_EL2 using NV_HCRX_GUEST_EXCLUDE

[Marc Zyngier]

On Thu, 02 Jul 2026 17:34:11 +0100,
sashiko-bot@kernel.org wrote:
> 
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [High] Missing RES0 Sanitization / Unsafe Register Merge for HCRX_EL2
> --
> 
> commit a8dae7a323ff7e33355d5cb4c144294087a9bbfa
> Author: Marc Zyngier <maz@kernel.org>
> KVM: arm64: Merge guest's HCRX_EL2 using NV_HCRX_GUEST_EXCLUDE
> 
> This commit changes how the guest-provided HCRX_EL2 value is merged with
> the host's value. It replaces positive masking with an exclusion list
> of bits that the guest is not allowed to override.
> 
> > diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
> > index 4bf624a49591d..8e5f492f39086 100644
> > --- a/arch/arm64/kvm/hyp/include/hyp/switch.h
> > +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
> 
> [ ... ]
> 
> > @@ -350,8 +368,8 @@ static inline void __activate_traps_common(struct kvm_vcpu *vcpu)
> >  		u64 hcrx = vcpu->arch.hcrx_el2;
> >  		if (is_nested_ctxt(vcpu)) {
> >  			u64 val = __vcpu_sys_reg(vcpu, HCRX_EL2);
> > -			hcrx |= val & __HCRX_EL2_MASK;
> > -			hcrx &= ~(~val & __HCRX_EL2_nMASK);
> > +			hcrx |= (val & ~NV_HCRX_GUEST_EXCLUDE);
> > +			hcrx &= ~(~val & ~NV_HCRX_GUEST_EXCLUDE);
> 
> [Severity: High]
> Does this allow an untrusted L1 guest to write garbage to the
> architecturally RES0 bits of the physical HCRX_EL2 register?

A guest is by definition untrusted. Reading a VNCR-backed system
register implies sanitisation to enforce RESx bits.

>
> By switching from a positive mask to a negative exclusion mask, any bits
> not explicitly excluded are now merged into the host value.
> 
> If the guest modifies its VNCR page at the VNCR_HCRX_EL2 offset and sets
> RES0 bits, these bits would be passed through. Since hcrx_feat_map lacks
> FORCE_RES0(HCRX_EL2_RES0), KVM's register mask application in
> kvm_vcpu_apply_reg_masks() will fail to clear them before they reach this
> merge point in __activate_traps_common().

No, the RES0 bits are implicitly handled. FORCE_RESx() is exclusively
sugar coating here. Case in point:

root@say-please:/home/maz# grep HCRX /sys/kernel/debug/kvm/976-14/resx
        SYS_HCRX_EL2:	RES0:0000000005ffcfe7	RES1:0000000000000000

Go away.

	M.

-- 
Jazz isn't dead. It just smells funny.

Re: [PATCH 04/28] KVM: arm64: Drop __HCRX_EL2_* masks

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Unsanitized architectural RES0/RES1 bits in HCRX_EL2 allow nested guests to enable hidden hardware features and trigger CONSTRAINED UNPREDICTABLE behavior.
--

commit c528765403e37bb993d380c50856d508033ad43a
Author: Marc Zyngier <maz@kernel.org>

KVM: arm64: Drop __HCRX_EL2_* masks

This patch removes the legacy __HCRX_EL2_* masks and the manual RES1
sanitization in get_reg_fixed_bits(). It intends to rely on existing
auto-generated masks and boot-time checks to prevent unhandled bits from
being exploited.

> diff --git a/arch/arm64/kvm/config.c b/arch/arm64/kvm/config.c
> index 0622162b089e5..16d8148dc3f12 100644
> --- a/arch/arm64/kvm/config.c
> +++ b/arch/arm64/kvm/config.c
[ ... ]
> @@ -933,7 +933,7 @@ static const struct reg_bits_to_feat_map hcrx_feat_map[] = {
>  };
>  
>  
> -static const DECLARE_FEAT_MAP(hcrx_desc, __HCRX_EL2,
> +static const DECLARE_FEAT_MAP(hcrx_desc, HCRX_EL2,
>  			      hcrx_feat_map, FEAT_HCX);

[Severity: High]
This is a pre-existing issue, but hcrx_feat_map lacks FORCE_RES0 and
FORCE_RES1 declarations. Unlike other system register maps, this means
compute_reg_resx_bits() will omit the architectural RES0/RES1 bits from
the KVM masks.

[ ... ]
> @@ -1579,7 +1579,6 @@ struct resx get_reg_fixed_bits(struct kvm *kvm, enum vcpu_sysreg reg)
>  		break;
>  	case HCRX_EL2:
>  		resx = compute_reg_resx_bits(kvm, &hcrx_desc, 0, 0);
> -		resx.res1 |= __HCRX_EL2_RES1;
>  		break;

[Severity: High]
Does removing this manual mask leave the RES1 bits completely unsanitized?

Since hcrx_feat_map doesn't supply the architectural bits to
compute_reg_resx_bits(), a nested guest might be able to write arbitrary
values to the HCRX_EL2 VNCR page. 

When KVM reads this in __activate_traps_common(), it only masks
NV_HCRX_GUEST_EXCLUDE before writing directly to the physical SYS_HCRX_EL2:

arch/arm64/kvm/hyp/include/hyp/switch.h:__activate_traps_common() {
        ...
        u64 val = __vcpu_sys_reg(vcpu, HCRX_EL2);
        hcrx |= (val & ~NV_HCRX_GUEST_EXCLUDE);
        hcrx &= ~(~val & ~NV_HCRX_GUEST_EXCLUDE);
        ...
        write_sysreg_s(hcrx, SYS_HCRX_EL2);
}

Could this allow the guest to bypass hypervisor feature emulation or trigger
CONSTRAINED UNPREDICTABLE hardware states?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702160248.1377250-1-maz@kernel.org?part=4

Re: [PATCH 27/28] KVM: arm64: Expose FEAT_NV3 to guests

[Marc Zyngier]

On Thu, 02 Jul 2026 17:39:02 +0100,
sashiko-bot@kernel.org wrote:
> 
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> 
> Pre-existing issues:
> - [Critical] A maliciously or incorrectly configured vCPU can trigger a host NULL pointer dereference by requesting an ID register configuration where NV_frac = 0, causing KVM to skip VNCR TLB allocation while still trapping VNCR accesses.
> --
> 
> commit ca23c995d48d1c5c8370918930aa81e2eeb147ab
> Author: Marc Zyngier <maz@kernel.org>
> 
> KVM: arm64: Expose FEAT_NV3 to guests
> 
> Further enable FEAT_NV3 by making it visible to NV guests.
> 
> > diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
> > index c9bf04944f9cb..64c8fb82fadf6 100644
> > --- a/arch/arm64/kvm/nested.c
> > +++ b/arch/arm64/kvm/nested.c
> > @@ -1728,7 +1728,7 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)
> >  		 * You get EITHER
> >  		 *
> >  		 * - FEAT_VHE without FEAT_E2H0
> > -		 * - FEAT_NV limited to FEAT_NV2(p1)
> > +		 * - FEAT_NV limited to FEAT_NV2(p1)/NV3
> >  		 * - HCR_EL2.NV1 being RES0
> >  		 *
> >  		 * OR
> > @@ -1740,7 +1740,9 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)
> >  		if (test_bit(KVM_ARM_VCPU_HAS_EL2_E2H0, kvm->arch.vcpu_features)) {
> >  			val = 0;
> 
> [Severity: Critical]
> This is a pre-existing issue, but could this cause a host NULL pointer
> dereference?
> 
> If userspace configures a vCPU with KVM_ARM_VCPU_HAS_EL2_E2H0, this branch
> forces the entire ID_AA64MMFR4_EL1 register to 0. This includes the NV_frac
> field. Alternatively, userspace could explicitly set ID_AA64MMFR4_EL1.NV_frac
> to 0 via KVM's ID register UAPI.
> 
> When KVM initializes the nested vCPU, it calls kvm_vcpu_allocate_vncr_tlb()
> which checks NV_frac:
> 
> arch/arm64/kvm/nested.c:kvm_vcpu_allocate_vncr_tlb() {
>         if (!kvm_has_feat(vcpu->kvm, ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY))
>                 return 0;
>         ...
> }
> 
> Because NV_frac is 0, the check evaluates to false and the function silently
> returns success without allocating vcpu->arch.vncr_tlb.
> 
> Since vcpu_has_nv() only checks for KVM_ARM_VCPU_HAS_EL2, the vCPU is allowed
> to run. If the guest then accesses a VNCR-trapped register or sets HCR_EL2.NV,

Beeeeeeeep!!!! It looked so good until then...

"VNCR-trapped register" means nothing, as the whole point of VNCR is
to *not* trap, but to write to memory instead. So OK, the page isn't
there, and a write to a register targeting VNCR_EL2 could generate a
fault.

But if ID_AA64MMFR4_EL1.NV_frac==0, then HCR_EL2.NV is RES0:

	NEEDS_FEAT(HCR_EL2_AT		| /* Ignore the original FEAT_NV */
		   HCR_EL2_NV2		|
		   HCR_EL2_NV,
		   feat_nv2),

and feat_nv2 reads:

static bool feat_nv2(struct kvm *kvm)
{
	return ((kvm_has_feat(kvm, ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY) &&
		 kvm_has_feat_enum(kvm, ID_AA64MMFR2_EL1, NV, NI)) ||
		kvm_has_feat(kvm, ID_AA64MMFR2_EL1, NV, NV2));
}

which will return false in the proposed case, making HCR_EL2.NV firmly
RES0. Yes, there's a reason why I spent so much time on this crap.

Since HCR_EL2.NV is RES0, we never ERET to an L2 running at EL2, and
therefore there is never a case where we fault on a guest provided
VNCR page.

> the host exit handler calls kvm_translate_vncr() or kvm_map_l1_vncr(). Both
> functions unconditionally dereference the unallocated vncr_tlb:
> 
> arch/arm64/kvm/nested.c:kvm_translate_vncr() {
>         struct vncr_tlb *vt = vcpu->arch.vncr_tlb;
>         ...
>         vt->valid = false;
>         ...
> }
> 
> Will this lead to a kernel panic when unprivileged userspace configures a
> vCPU this way?

Don't think so.

	M.

-- 
Jazz isn't dead. It just smells funny.

Re: [PATCH 26/28] KVM: arm64: Add FEAT_NV3 detection

[Marc Zyngier]

On Thu, 02 Jul 2026 17:39:50 +0100,
sashiko-bot@kernel.org wrote:
> 
> Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
> - [High] Writes to the SYS_NVHCR_EL2 control system register are not followed by an immediate isb().
> - [High] Writes to CNTKCTL_EL1 and CNTHCTL_EL2 control registers using sysreg_clear_set() lack an immediate isb().
> --
> 
> commit b72de4ae5666d0aaa35b94915af5eab1152df83b
> Author: Marc Zyngier <maz@kernel.org>
> KVM: arm64: Add FEAT_NV3 detection
> 
> This commit adds the detection code for FEAT_NV3 to the cpufeature
> capabilities. This engages the FEAT_NV3 fast-path when the hardware
> supports it.
> 
> > diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> > index c9c124b0ccc8e..6ae1c816e2010 100644
> > --- a/arch/arm64/kernel/cpufeature.c
> > +++ b/arch/arm64/kernel/cpufeature.c
> > @@ -2627,6 +2627,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
> >  		.matches = has_cpuid_feature,
> >  		ARM64_CPUID_FIELDS(ID_AA64MMFR4_EL1, NV_frac, NV2P1)
> >  	},
> 
> [Severity: High]
> Could the system operate under a stale configuration when updating timer
> control registers based on the ARM64_HAS_NV2P1 capability?
> 
> In arch/arm64/kvm/arch_timer.c:timer_set_traps(), the control registers
> are modified without an immediate isb() following the update:
> 
> arch/arm64/kvm/arch_timer.c:timer_set_traps()
> 	if (!cpus_have_final_cap(ARM64_HAS_NV2P1))
> 		sysreg_clear_set(cnthctl_el2, clr, set);
> 	else
> 		sysreg_clear_set(cntkctl_el1, clr, set);
> 
> The sysreg_clear_set() macro expands to a read-modify-write sequence that
> does not include an isb() as the very next instruction. Is an explicit
> barrier required here to prevent undefined pipeline state?

No. And "undefined pipeline state" means fsck all.

> 
> > +	{
> > +		.desc = "FEAT_NV3",
> > +		.capability = ARM64_HAS_NV3,
> > +		.type = ARM64_CPUCAP_SYSTEM_FEATURE,
> > +		.matches = has_cpuid_feature,
> > +		ARM64_CPUID_FIELDS(ID_AA64MMFR4_EL1, NV_frac, NV3)
> > +	},
> 
> [Severity: High]
> Does enabling the ARM64_HAS_NV3 capability expose missing synchronization
> when writing the SYS_NVHCR_EL2 control register?
> 
> Looking at __compute_hcr() in arch/arm64/kvm/hyp/vhe/switch.c, the guest
> view is published to SYS_NVHCR_EL2 without an immediate isb():
> 
> arch/arm64/kvm/hyp/vhe/switch.c:__compute_hcr()
> 		if (cpus_have_final_cap(ARM64_HAS_NV3) && vcpu_el2_e2h_is_set(vcpu))
> 			write_sysreg_s(__vcpu_sys_reg(vcpu, HCR_EL2), SYS_NVHCR_EL2);
> 		else
> 			__vcpu_assign_sys_reg(vcpu, NVHCR_EL2, __vcpu_sys_reg(vcpu, HCR_EL2));
> 
> Similarly, the L2 view of HCR_EL2 is restored without an immediate isb()
> before returning:

ERET in the guest is a CSE. The end.

	M.

-- 
Jazz isn't dead. It just smells funny.

Re: [PATCH 23/28] KVM: arm64: Add NVHCR_EL2 context switching

[Marc Zyngier]

On Thu, 02 Jul 2026 17:43:09 +0100,
sashiko-bot@kernel.org wrote:
> 
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [High] Cross-VM Information Leak via unswitched SYS_NVHCR_EL2 physical register
> --
> 
> commit ce2ffdc8cef6935283480a976d01a70bb788a885
> Author: Marc Zyngier <maz@kernel.org>
> 
> KVM: arm64: Add NVHCR_EL2 context switching
>     
> This commit adds context switching for the NVHCR_EL2 register to manage
> the HCR_EL2 state of an EL1 guest. The switch is conditional on the guest
> using NV3.
> 
> > diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
> > --- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
> > +++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
> > @@ -172,6 +172,10 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
> >  
> >  	if (ctxt_has_sctlr2(ctxt))
> >  		ctxt_sys_reg(ctxt, SCTLR2_EL1) = read_sysreg_el1(SYS_SCTLR2);
> > +
> > +	/* Retrieve L2's HCR_EL2, and save it for future use */
> > +	if (is_nested_nv3_ctxt(ctxt_to_vcpu(ctxt)))
> > +		ctxt_sys_reg(ctxt, NVHCR_EL2) = read_sysreg_s(SYS_NVHCR_EL2);
> 
> [Severity: High]
> Could this leak information between VMs when the host hardware supports NV3
> but the guest doesn't have it enabled?
> 
> If the host hardware implements FEAT_NV3 and runs an L2 guest with physical
> HCR_EL2.NV=1, the hardware will redirect EL1 accesses of HCR_EL2 to the
> physical SYS_NVHCR_EL2 register.

No. If L1 doesn't have NV3 enabled (meaning that its view of
HCRX_EL2.NVTGE == 0), then HCRX_EL2.NVTGE ==0 on the host as well, as
per the propagation of HCRX_EL2 in __activate_traps_common().

As per the architecture, an HCR_EL2 access from EL1 with NVTGE==0
results in accessing the corresponding VNCR_EL2 slot. Not the
NVHCR_EL2 register.

Stop making things up, specially when you don't even have access to
the spec.

> Since is_nested_nv3_ctxt() evaluates to false when the guest lacks the
> virtual NV3 capability, __sysreg_save_el1_state() skips saving this register.
> Does this allow an untrusted L2 guest to read the SYS_NVHCR_EL2 state left
> behind by a previously running VM, bypassing software isolation?

No.

>
> >  }
> >  
> [ ... ]
> > @@ -285,6 +289,13 @@ static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt,
> >  
> >  	if (ctxt_has_sctlr2(ctxt))
> >  		write_sysreg_el1(ctxt_sys_reg(ctxt, SCTLR2_EL1), SYS_SCTLR2);
> > +
> > +	/*
> > +	 * Publish the L2 view of HCR_EL2 to the HW if L1 is using NV3.
> > +	 * Otherwise, the data is already in place in the L1's own VNCR.
> > +	 */
> > +	if (is_nested_nv3_ctxt(ctxt_to_vcpu(ctxt)))
> > +		write_sysreg_s(ctxt_sys_reg(ctxt, NVHCR_EL2), SYS_NVHCR_EL2);
> 
> [Severity: High]
> Similar to the save path above in __sysreg_save_el1_state(), should this
> context switch be based on the host's hardware capability rather than the
> guest's software configuration?
> 
> If the physical register isn't explicitly overwritten in
> __sysreg_restore_el1_state() because is_nested_nv3_ctxt() is false, the
> L2 guest could potentially read the stale configuration state of a
> completely different VM.

No.

	M.

-- 
Jazz isn't dead. It just smells funny.

Re: [PATCH 25/28] KVM: arm64: Engage NV3 TLBI trap elision

[Marc Zyngier]

On Thu, 02 Jul 2026 17:45:14 +0100,
sashiko-bot@kernel.org wrote:
> 
> Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
> 
> New issues:
> - [Critical] Enabling FEAT_NV3 TLBI elision (NVnTTLB) breaks KVM's nested S2 MMU caching, leaving stale L2 guest S1 TLB entries across cached physical VMIDs.
> - [High] TLBI trap elision for NV1 guests is nested inside a check requiring the guest to use VHE, rendering the optimization dead code.
> - [Medium] NV3 TLBI trap elision is incomplete because HCRX_EL2_NVnTTLBOS is omitted from the TLBI elision mask.
> 
> Pre-existing issues:
> - [High] vcpu_set_hcrx() statically evaluates the guest's dynamic HCR_EL2.E2H state at boot, permanently disabling HCRX_EL2_NVTGE even if the guest later enables VHE.
> --
> 
> commit e34c9c8b4a16d0026abc090c888c49e7f1c66d8a
> Author: Marc Zyngier <maz@kernel.org>
> 
> KVM: arm64: Engage NV3 TLBI trap elision
> 
> This commit enables FEAT_NV3 TLBI trap elision for guests that are not
> NV2-aware. It relaxes TLBI traps to only TGE=0 using HCRX_EL2 bits.
> 
> > diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> > index b32870a5e1236..d6f432b1558f5 100644
> > --- a/arch/arm64/include/asm/kvm_emulate.h
> > +++ b/arch/arm64/include/asm/kvm_emulate.h
> > @@ -715,6 +715,15 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu)
> >  		if (cpus_have_final_cap(ARM64_HAS_NV3) &&
> >  		    vcpu_has_nv(vcpu) && vcpu_el2_e2h_is_set(vcpu)) {
> 
> [Severity: High]
> This isn't a bug introduced by this patch, but because vcpu_set_hcrx is
> called once during kvm_calculate_traps at the first VCPU run, the guest
> has not yet executed and its virtual HCR_EL2.E2H is 0. 
> 
> Since vcpu_el2_e2h_is_set checks the dynamic HCR_EL2 register, does this
> evaluate to false and permanently disable HCRX_EL2_NVTGE even if the guest
> later sets HCR_EL2.E2H at runtime?

No. E2H is never configurable in KVM. You should know it by now.

> 
> [Severity: High]
> Does checking vcpu_el2_e2h_is_set here prevent the NV1 TLBI elision
> optimization from ever being reached?
>
> Since an NV1 guest does not have VHE enabled (E2H=0), this outer condition
> will evaluate to false, meaning the inner !kvm_has_nv2(vcpu->kvm) block
> added below will be skipped.

And that's on purpose.

FEAT_NV3 doesn't apply to E2H==0. Because frankly, it is about as
useful as a concrete parachute.

> 
> >  			vcpu->arch.hcrx_el2 |= HCRX_EL2_NVTGE;
> > +
> > +			/*
> > +			 * If the guest is NV2-capable, then we need to see
> > +			 * all the TLBIs, as configured in HCR_EL2.
> > +			 * Otherwise, relax the TLBI traps to only TGE=0.
> > +			 */
> > +			if (!kvm_has_nv2(vcpu->kvm))
> > +				vcpu->arch.hcrx_el2 |= (HCRX_EL2_NVnTTLB  |
> > +							HCRX_EL2_NVnTTLBIS);
> 
> [Severity: Medium]
> Does this mask omit the Outer Shareable S1 TLBI elision bit
> (HCRX_EL2_NVnTTLBOS)?

It does.

> 
> Without it, will Outer Shareable S1 TLBI instructions continue to
> trap to KVM, missing out on the intended performance optimization?
>

Yeah. Not that it matters, but might as well be consistent. But that
can't be unconditional (we rely on trapping all OS TLBIs if OS TLBIs
are not advertised to the guest.

> [Severity: Critical]
> Does enabling hardware TLBI elision here break KVM's nested S2 MMU
> caching?
> 
> KVM caches multiple S2 MMU contexts (with distinct physical VMIDs) for
> the same virtual VMID, and defers physical VMID switches until guest entry.
>

Not really, but hey, I'm getting tired of explaining stuff to a
machine that won't even read my emails.

> If a guest hypervisor writes to VTTBR_EL2 and executes an S1 TLBI natively,
> hardware will only invalidate the currently active physical VMID.
>

This depends on the value of NVHCR_EL2.TGE.

> Could this cause the hardware to invalidate the wrong context (the
> hypervisor's own S1 TLB instead of the L2 guest's) and miss other cached
> physical VMIDs sharing the virtual VMID, potentially leading to stale
> TLB entries?

If NVHCR_EL2.TGE == 1, then this will only invalidate L1's own TLBs,
as per the architecture (TGE controls whether TLBI S1E1* affects the
host or the guest). If L1 does the wrong thing, that's its problem,
not the host's, and this is no different from what already happens in
KVM or on bare metal.

So no, that doesn't cause anything wrong.

	M.

-- 
Jazz isn't dead. It just smells funny.

Re: [PATCH 04/28] KVM: arm64: Drop __HCRX_EL2_* masks

[Marc Zyngier]

On Thu, 02 Jul 2026 19:34:49 +0100,
sashiko-bot@kernel.org wrote:
> 
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> 
> Pre-existing issues:
> - [High] Unsanitized architectural RES0/RES1 bits in HCRX_EL2 allow nested guests to enable hidden hardware features and trigger CONSTRAINED UNPREDICTABLE behavior.
> --
> 
> commit c528765403e37bb993d380c50856d508033ad43a
> Author: Marc Zyngier <maz@kernel.org>
> 
> KVM: arm64: Drop __HCRX_EL2_* masks
> 
> This patch removes the legacy __HCRX_EL2_* masks and the manual RES1
> sanitization in get_reg_fixed_bits(). It intends to rely on existing
> auto-generated masks and boot-time checks to prevent unhandled bits from
> being exploited.
> 
> > diff --git a/arch/arm64/kvm/config.c b/arch/arm64/kvm/config.c
> > index 0622162b089e5..16d8148dc3f12 100644
> > --- a/arch/arm64/kvm/config.c
> > +++ b/arch/arm64/kvm/config.c
> [ ... ]
> > @@ -933,7 +933,7 @@ static const struct reg_bits_to_feat_map hcrx_feat_map[] = {
> >  };
> >  
> >  
> > -static const DECLARE_FEAT_MAP(hcrx_desc, __HCRX_EL2,
> > +static const DECLARE_FEAT_MAP(hcrx_desc, HCRX_EL2,
> >  			      hcrx_feat_map, FEAT_HCX);
> 
> [Severity: High]
> This is a pre-existing issue, but hcrx_feat_map lacks FORCE_RES0 and
> FORCE_RES1 declarations. Unlike other system register maps, this means
> compute_reg_resx_bits() will omit the architectural RES0/RES1 bits from
> the KVM masks.

No. Again.

	M.

-- 
Jazz isn't dead. It just smells funny.