* [PATCH v4 01/29] arm64: sysreg: Emit RESx/UNKN values for Mapping/Fields definitions
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
@ 2026-07-30 7:09 ` Marc Zyngier
2026-07-30 7:09 ` [PATCH v4 02/29] arm64: Update ID_AA64MMFR4_EL1 description to 2026-03 JSON release Marc Zyngier
` (27 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:09 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
The sysreg file is using the Mapping or Fields qualifiers to indicate
that a given encoding is only a mapping to a particular register (or
an instance of a more generic register definition).
As a result, we don't output any definition, and instead expect
the canonical definitions to be used.
This works rather well for individual fields, but creates problems
for macros that refer to more generic classes of bits such as RESx.
Relax the above rule by emitting the RESx and UNKN values for Mapping
and Fields qualifiers as well.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/tools/gen-sysreg.awk | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/tools/gen-sysreg.awk b/arch/arm64/tools/gen-sysreg.awk
index 86860ab672dc7..53844f5c5e525 100755
--- a/arch/arm64/tools/gen-sysreg.awk
+++ b/arch/arm64/tools/gen-sysreg.awk
@@ -228,7 +228,7 @@ $1 == "EndSysreg" && block_current() == "Sysreg" {
}
# Currently this is effectivey a comment, in future we may want to emit
-# defines for the fields.
+# defines for the fields. We do emit RESx and UNKN values in any case.
($1 == "Fields" || $1 == "Mapping") && block_current() == "Sysreg" {
expect_fields(2)
@@ -239,9 +239,9 @@ $1 == "EndSysreg" && block_current() == "Sysreg" {
print ""
next_bit = -1
- res0 = null
- res1 = null
- unkn = null
+ res0 = $2 "_RES0"
+ res1 = $2 "_RES1"
+ unkn = $2 "_UNKN"
next
}
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 02/29] arm64: Update ID_AA64MMFR4_EL1 description to 2026-03 JSON release
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
2026-07-30 7:09 ` [PATCH v4 01/29] arm64: sysreg: Emit RESx/UNKN values for Mapping/Fields definitions Marc Zyngier
@ 2026-07-30 7:09 ` Marc Zyngier
2026-07-30 9:48 ` Joey Gouly
2026-07-30 7:09 ` [PATCH v4 03/29] KVM: arm64: Merge guest's HCRX_EL2 using NV_HCRX_GUEST_EXCLUDE Marc Zyngier
` (26 subsequent siblings)
28 siblings, 1 reply; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:09 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
ID_AA64MMFR4_EL1 has gained a few fields and enum values in the past
few months, so resync its definition with the 2026-03 JSON release.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/tools/sysreg | 32 ++++++++++++++++++++++++++++----
1 file changed, 28 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/tools/sysreg b/arch/arm64/tools/sysreg
index 7cb61aca3797f..3171c87eb2447 100644
--- a/arch/arm64/tools/sysreg
+++ b/arch/arm64/tools/sysreg
@@ -2386,17 +2386,40 @@ EndEnum
EndSysreg
Sysreg ID_AA64MMFR4_EL1 3 0 0 7 4
-Res0 63:48
-UnsignedEnum 47:44 SRMASK
+UnsignedEnum 63:60 MTEFGT
+ 0b0000 NI
+ 0b0001 IMP
+EndEnum
+UnsignedEnum 59:56 SCRX
+ 0b0000 NI
+ 0b0001 IMP
+EndEnum
+UnsignedEnum 55:52 TEV
+ 0b0000 NI
+ 0b0001 IMP
+EndEnum
+UnsignedEnum 51:48 TPS
+ 0b0000 VAL_0000
+ 0b0001 VAL_0001
+ 0b0010 VAL_0010
+EndEnum
+UnsignedEnum 47:44 SRMASK
+ 0b0000 NI
+ 0b0001 IMP
+ 0b0010 SRMASK2
+EndEnum
+UnsignedEnum 43:40 TLBID
0b0000 NI
0b0001 IMP
EndEnum
-Res0 43:40
UnsignedEnum 39:36 E3DSE
0b0000 NI
0b0001 IMP
EndEnum
-Res0 35:32
+UnsignedEnum 35:32 EAESR
+ 0b0000 NI
+ 0b0001 IMP
+EndEnum
UnsignedEnum 31:28 RMEGDI
0b0000 NI
0b0001 IMP
@@ -2410,6 +2433,7 @@ UnsignedEnum 23:20 NV_frac
0b0000 NV_NV2
0b0001 NV2_ONLY
0b0010 NV2P1
+ 0b0011 NV3
EndEnum
UnsignedEnum 19:16 FGWTE3
0b0000 NI
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [PATCH v4 02/29] arm64: Update ID_AA64MMFR4_EL1 description to 2026-03 JSON release
2026-07-30 7:09 ` [PATCH v4 02/29] arm64: Update ID_AA64MMFR4_EL1 description to 2026-03 JSON release Marc Zyngier
@ 2026-07-30 9:48 ` Joey Gouly
0 siblings, 0 replies; 32+ messages in thread
From: Joey Gouly @ 2026-07-30 9:48 UTC (permalink / raw)
To: Marc Zyngier
Cc: kvmarm, linux-arm-kernel, kvm, Steffen Eiden, Suzuki K Poulose,
Oliver Upton, Zenghui Yu, Yao Yuan, Wei-Lin Chang,
Vladimir Murzin
On Thu, Jul 30, 2026 at 08:09:55AM +0100, Marc Zyngier wrote:
> ID_AA64MMFR4_EL1 has gained a few fields and enum values in the past
> few months, so resync its definition with the 2026-03 JSON release.
>
> Signed-off-by: Marc Zyngier <maz@kernel.org>
Checked the values.
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
> ---
> arch/arm64/tools/sysreg | 32 ++++++++++++++++++++++++++++----
> 1 file changed, 28 insertions(+), 4 deletions(-)
>
> diff --git a/arch/arm64/tools/sysreg b/arch/arm64/tools/sysreg
> index 7cb61aca3797f..3171c87eb2447 100644
> --- a/arch/arm64/tools/sysreg
> +++ b/arch/arm64/tools/sysreg
> @@ -2386,17 +2386,40 @@ EndEnum
> EndSysreg
>
> Sysreg ID_AA64MMFR4_EL1 3 0 0 7 4
> -Res0 63:48
> -UnsignedEnum 47:44 SRMASK
> +UnsignedEnum 63:60 MTEFGT
> + 0b0000 NI
> + 0b0001 IMP
> +EndEnum
> +UnsignedEnum 59:56 SCRX
> + 0b0000 NI
> + 0b0001 IMP
> +EndEnum
> +UnsignedEnum 55:52 TEV
> + 0b0000 NI
> + 0b0001 IMP
> +EndEnum
> +UnsignedEnum 51:48 TPS
> + 0b0000 VAL_0000
> + 0b0001 VAL_0001
> + 0b0010 VAL_0010
> +EndEnum
> +UnsignedEnum 47:44 SRMASK
> + 0b0000 NI
> + 0b0001 IMP
> + 0b0010 SRMASK2
> +EndEnum
> +UnsignedEnum 43:40 TLBID
> 0b0000 NI
> 0b0001 IMP
> EndEnum
> -Res0 43:40
> UnsignedEnum 39:36 E3DSE
> 0b0000 NI
> 0b0001 IMP
> EndEnum
> -Res0 35:32
> +UnsignedEnum 35:32 EAESR
> + 0b0000 NI
> + 0b0001 IMP
> +EndEnum
> UnsignedEnum 31:28 RMEGDI
> 0b0000 NI
> 0b0001 IMP
> @@ -2410,6 +2433,7 @@ UnsignedEnum 23:20 NV_frac
> 0b0000 NV_NV2
> 0b0001 NV2_ONLY
> 0b0010 NV2P1
> + 0b0011 NV3
> EndEnum
> UnsignedEnum 19:16 FGWTE3
> 0b0000 NI
> --
> 2.47.3
>
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH v4 03/29] KVM: arm64: Merge guest's HCRX_EL2 using NV_HCRX_GUEST_EXCLUDE
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
2026-07-30 7:09 ` [PATCH v4 01/29] arm64: sysreg: Emit RESx/UNKN values for Mapping/Fields definitions Marc Zyngier
2026-07-30 7:09 ` [PATCH v4 02/29] arm64: Update ID_AA64MMFR4_EL1 description to 2026-03 JSON release Marc Zyngier
@ 2026-07-30 7:09 ` Marc Zyngier
2026-07-30 9:42 ` Joey Gouly
2026-07-30 7:09 ` [PATCH v4 04/29] KVM: arm64: Drop __HCRX_EL2_* masks Marc Zyngier
` (25 subsequent siblings)
28 siblings, 1 reply; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:09 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
The way we merge the guest-provided HCRX_EL2 value with the host's
is bonkers. We try to make it look like the FGT registers by using
positive and negative polarities for traps, but most of these bits
are not strictly about trapping, as they actively change the way
some architectural state is managed.
It would be far better to deal with these bits like we do for
HCR_EL2, by enumerating the list of bits we don't allow the guest
to override. This is simplified by the fact that HCRX_EL2 only
affects EL1, and not EL2.
Re-jig the HCRX_EL2 handling with a macro that list the bits excluded
from the merge (TMEA, PTTWI, EnIDCP128).
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/include/hyp/switch.h | 22 ++++++++++++++++++++--
1 file changed, 20 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index 4bf624a49591d..8e5f492f39086 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -325,6 +325,24 @@ static inline void __deactivate_traps_mpam(void)
write_sysreg_s(MPAMHCR_HOST_FLAGS, SYS_MPAMHCR_EL2);
}
+/*
+ * Just like for HCR_EL2, we can't let the guest mess with some of the
+ * basics we rely on in HCRX_EL2. However, the major difference is that
+ * HCRX_EL2 only affects EL1, and never EL2 (sudden outburst of sanity, I
+ * guess). So it is always the guest inflicting it on its own guestx.
+ *
+ * Things we don't want to let the guest control are:
+ *
+ * - TMEA: That's for us to decide how an SEA is routed, not the guest.
+ *
+ * - PTTWI: Similarly, it is for us to decide whether Reduced Coherency for
+ * the PTW is a thing. It really isn't.
+ *
+ * - EnIDCP128: We don't allow IMPDEF sysregs -- full stop.
+ */
+#define NV_HCRX_GUEST_EXCLUDE (HCRX_EL2_TMEA | HCRX_EL2_PTTWI | \
+ HCRX_EL2_EnIDCP128)
+
static inline void __activate_traps_common(struct kvm_vcpu *vcpu)
{
struct kvm_cpu_context *hctxt = host_data_ptr(host_ctxt);
@@ -350,8 +368,8 @@ static inline void __activate_traps_common(struct kvm_vcpu *vcpu)
u64 hcrx = vcpu->arch.hcrx_el2;
if (is_nested_ctxt(vcpu)) {
u64 val = __vcpu_sys_reg(vcpu, HCRX_EL2);
- hcrx |= val & __HCRX_EL2_MASK;
- hcrx &= ~(~val & __HCRX_EL2_nMASK);
+ hcrx |= (val & ~NV_HCRX_GUEST_EXCLUDE);
+ hcrx &= ~(~val & ~NV_HCRX_GUEST_EXCLUDE);
}
ctxt_sys_reg(hctxt, HCRX_EL2) = read_sysreg_s(SYS_HCRX_EL2);
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [PATCH v4 03/29] KVM: arm64: Merge guest's HCRX_EL2 using NV_HCRX_GUEST_EXCLUDE
2026-07-30 7:09 ` [PATCH v4 03/29] KVM: arm64: Merge guest's HCRX_EL2 using NV_HCRX_GUEST_EXCLUDE Marc Zyngier
@ 2026-07-30 9:42 ` Joey Gouly
0 siblings, 0 replies; 32+ messages in thread
From: Joey Gouly @ 2026-07-30 9:42 UTC (permalink / raw)
To: Marc Zyngier
Cc: kvmarm, linux-arm-kernel, kvm, Steffen Eiden, Suzuki K Poulose,
Oliver Upton, Zenghui Yu, Yao Yuan, Wei-Lin Chang,
Vladimir Murzin
On Thu, Jul 30, 2026 at 08:09:56AM +0100, Marc Zyngier wrote:
> The way we merge the guest-provided HCRX_EL2 value with the host's
> is bonkers. We try to make it look like the FGT registers by using
> positive and negative polarities for traps, but most of these bits
> are not strictly about trapping, as they actively change the way
> some architectural state is managed.
>
> It would be far better to deal with these bits like we do for
> HCR_EL2, by enumerating the list of bits we don't allow the guest
> to override. This is simplified by the fact that HCRX_EL2 only
> affects EL1, and not EL2.
>
> Re-jig the HCRX_EL2 handling with a macro that list the bits excluded
> from the merge (TMEA, PTTWI, EnIDCP128).
>
> Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
> Signed-off-by: Marc Zyngier <maz@kernel.org>
> ---
> arch/arm64/kvm/hyp/include/hyp/switch.h | 22 ++++++++++++++++++++--
> 1 file changed, 20 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
> index 4bf624a49591d..8e5f492f39086 100644
> --- a/arch/arm64/kvm/hyp/include/hyp/switch.h
> +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
> @@ -325,6 +325,24 @@ static inline void __deactivate_traps_mpam(void)
> write_sysreg_s(MPAMHCR_HOST_FLAGS, SYS_MPAMHCR_EL2);
> }
>
> +/*
> + * Just like for HCR_EL2, we can't let the guest mess with some of the
> + * basics we rely on in HCRX_EL2. However, the major difference is that
> + * HCRX_EL2 only affects EL1, and never EL2 (sudden outburst of sanity, I
> + * guess). So it is always the guest inflicting it on its own guestx.
Unsure if "guestx" is a typo or a funny way of saying guest's HCRX_EL2!
> + *
> + * Things we don't want to let the guest control are:
> + *
> + * - TMEA: That's for us to decide how an SEA is routed, not the guest.
> + *
> + * - PTTWI: Similarly, it is for us to decide whether Reduced Coherency for
> + * the PTW is a thing. It really isn't.
> + *
> + * - EnIDCP128: We don't allow IMPDEF sysregs -- full stop.
> + */
> +#define NV_HCRX_GUEST_EXCLUDE (HCRX_EL2_TMEA | HCRX_EL2_PTTWI | \
> + HCRX_EL2_EnIDCP128)
> +
> static inline void __activate_traps_common(struct kvm_vcpu *vcpu)
> {
> struct kvm_cpu_context *hctxt = host_data_ptr(host_ctxt);
> @@ -350,8 +368,8 @@ static inline void __activate_traps_common(struct kvm_vcpu *vcpu)
> u64 hcrx = vcpu->arch.hcrx_el2;
> if (is_nested_ctxt(vcpu)) {
> u64 val = __vcpu_sys_reg(vcpu, HCRX_EL2);
> - hcrx |= val & __HCRX_EL2_MASK;
> - hcrx &= ~(~val & __HCRX_EL2_nMASK);
> + hcrx |= (val & ~NV_HCRX_GUEST_EXCLUDE);
> + hcrx &= ~(~val & ~NV_HCRX_GUEST_EXCLUDE);
> }
>
> ctxt_sys_reg(hctxt, HCRX_EL2) = read_sysreg_s(SYS_HCRX_EL2);
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH v4 04/29] KVM: arm64: Drop __HCRX_EL2_* masks
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (2 preceding siblings ...)
2026-07-30 7:09 ` [PATCH v4 03/29] KVM: arm64: Merge guest's HCRX_EL2 using NV_HCRX_GUEST_EXCLUDE Marc Zyngier
@ 2026-07-30 7:09 ` Marc Zyngier
2026-07-30 7:09 ` [PATCH v4 05/29] KVM: arm64: Plumb HCRX_EL2.SRMASKEn in HCRX_EL2 sanitisation Marc Zyngier
` (24 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:09 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
The __HCRX_EL2_* masks are a leftover from a time where we didn't
have much sanitisation for the system registers. Since we are now
in a better place, rely on the existing checks to detect unhandled
bits in HCRX_EL2.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_arm.h | 15 ---------------
arch/arm64/kvm/config.c | 5 +++--
arch/arm64/kvm/emulate-nested.c | 5 -----
3 files changed, 3 insertions(+), 22 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 3f9233b5a1308..f6cd851047947 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -287,21 +287,6 @@
GENMASK(19, 18) | \
GENMASK(15, 0))
-/*
- * Polarity masks for HCRX_EL2, limited to the bits that we know about
- * at this point in time. It doesn't mean that we actually *handle*
- * them, but that at least those that are not advertised to a guest
- * will be RES0 for that guest.
- */
-#define __HCRX_EL2_MASK (BIT_ULL(6))
-#define __HCRX_EL2_nMASK (GENMASK_ULL(24, 14) | \
- GENMASK_ULL(11, 7) | \
- GENMASK_ULL(5, 0))
-#define __HCRX_EL2_RES0 ~(__HCRX_EL2_nMASK | __HCRX_EL2_MASK)
-#define __HCRX_EL2_RES1 ~(__HCRX_EL2_nMASK | \
- __HCRX_EL2_MASK | \
- __HCRX_EL2_RES0)
-
/* Hyp Prefetch Fault Address Register (HPFAR/HDFAR) */
#define HPFAR_MASK (~UL(0xf))
/*
diff --git a/arch/arm64/kvm/config.c b/arch/arm64/kvm/config.c
index 0622162b089e5..96ddcefc71eb5 100644
--- a/arch/arm64/kvm/config.c
+++ b/arch/arm64/kvm/config.c
@@ -930,10 +930,12 @@ static const struct reg_bits_to_feat_map hcrx_feat_map[] = {
NEEDS_FEAT(HCRX_EL2_EnASR, FEAT_LS64_V),
NEEDS_FEAT(HCRX_EL2_EnALS, FEAT_LS64),
NEEDS_FEAT(HCRX_EL2_EnAS0, FEAT_LS64_ACCDATA),
+ FORCE_RES0(HCRX_EL2_RES0),
+ FORCE_RES1(HCRX_EL2_RES1),
};
-static const DECLARE_FEAT_MAP(hcrx_desc, __HCRX_EL2,
+static const DECLARE_FEAT_MAP(hcrx_desc, HCRX_EL2,
hcrx_feat_map, FEAT_HCX);
static const struct reg_bits_to_feat_map hcr_feat_map[] = {
@@ -1579,7 +1581,6 @@ struct resx get_reg_fixed_bits(struct kvm *kvm, enum vcpu_sysreg reg)
break;
case HCRX_EL2:
resx = compute_reg_resx_bits(kvm, &hcrx_desc, 0, 0);
- resx.res1 |= __HCRX_EL2_RES1;
break;
case HCR_EL2:
resx = compute_reg_resx_bits(kvm, &hcr_desc, 0, 0);
diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c
index 3c82f392845d1..b7f3d86a94031 100644
--- a/arch/arm64/kvm/emulate-nested.c
+++ b/arch/arm64/kvm/emulate-nested.c
@@ -2320,7 +2320,6 @@ int __init populate_nv_trap_config(void)
BUILD_BUG_ON(__NR_CGT_GROUP_IDS__ > BIT(TC_CGT_BITS));
BUILD_BUG_ON(__NR_FGT_GROUP_IDS__ > BIT(TC_FGT_BITS));
BUILD_BUG_ON(__NR_FG_FILTER_IDS__ > BIT(TC_FGF_BITS));
- BUILD_BUG_ON(__HCRX_EL2_MASK & __HCRX_EL2_nMASK);
for (int i = 0; i < ARRAY_SIZE(encoding_to_cgt); i++) {
const struct encoding_to_trap_config *cgt = &encoding_to_cgt[i];
@@ -2346,10 +2345,6 @@ int __init populate_nv_trap_config(void)
}
}
- if (__HCRX_EL2_RES0 != HCRX_EL2_RES0)
- kvm_info("Sanitised HCR_EL2_RES0 = %016llx, expecting %016llx\n",
- __HCRX_EL2_RES0, HCRX_EL2_RES0);
-
kvm_info("nv: %ld coarse grained trap handlers\n",
ARRAY_SIZE(encoding_to_cgt));
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 05/29] KVM: arm64: Plumb HCRX_EL2.SRMASKEn in HCRX_EL2 sanitisation
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (3 preceding siblings ...)
2026-07-30 7:09 ` [PATCH v4 04/29] KVM: arm64: Drop __HCRX_EL2_* masks Marc Zyngier
@ 2026-07-30 7:09 ` Marc Zyngier
2026-07-30 7:09 ` [PATCH v4 06/29] KVM: arm64: Classify CPTR_EL2 as a SR_LOC_SPECIAL register Marc Zyngier
` (23 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:09 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
HCRX_EL2.SRMASKEn is a new bit enabling FEAT_SRMASK for a guest.
We don't plan to support it any time soon, but it doesn't hurt to
actively document it, specially as we are going to add more bits
we actually care about.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/config.c | 1 +
arch/arm64/tools/sysreg | 4 +++-
2 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kvm/config.c b/arch/arm64/kvm/config.c
index 96ddcefc71eb5..c61d62b796023 100644
--- a/arch/arm64/kvm/config.c
+++ b/arch/arm64/kvm/config.c
@@ -904,6 +904,7 @@ static const DECLARE_FEAT_MAP_FGT(hdfgwtr2_desc, hdfgwtr2_masks,
static const struct reg_bits_to_feat_map hcrx_feat_map[] = {
+ NEEDS_FEAT(HCRX_EL2_SRMASKEn, FEAT_SRMASK),
NEEDS_FEAT(HCRX_EL2_PACMEn, feat_pauth_lr),
NEEDS_FEAT(HCRX_EL2_EnFPM, FEAT_FPMR),
NEEDS_FEAT(HCRX_EL2_GCSEn, FEAT_GCS),
diff --git a/arch/arm64/tools/sysreg b/arch/arm64/tools/sysreg
index 3171c87eb2447..28755f70f0bf0 100644
--- a/arch/arm64/tools/sysreg
+++ b/arch/arm64/tools/sysreg
@@ -4545,7 +4545,9 @@ Fields ZCR_ELx
EndSysreg
Sysreg HCRX_EL2 3 4 1 2 2
-Res0 63:25
+Res0 63:27
+Field 26 SRMASKEn
+Res0 25
Field 24 PACMEn
Field 23 EnFPM
Field 22 GCSEn
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 06/29] KVM: arm64: Classify CPTR_EL2 as a SR_LOC_SPECIAL register
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (4 preceding siblings ...)
2026-07-30 7:09 ` [PATCH v4 05/29] KVM: arm64: Plumb HCRX_EL2.SRMASKEn in HCRX_EL2 sanitisation Marc Zyngier
@ 2026-07-30 7:09 ` Marc Zyngier
2026-07-30 7:10 ` [PATCH v4 07/29] KVM: arm64: Don't evaluate HCR_EL2.NV nor HFGITR_EL2.ERET on ERET fast path Marc Zyngier
` (22 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:09 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
It may not be obvious unless you look at it closely, but CPTR_EL2
is treated very differently from other registers. It is one the
registers that, despite looking very similar between EL1 and EL2
when E2H==1, have RES0 bits that get in the way.
Make it clear that CPTR_EL2 is odd by classifying it as SR_LOC_SPECIAL,
just like CNTHCTL_EL2 (and for the same reasons). This makes it
possible to use vcpu_read_sys_reg() with it, and will be necessary
once we support FEAT_NV2P1.
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_emulate.h | 2 +-
arch/arm64/kvm/sys_regs.c | 20 ++++++++++++++++++--
2 files changed, 19 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index 5bf3d7e1d92c7..9831166695186 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -617,7 +617,7 @@ static __always_inline void kvm_incr_pc(struct kvm_vcpu *vcpu)
*/
static inline u64 vcpu_sanitised_cptr_el2(const struct kvm_vcpu *vcpu)
{
- u64 cptr = __vcpu_sys_reg(vcpu, CPTR_EL2);
+ u64 cptr = vcpu_read_sys_reg(vcpu, CPTR_EL2);
if (!vcpu_el2_e2h_is_set(vcpu))
cptr = translate_cptr_el2_to_cpacr_el1(cptr);
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 5d5c579d45790..6b47d936efb32 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -183,8 +183,6 @@ static void locate_register(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg,
switch (reg) {
MAPPED_EL2_SYSREG(SCTLR_EL2, SCTLR_EL1,
translate_sctlr_el2_to_sctlr_el1 );
- MAPPED_EL2_SYSREG(CPTR_EL2, CPACR_EL1,
- translate_cptr_el2_to_cpacr_el1 );
MAPPED_EL2_SYSREG(TTBR0_EL2, TTBR0_EL1,
translate_ttbr0_el2_to_ttbr0_el1 );
MAPPED_EL2_SYSREG(TTBR1_EL2, TTBR1_EL1, NULL );
@@ -210,6 +208,19 @@ static void locate_register(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg,
loc->loc = ((is_hyp_ctxt(vcpu) && vcpu_el2_e2h_is_set(vcpu)) ?
SR_LOC_SPECIAL : SR_LOC_MEMORY);
break;
+ case CPTR_EL2:
+ /*
+ * CPTR_EL2 is just as special, and needs a certain amount
+ * of handholding. It always lives in memory, due to being
+ * heavily trapped thanks to CPACR_EL1.TCPAC being RES0.
+ * FEAT_NV2p1 fixes this.
+ */
+ locate_mapped_el2_register(vcpu, CPTR_EL2, CPACR_EL1,
+ translate_cptr_el2_to_cpacr_el1,
+ loc);
+ if (is_hyp_ctxt(vcpu) && vcpu_el2_e2h_is_set(vcpu))
+ loc->loc = SR_LOC_SPECIAL;
+ break;
default:
loc->loc = locate_direct_register(vcpu, reg);
}
@@ -314,6 +325,8 @@ u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg)
val &= CNTKCTL_VALID_BITS;
val |= __vcpu_sys_reg(vcpu, reg) & ~CNTKCTL_VALID_BITS;
return val;
+ case CPTR_EL2:
+ return __vcpu_sys_reg(vcpu, reg);
default:
WARN_ON_ONCE(1);
}
@@ -359,6 +372,9 @@ void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, enum vcpu_sysreg reg)
*/
write_sysreg_el1(val, SYS_CNTKCTL);
break;
+ case CPTR_EL2:
+ write_sysreg_el1(val, SYS_CPACR);
+ break;
default:
WARN_ON_ONCE(1);
}
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 07/29] KVM: arm64: Don't evaluate HCR_EL2.NV nor HFGITR_EL2.ERET on ERET fast path
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (5 preceding siblings ...)
2026-07-30 7:09 ` [PATCH v4 06/29] KVM: arm64: Classify CPTR_EL2 as a SR_LOC_SPECIAL register Marc Zyngier
@ 2026-07-30 7:10 ` Marc Zyngier
2026-07-30 7:10 ` [PATCH v4 08/29] arm64: Add ARM64_HAS_NV2P1 capability Marc Zyngier
` (21 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:10 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
We currently avoid using the ERET fast path if the guest has HCR_EL2.NV
set. This is an odd check, as NV doesn't mean much if HCR_EL2.TGE==1.
Similarly, evaluating HFGITR_EL2.ERET makes little sense, as
this only applies to the nested context, while the ERET fast-path
is purely for the benefit of L1.
Replace these bizarre checks with is_nested_ctxt() which makes a lot
more sense: if we are running an L2, the ERET trap must go to L1.
Fixes: dd0717a998f77 ("KVM: arm64: nv: Fast-track 'InHost' exception returns")
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/vhe/switch.c | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index bbe9cebd3d9d5..684cebf23aa0e 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -344,13 +344,15 @@ static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code)
* if this is a VHE guest hypervisor returning to its own
* userspace, or the hypervisor performing a local exception
* return. No need to save/restore registers, no need to
- * switch S2 MMU. Just do the canonical ERET.
+ * switch S2 MMU. Just do the canonical ERET unless we are in
+ * nested context.
*
- * Unless the trap has to be forwarded further down the line,
- * of course...
+ * Note that this is made possible because KVM itself never traps
+ * ERET when running an L2. The consequence is that any ERET trap is
+ * the result of HCR_EL2 or HFGITR_EL2 programming by L1 for its own
+ * guest, and the exception must be forwarded to L1.
*/
- if ((__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_NV) ||
- (__vcpu_sys_reg(vcpu, HFGITR_EL2) & HFGITR_EL2_ERET))
+ if (is_nested_ctxt(vcpu))
return false;
spsr = read_sysreg_el1(SYS_SPSR);
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 08/29] arm64: Add ARM64_HAS_NV2P1 capability
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (6 preceding siblings ...)
2026-07-30 7:10 ` [PATCH v4 07/29] KVM: arm64: Don't evaluate HCR_EL2.NV nor HFGITR_EL2.ERET on ERET fast path Marc Zyngier
@ 2026-07-30 7:10 ` Marc Zyngier
2026-07-30 7:10 ` [PATCH v4 09/29] KVM: arm64: Relax CPTR_EL2 handling when FEAT_NV2p1 is present Marc Zyngier
` (20 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:10 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
As we're about to deal with FEAT_NV2P1, add a new capability that
will be used to key any support for it.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/tools/cpucaps | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps
index 9b85a84f6fd49..242dc211d8efa 100644
--- a/arch/arm64/tools/cpucaps
+++ b/arch/arm64/tools/cpucaps
@@ -51,6 +51,7 @@ HAS_LS64_V
HAS_LSUI
HAS_MOPS
HAS_NESTED_VIRT
+HAS_NV2P1
HAS_BBML2_NOABORT
HAS_PAN
HAS_PMUV3
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 09/29] KVM: arm64: Relax CPTR_EL2 handling when FEAT_NV2p1 is present
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (7 preceding siblings ...)
2026-07-30 7:10 ` [PATCH v4 08/29] arm64: Add ARM64_HAS_NV2P1 capability Marc Zyngier
@ 2026-07-30 7:10 ` Marc Zyngier
2026-07-30 7:10 ` [PATCH v4 10/29] KVM: arm64: Relax CNTHCTL_EL2 " Marc Zyngier
` (19 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:10 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
With FEAT_NV2P1, it is no longer necessary to trap CPTR_EL2 accesses
via CPACR_EL1, as CPACR_EL1.TCPAC is guaranteed to be stateful.
Prevent such trapping and context switch CPACTR_EL1 in NV contexts
when NV2P1 is present.
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/include/hyp/switch.h | 5 +++--
arch/arm64/kvm/hyp/vhe/switch.c | 3 +++
arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 8 +++++---
arch/arm64/kvm/sys_regs.c | 5 ++++-
4 files changed, 15 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index 8e5f492f39086..7b27296c94607 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -108,9 +108,10 @@ static inline void __activate_cptr_traps_vhe(struct kvm_vcpu *vcpu)
* The architecture is a bit crap (what a surprise): an EL2 guest
* writing to CPTR_EL2 via CPACR_EL1 can't set any of TCPAC or TTA,
* as they are RES0 in the guest's view. To work around it, trap the
- * sucker using the very same bit it can't set...
+ * sucker using the very same bit it can't set. FEAT_NV2p1 fixes it.
*/
- if (vcpu_el2_e2h_is_set(vcpu) && is_hyp_ctxt(vcpu))
+ if (!cpus_have_final_cap(ARM64_HAS_NV2P1) &&
+ vcpu_el2_e2h_is_set(vcpu) && is_hyp_ctxt(vcpu))
val |= CPTR_EL2_TCPAC;
/*
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 684cebf23aa0e..a8da40568a048 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -443,6 +443,9 @@ static bool kvm_hyp_handle_cpacr_el1(struct kvm_vcpu *vcpu, u64 *exit_code)
u64 esr = kvm_vcpu_get_esr(vcpu);
int rt;
+ if (cpus_have_final_cap(ARM64_HAS_NV2P1))
+ return false;
+
if (!is_hyp_ctxt(vcpu) || esr_sys64_to_sysreg(esr) != SYS_CPACR_EL1)
return false;
diff --git a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c
index be685b63e8cf2..6f0f046e4ca4e 100644
--- a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c
@@ -42,10 +42,12 @@ static void __sysreg_save_vel2_state(struct kvm_vcpu *vcpu)
u64 val;
/*
- * We don't save CPTR_EL2, as accesses to CPACR_EL1
- * are always trapped, ensuring that the in-memory
- * copy is always up-to-date. A small blessing...
+ * Without FEAT_NV2p1, we don't save CPTR_EL2, as accesses
+ * to CPACR_EL1 are always trapped, ensuring that the
+ * in-memory copy is always up-to-date. A small blessing...
*/
+ if (cpus_have_final_cap(ARM64_HAS_NV2P1))
+ __vcpu_assign_sys_reg(vcpu, CPTR_EL2, read_sysreg_el1(SYS_CPACR));
__vcpu_assign_sys_reg(vcpu, SCTLR_EL2, read_sysreg_el1(SYS_SCTLR));
__vcpu_assign_sys_reg(vcpu, TTBR0_EL2, read_sysreg_el1(SYS_TTBR0));
__vcpu_assign_sys_reg(vcpu, TTBR1_EL2, read_sysreg_el1(SYS_TTBR1));
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 6b47d936efb32..1dfc1f88bec82 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -326,7 +326,10 @@ u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg)
val |= __vcpu_sys_reg(vcpu, reg) & ~CNTKCTL_VALID_BITS;
return val;
case CPTR_EL2:
- return __vcpu_sys_reg(vcpu, reg);
+ if (cpus_have_final_cap(ARM64_HAS_NV2P1))
+ return read_sysreg_el1(SYS_CPACR);
+ else
+ return __vcpu_sys_reg(vcpu, reg);
default:
WARN_ON_ONCE(1);
}
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 10/29] KVM: arm64: Relax CNTHCTL_EL2 handling when FEAT_NV2p1 is present
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (8 preceding siblings ...)
2026-07-30 7:10 ` [PATCH v4 09/29] KVM: arm64: Relax CPTR_EL2 handling when FEAT_NV2p1 is present Marc Zyngier
@ 2026-07-30 7:10 ` Marc Zyngier
2026-07-30 7:10 ` [PATCH v4 11/29] KVM: arm64: Expose FEAT_NV2p1 to NV guests Marc Zyngier
` (18 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:10 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
With NV2p1, it is no longer necessary to use the split approach
where bits of CNTHCTL_EL2 cannot be accessed via CNTKCTL_EL1,
and we can treat the CNTKCTL_EL1 accessor as if it was "normal".
Key the special casing on FEAT_NV2P1 not being implemented.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/arch_timer.c | 10 ++++++++--
arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 13 ++++++++++---
arch/arm64/kvm/sys_regs.c | 6 ++++--
3 files changed, 22 insertions(+), 7 deletions(-)
diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
index 4155fe89b58a1..db60facad9f3c 100644
--- a/arch/arm64/kvm/arch_timer.c
+++ b/arch/arm64/kvm/arch_timer.c
@@ -876,8 +876,14 @@ static void timer_set_traps(struct kvm_vcpu *vcpu, struct timer_map *map)
assign_clear_set_bit(tvt02, CNTHCTL_EL1NVVCT, clr, set);
assign_clear_set_bit(tpt02, CNTHCTL_EL1NVPCT, clr, set);
- /* This only happens on VHE, so use the CNTHCTL_EL2 accessor. */
- sysreg_clear_set(cnthctl_el2, clr, set);
+ /*
+ * This only happens on VHE, so use the CNTHCTL_EL2 accessor, unless
+ * we are sure CNTKCTL_EL1 is completely stateful with FEAT_NV2p1.
+ */
+ if (!cpus_have_final_cap(ARM64_HAS_NV2P1))
+ sysreg_clear_set(cnthctl_el2, clr, set);
+ else
+ sysreg_clear_set(cntkctl_el1, clr, set);
}
void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu)
diff --git a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c
index 6f0f046e4ca4e..0c4ef1ce32ae7 100644
--- a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c
@@ -69,11 +69,18 @@ static void __sysreg_save_vel2_state(struct kvm_vcpu *vcpu)
* The EL1 view of CNTKCTL_EL1 has a bunch of RES0 bits where
* the interesting CNTHCTL_EL2 bits live. So preserve these
* bits when reading back the guest-visible value.
+ *
+ * While NV2p1 fixes some of that, it makes CNTHCTL_EL2.ECV
+ * even more broken than it already was with NV2.
*/
val = read_sysreg_el1(SYS_CNTKCTL);
- val &= CNTKCTL_VALID_BITS;
- __vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, &=, ~CNTKCTL_VALID_BITS);
- __vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, |=, val);
+ if (!cpus_have_final_cap(ARM64_HAS_NV2P1)) {
+ val &= CNTKCTL_VALID_BITS;
+ __vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, &=, ~CNTKCTL_VALID_BITS);
+ __vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, |=, val);
+ } else {
+ __vcpu_assign_sys_reg(vcpu, CNTHCTL_EL2, val);
+ }
}
__vcpu_assign_sys_reg(vcpu, SP_EL2, read_sysreg(sp_el1));
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 1dfc1f88bec82..9439c5b2b1fe8 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -322,8 +322,10 @@ u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg)
switch (reg) {
case CNTHCTL_EL2:
val = read_sysreg_el1(SYS_CNTKCTL);
- val &= CNTKCTL_VALID_BITS;
- val |= __vcpu_sys_reg(vcpu, reg) & ~CNTKCTL_VALID_BITS;
+ if (!cpus_have_final_cap(ARM64_HAS_NV2P1)) {
+ val &= CNTKCTL_VALID_BITS;
+ val |= __vcpu_sys_reg(vcpu, reg) & ~CNTKCTL_VALID_BITS;
+ }
return val;
case CPTR_EL2:
if (cpus_have_final_cap(ARM64_HAS_NV2P1))
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 11/29] KVM: arm64: Expose FEAT_NV2p1 to NV guests
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (9 preceding siblings ...)
2026-07-30 7:10 ` [PATCH v4 10/29] KVM: arm64: Relax CNTHCTL_EL2 " Marc Zyngier
@ 2026-07-30 7:10 ` Marc Zyngier
2026-07-30 7:10 ` [PATCH v4 12/29] arm64: Add FEAT_NV2p1 detection Marc Zyngier
` (17 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:10 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
Since NV2p1 is reducing the number of traps, it is valuable to expose
it to NV guests. Do so.
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/nested.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index dfb96edbdc43c..35a1c42ad3f84 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -1728,7 +1728,7 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)
* You get EITHER
*
* - FEAT_VHE without FEAT_E2H0
- * - FEAT_NV limited to FEAT_NV2
+ * - FEAT_NV limited to FEAT_NV2(p1)
* - HCR_EL2.NV1 being RES0
*
* OR
@@ -1740,7 +1740,11 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)
if (test_bit(KVM_ARM_VCPU_HAS_EL2_E2H0, kvm->arch.vcpu_features)) {
val = 0;
} else {
- val = SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY);
+ val &= ID_AA64MMFR4_EL1_NV_frac;
+ if (cpus_have_final_cap(ARM64_HAS_NV2P1))
+ val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64MMFR4_EL1, NV_frac, NV2P1);
+ else
+ val = SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY);
val |= SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, E2H0, NI_NV1);
}
break;
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 12/29] arm64: Add FEAT_NV2p1 detection
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (10 preceding siblings ...)
2026-07-30 7:10 ` [PATCH v4 11/29] KVM: arm64: Expose FEAT_NV2p1 to NV guests Marc Zyngier
@ 2026-07-30 7:10 ` Marc Zyngier
2026-07-30 7:10 ` [PATCH v4 13/29] arm64: sysreg: Add NVHCR_EL2 description as a mirror of HCR_EL2 Marc Zyngier
` (16 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:10 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
Add the necessary NV2p1 probing to the cpufeature infrastructure.
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kernel/cpufeature.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 9a22df0c5120f..c9c124b0ccc8e 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -2620,6 +2620,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{ /* Sentinel */ }
},
},
+ {
+ .desc = "FEAT_NV2p1",
+ .capability = ARM64_HAS_NV2P1,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
+ .matches = has_cpuid_feature,
+ ARM64_CPUID_FIELDS(ID_AA64MMFR4_EL1, NV_frac, NV2P1)
+ },
{
.capability = ARM64_HAS_32BIT_EL0_DO_NOT_USE,
.type = ARM64_CPUCAP_SYSTEM_FEATURE,
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 13/29] arm64: sysreg: Add NVHCR_EL2 description as a mirror of HCR_EL2
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (11 preceding siblings ...)
2026-07-30 7:10 ` [PATCH v4 12/29] arm64: Add FEAT_NV2p1 detection Marc Zyngier
@ 2026-07-30 7:10 ` Marc Zyngier
2026-07-30 7:10 ` [PATCH v4 14/29] arm64: sysreg: Add HCRX_EL2 bits related to FEAT_NV3 Marc Zyngier
` (15 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:10 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
FEAT_NV3 introduces a new register that contains the HCR_EL2 value
exposed to a NV guest. As such, it has the exact same layout as
HCR_EL2.
Describe NVHCR_EL2 as a mapping to HCR_EL2.
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/tools/sysreg | 3 +++
1 file changed, 3 insertions(+)
diff --git a/arch/arm64/tools/sysreg b/arch/arm64/tools/sysreg
index 28755f70f0bf0..42839d2b15ae4 100644
--- a/arch/arm64/tools/sysreg
+++ b/arch/arm64/tools/sysreg
@@ -4266,6 +4266,9 @@ Field 1 E2TRE
Field 0 E0HTRE
EndSysreg
+Sysreg NVHCR_EL2 3 4 1 5 0
+Mapping HCR_EL2
+EndSysreg
Sysreg HDFGRTR2_EL2 3 4 3 1 0
Res0 63:25
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 14/29] arm64: sysreg: Add HCRX_EL2 bits related to FEAT_NV3
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (12 preceding siblings ...)
2026-07-30 7:10 ` [PATCH v4 13/29] arm64: sysreg: Add NVHCR_EL2 description as a mirror of HCR_EL2 Marc Zyngier
@ 2026-07-30 7:10 ` Marc Zyngier
2026-07-30 7:10 ` [PATCH v4 15/29] arm64: Add ARM64_HAS_NV3 capability Marc Zyngier
` (14 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:10 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
FEAT_NV3 introduces 4 new HCRX_EL2 control bits. Describe them
in the sysreg file.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/tools/sysreg | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/tools/sysreg b/arch/arm64/tools/sysreg
index 42839d2b15ae4..67d013b376a3c 100644
--- a/arch/arm64/tools/sysreg
+++ b/arch/arm64/tools/sysreg
@@ -4548,7 +4548,12 @@ Fields ZCR_ELx
EndSysreg
Sysreg HCRX_EL2 3 4 1 2 2
-Res0 63:27
+Res0 63:35
+Field 34 NVnTTLBOS
+Field 33 NVnTTLBIS
+Field 32 NVnTTLB
+Res0 31:28
+Field 27 NVTGE
Field 26 SRMASKEn
Res0 25
Field 24 PACMEn
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 15/29] arm64: Add ARM64_HAS_NV3 capability
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (13 preceding siblings ...)
2026-07-30 7:10 ` [PATCH v4 14/29] arm64: sysreg: Add HCRX_EL2 bits related to FEAT_NV3 Marc Zyngier
@ 2026-07-30 7:10 ` Marc Zyngier
2026-07-30 7:10 ` [PATCH v4 16/29] KVM: arm64: Split NV-specific exit fixups from the non-NV handling Marc Zyngier
` (13 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:10 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
As a bunch of KVM code is going to depend on FEAT_NV3 being detected
on the host, add a new capability that will describe it.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/tools/cpucaps | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps
index 242dc211d8efa..7e0414509e89a 100644
--- a/arch/arm64/tools/cpucaps
+++ b/arch/arm64/tools/cpucaps
@@ -52,6 +52,7 @@ HAS_LSUI
HAS_MOPS
HAS_NESTED_VIRT
HAS_NV2P1
+HAS_NV3
HAS_BBML2_NOABORT
HAS_PAN
HAS_PMUV3
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 16/29] KVM: arm64: Split NV-specific exit fixups from the non-NV handling
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (14 preceding siblings ...)
2026-07-30 7:10 ` [PATCH v4 15/29] arm64: Add ARM64_HAS_NV3 capability Marc Zyngier
@ 2026-07-30 7:10 ` Marc Zyngier
2026-07-30 7:10 ` [PATCH v4 17/29] KVM: arm64: Add NV3 control bits to HCRX_EL2 sanitisation Marc Zyngier
` (12 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:10 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
In order to facilitate further changes, move the NV handling of
early fixups in its own helper. This also makes the code slightly
simpler to parse.
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/vhe/switch.c | 18 +++++++++++-------
1 file changed, 11 insertions(+), 7 deletions(-)
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index a8da40568a048..53af19212895d 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -539,18 +539,15 @@ static const exit_handler_fn hyp_exit_handlers[] = {
[0x3F] = kvm_hyp_handle_impdef,
};
-static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
+static void fixup_nv_guest_exit(struct kvm_vcpu *vcpu)
{
- synchronize_vcpu_pstate(vcpu);
-
/*
* If we were in HYP context on entry, adjust the PSTATE view
* so that the usual helpers work correctly. This enforces our
* invariant that the guest's HYP context status is preserved
* across a run.
*/
- if (vcpu_has_nv(vcpu) &&
- unlikely(host_data_test_flag(VCPU_IN_HYP_CONTEXT))) {
+ if (unlikely(host_data_test_flag(VCPU_IN_HYP_CONTEXT))) {
u64 mode = *vcpu_cpsr(vcpu) & (PSR_MODE_MASK | PSR_MODE32_BIT);
switch (mode) {
@@ -567,8 +564,15 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
}
/* Apply extreme paranoia! */
- BUG_ON(vcpu_has_nv(vcpu) &&
- !!host_data_test_flag(VCPU_IN_HYP_CONTEXT) != is_hyp_ctxt(vcpu));
+ BUG_ON(!!host_data_test_flag(VCPU_IN_HYP_CONTEXT) != is_hyp_ctxt(vcpu));
+}
+
+static bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
+{
+ synchronize_vcpu_pstate(vcpu);
+
+ if (vcpu_has_nv(vcpu))
+ fixup_nv_guest_exit(vcpu);
return __fixup_guest_exit(vcpu, exit_code, hyp_exit_handlers);
}
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 17/29] KVM: arm64: Add NV3 control bits to HCRX_EL2 sanitisation
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (15 preceding siblings ...)
2026-07-30 7:10 ` [PATCH v4 16/29] KVM: arm64: Split NV-specific exit fixups from the non-NV handling Marc Zyngier
@ 2026-07-30 7:10 ` Marc Zyngier
2026-07-30 7:10 ` [PATCH v4 18/29] KVM: arm64: Add kvm_has_nv{2,3}() predicates Marc Zyngier
` (11 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:10 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
Expose the FEAT_NV3 control bits to the sanitisation code so that
KVM stops moaning about the unattributed bits.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/config.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/arch/arm64/kvm/config.c b/arch/arm64/kvm/config.c
index c61d62b796023..467f57eb8e6bb 100644
--- a/arch/arm64/kvm/config.c
+++ b/arch/arm64/kvm/config.c
@@ -225,6 +225,7 @@ struct reg_feat_map_desc {
#define FEAT_HCX ID_AA64MMFR1_EL1, HCX, IMP
#define FEAT_S2PIE ID_AA64MMFR3_EL1, S2PIE, IMP
#define FEAT_GCIE ID_AA64PFR2_EL1, GCIE, IMP
+#define FEAT_NV3 ID_AA64MMFR4_EL1, NV_frac, NV3
static bool not_feat_aa64el3(struct kvm *kvm)
{
@@ -904,6 +905,11 @@ static const DECLARE_FEAT_MAP_FGT(hdfgwtr2_desc, hdfgwtr2_masks,
static const struct reg_bits_to_feat_map hcrx_feat_map[] = {
+ NEEDS_FEAT(HCRX_EL2_NVTGE |
+ HCRX_EL2_NVnTTLB |
+ HCRX_EL2_NVnTTLBIS |
+ HCRX_EL2_NVnTTLBOS,
+ FEAT_NV3),
NEEDS_FEAT(HCRX_EL2_SRMASKEn, FEAT_SRMASK),
NEEDS_FEAT(HCRX_EL2_PACMEn, feat_pauth_lr),
NEEDS_FEAT(HCRX_EL2_EnFPM, FEAT_FPMR),
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 18/29] KVM: arm64: Add kvm_has_nv{2,3}() predicates
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (16 preceding siblings ...)
2026-07-30 7:10 ` [PATCH v4 17/29] KVM: arm64: Add NV3 control bits to HCRX_EL2 sanitisation Marc Zyngier
@ 2026-07-30 7:10 ` Marc Zyngier
2026-07-30 7:10 ` [PATCH v4 19/29] KVM: arm64: Make HCR_EL2 a non-VNCR register Marc Zyngier
` (10 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:10 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
Add a new set of predicates indicating whether VM is capable of
NV2, NV3, and is in a nested NV3 context.
This is going to become useful as we start dealing with a mix of
behaviours (NV2, NV3, NV2 on NV3...).
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_emulate.h | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index 9831166695186..bdb7dfaa271cd 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -266,6 +266,25 @@ static inline bool vserror_state_is_nested(struct kvm_vcpu *vcpu)
(__vcpu_sys_reg(vcpu, HCRX_EL2) & HCRX_EL2_TMEA);
}
+static inline bool kvm_has_nv2(struct kvm *kvm)
+{
+ return (cpus_have_final_cap(ARM64_HAS_NESTED_VIRT) &&
+ kvm_has_feat(kvm, ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY));
+}
+
+static inline bool kvm_has_nv3(struct kvm *kvm)
+{
+ return (cpus_have_final_cap(ARM64_HAS_NV3) &&
+ kvm_has_feat(kvm, ID_AA64MMFR4_EL1, NV_frac, NV3));
+}
+
+static inline bool is_nested_nv3_ctxt(struct kvm_vcpu *vcpu)
+{
+ return (has_vhe() && kvm_has_nv3(vcpu->kvm) && is_nested_ctxt(vcpu) &&
+ (__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_EL2_NV) &&
+ (__vcpu_sys_reg(vcpu, HCRX_EL2) & HCRX_EL2_NVTGE));
+}
+
/*
* The layout of SPSR for an AArch32 state is different when observed from an
* AArch64 SPSR_ELx or an AArch32 SPSR_*. This function generates the AArch32
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 19/29] KVM: arm64: Make HCR_EL2 a non-VNCR register
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (17 preceding siblings ...)
2026-07-30 7:10 ` [PATCH v4 18/29] KVM: arm64: Add kvm_has_nv{2,3}() predicates Marc Zyngier
@ 2026-07-30 7:10 ` Marc Zyngier
2026-07-30 7:10 ` [PATCH v4 20/29] KVM: arm64: Add sanitisation for NVHCR_EL2 Marc Zyngier
` (9 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:10 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
FEAT_NV3 makes a fundamental change to the architecture, by moving
guest-initiated HCR_EL2 accesses to the NVHCR_EL2 register. As the
names suggests, this is HCR_EL2 for a NV guest.
But where do NVHCR_EL2 accesses from a guest go? The are redirected
to the VNCR page, right where HCR_EL2 is stored in the NV2 case.
Does it hurt? Good. There's more coming.
The challenge here is to make KVM work seamlessly, without rewriting
everything. Which implies that things such as __vcpu_sys_reg(HCR_EL2)
must work, no matter the underlying NV implementation.
A simple way to deal with it is to move HCR_EL2's canonical storage
outside of VNCR for the vast majority of the KVM code, and only have
a copy at entry/exit times. Given that we don't really support NV3
yet, this is pretty simple.
In the process, advertise NVHCR_EL2 as the register that now holds
offset 0x78 in the VNCR page.
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 3 ++-
arch/arm64/include/asm/vncr_mapping.h | 2 +-
arch/arm64/kvm/hyp/vhe/switch.c | 9 +++++++++
3 files changed, 12 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index bae2c4f92ef5c..2648c8a717ba0 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -543,6 +543,7 @@ enum vcpu_sysreg {
MDCR_EL2, /* Monitor Debug Configuration Register (EL2) */
CNTHCTL_EL2, /* Counter-timer Hypervisor Control register */
ZCR_EL2, /* SVE Control Register (EL2) */
+ HCR_EL2, /* Hypervisor Control Register */
/* Any VNCR-capable reg goes after this point */
MARKER(__VNCR_START__),
@@ -571,7 +572,7 @@ enum vcpu_sysreg {
VNCR(TFSR_EL1), /* Tag Fault Status Register (EL1) */
VNCR(VPIDR_EL2),/* Virtualization Processor ID Register */
VNCR(VMPIDR_EL2),/* Virtualization Multiprocessor ID Register */
- VNCR(HCR_EL2), /* Hypervisor Configuration Register */
+ VNCR(NVHCR_EL2),/* NV Hypervisor Configuration Register */
VNCR(HSTR_EL2), /* Hypervisor System Trap Register */
VNCR(VTTBR_EL2),/* Virtualization Translation Table Base Register */
VNCR(VTCR_EL2), /* Virtualization Translation Control Register */
diff --git a/arch/arm64/include/asm/vncr_mapping.h b/arch/arm64/include/asm/vncr_mapping.h
index 14366d35ce82f..9e8a49fa8b638 100644
--- a/arch/arm64/include/asm/vncr_mapping.h
+++ b/arch/arm64/include/asm/vncr_mapping.h
@@ -11,7 +11,7 @@
#define VNCR_VTCR_EL2 0x040
#define VNCR_VMPIDR_EL2 0x050
#define VNCR_CNTVOFF_EL2 0x060
-#define VNCR_HCR_EL2 0x078
+#define VNCR_NVHCR_EL2 0x078
#define VNCR_HSTR_EL2 0x080
#define VNCR_VPIDR_EL2 0x088
#define VNCR_TPIDR_EL2 0x090
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 53af19212895d..e5a51dc3cbe5e 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -70,6 +70,9 @@ static u64 __compute_hcr(struct kvm_vcpu *vcpu)
if (!vcpu_el2_e2h_is_set(vcpu))
hcr |= HCR_NV1;
+ /* Publish the guest's view of HCR_EL2 to the HW */
+ __vcpu_assign_sys_reg(vcpu, NVHCR_EL2, __vcpu_sys_reg(vcpu, HCR_EL2));
+
/*
* Nothing in HCR_EL2 should impact running in hypervisor
* context, apart from bits we have defined as RESx (E2H,
@@ -549,6 +552,7 @@ static void fixup_nv_guest_exit(struct kvm_vcpu *vcpu)
*/
if (unlikely(host_data_test_flag(VCPU_IN_HYP_CONTEXT))) {
u64 mode = *vcpu_cpsr(vcpu) & (PSR_MODE_MASK | PSR_MODE32_BIT);
+ u64 hcr;
switch (mode) {
case PSR_MODE_EL1t:
@@ -561,6 +565,11 @@ static void fixup_nv_guest_exit(struct kvm_vcpu *vcpu)
*vcpu_cpsr(vcpu) &= ~(PSR_MODE_MASK | PSR_MODE32_BIT);
*vcpu_cpsr(vcpu) |= mode;
+
+ /* Publish the latest HCR_EL2 to the emulation */
+ hcr = __vcpu_sys_reg(vcpu, NVHCR_EL2);
+
+ __vcpu_assign_sys_reg(vcpu, HCR_EL2, hcr);
}
/* Apply extreme paranoia! */
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 20/29] KVM: arm64: Add sanitisation for NVHCR_EL2
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (18 preceding siblings ...)
2026-07-30 7:10 ` [PATCH v4 19/29] KVM: arm64: Make HCR_EL2 a non-VNCR register Marc Zyngier
@ 2026-07-30 7:10 ` Marc Zyngier
2026-07-30 7:10 ` [PATCH v4 21/29] KVM: arm64: Add NVHCR_EL2 handling to the sysreg array Marc Zyngier
` (8 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:10 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
Just like any other VNCR-based register, NVHCR_EL2 requires some
level of sanitisation. Being specified as a live copy of HCR_EL2,
it adopts the exact same format, but depends on FEAT_NV3 instead.
A subtle aspect is that we only want to apply the sanitisation if
FEAT_NV3 is actually present, as the VNCR location is otherwise
used to back accesses to HCR_EL2.
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/config.c | 15 +++++++++++++++
arch/arm64/kvm/nested.c | 4 ++++
2 files changed, 19 insertions(+)
diff --git a/arch/arm64/kvm/config.c b/arch/arm64/kvm/config.c
index 467f57eb8e6bb..1053676551aff 100644
--- a/arch/arm64/kvm/config.c
+++ b/arch/arm64/kvm/config.c
@@ -1019,6 +1019,9 @@ static const struct reg_bits_to_feat_map hcr_feat_map[] = {
static const DECLARE_FEAT_MAP(hcr_desc, HCR_EL2,
hcr_feat_map, FEAT_AA64EL2);
+static const DECLARE_FEAT_MAP(nvhcr_desc, NVHCR_EL2,
+ hcr_feat_map, FEAT_NV3);
+
static const struct reg_bits_to_feat_map sctlr2_feat_map[] = {
NEEDS_FEAT(SCTLR2_EL1_NMEA |
SCTLR2_EL1_EASE,
@@ -1393,6 +1396,7 @@ void __init check_feature_map(void)
check_reg_desc(&hdfgwtr2_desc);
check_reg_desc(&hcrx_desc);
check_reg_desc(&hcr_desc);
+ check_reg_desc(&nvhcr_desc);
check_reg_desc(&sctlr2_desc);
check_reg_desc(&tcr2_el2_desc);
check_reg_desc(&sctlr_el1_desc);
@@ -1592,6 +1596,17 @@ struct resx get_reg_fixed_bits(struct kvm *kvm, enum vcpu_sysreg reg)
case HCR_EL2:
resx = compute_reg_resx_bits(kvm, &hcr_desc, 0, 0);
break;
+ case NVHCR_EL2:
+ /*
+ * Only apply sanitisation if we do have FEAT_NV3.
+ * Otherwise, the register aliases with HCR_EL2 in VNCR,
+ * and we're better off relying on data transfers between
+ * NVHCR_EL2 and HCR_EL2 to sanitise things.
+ */
+ resx = (kvm_has_nv3(kvm) ?
+ compute_reg_resx_bits(kvm, &nvhcr_desc, 0, 0) :
+ (typeof(resx)){});
+ break;
case SCTLR2_EL1:
case SCTLR2_EL2:
resx = compute_reg_resx_bits(kvm, &sctlr2_desc, 0, 0);
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 35a1c42ad3f84..f646894fec9a9 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -1830,6 +1830,10 @@ int kvm_init_nv_sysregs(struct kvm_vcpu *vcpu)
resx = get_reg_fixed_bits(kvm, HCR_EL2);
set_sysreg_masks(kvm, HCR_EL2, resx);
+ /* NVHCR_EL2 */
+ resx = get_reg_fixed_bits(kvm, NVHCR_EL2);
+ set_sysreg_masks(kvm, NVHCR_EL2, resx);
+
/* HCRX_EL2 */
resx = get_reg_fixed_bits(kvm, HCRX_EL2);
set_sysreg_masks(kvm, HCRX_EL2, resx);
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 21/29] KVM: arm64: Add NVHCR_EL2 handling to the sysreg array
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (19 preceding siblings ...)
2026-07-30 7:10 ` [PATCH v4 20/29] KVM: arm64: Add sanitisation for NVHCR_EL2 Marc Zyngier
@ 2026-07-30 7:10 ` Marc Zyngier
2026-07-30 7:10 ` [PATCH v4 22/29] KVM: arm64: Add routing for NVHCR_EL2 trap Marc Zyngier
` (7 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:10 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
Expose NVHCR_EL2 to userspace, and treat the direct access as UNDEF,
as that would only outline a bug in our exception routing.
The generic accessors are also updated to deal with the relatively
uncommon location of that register.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/sys_regs.c | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 9439c5b2b1fe8..0aeb2e736fde3 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -221,6 +221,20 @@ static void locate_register(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg,
if (is_hyp_ctxt(vcpu) && vcpu_el2_e2h_is_set(vcpu))
loc->loc = SR_LOC_SPECIAL;
break;
+ case NVHCR_EL2:
+ /*
+ * Yes, NVHCR_EL2 maps to itself when loaded in nested
+ * context. If you feel like the architecture is double
+ * backing on itself upside down, you're not alone.
+ */
+ WARN_ON_ONCE(!kvm_has_nv3(vcpu->kvm));
+ if (is_hyp_ctxt(vcpu)) {
+ loc->loc = SR_LOC_MEMORY;
+ } else {
+ loc->loc = SR_LOC_LOADED | SR_LOC_MAPPED;
+ loc->map_reg = NVHCR_EL2;
+ }
+ break;
default:
loc->loc = locate_direct_register(vcpu, reg);
}
@@ -260,6 +274,7 @@ static u64 read_sr_from_cpu(enum vcpu_sysreg reg)
case DACR32_EL2: val = read_sysreg_s(SYS_DACR32_EL2); break;
case IFSR32_EL2: val = read_sysreg_s(SYS_IFSR32_EL2); break;
case DBGVCR32_EL2: val = read_sysreg_s(SYS_DBGVCR32_EL2); break;
+ case NVHCR_EL2: val = read_sysreg_s(SYS_NVHCR_EL2); break;
default: WARN_ON_ONCE(1);
}
@@ -298,6 +313,7 @@ static void write_sr_to_cpu(enum vcpu_sysreg reg, u64 val)
case DACR32_EL2: write_sysreg_s(val, SYS_DACR32_EL2); break;
case IFSR32_EL2: write_sysreg_s(val, SYS_IFSR32_EL2); break;
case DBGVCR32_EL2: write_sysreg_s(val, SYS_DBGVCR32_EL2); break;
+ case NVHCR_EL2: write_sysreg_s(val, SYS_NVHCR_EL2); break;
default: WARN_ON_ONCE(1);
}
}
@@ -2861,6 +2877,16 @@ static unsigned int vncr_el2_visibility(const struct kvm_vcpu *vcpu,
return REG_HIDDEN;
}
+static unsigned int nvhcr_el2_visibility(const struct kvm_vcpu *vcpu,
+ const struct sys_reg_desc *rd)
+{
+ if (el2_visibility(vcpu, rd) == 0 &&
+ kvm_has_feat(vcpu->kvm, ID_AA64MMFR4_EL1, NV_frac, NV3))
+ return 0;
+
+ return REG_HIDDEN;
+}
+
static unsigned int sctlr2_visibility(const struct kvm_vcpu *vcpu,
const struct sys_reg_desc *rd)
{
@@ -3774,6 +3800,8 @@ static const struct sys_reg_desc sys_reg_descs[] = {
sve_el2_visibility),
EL2_REG_VNCR(HCRX_EL2, reset_val, 0),
+ EL2_REG_FILTERED(NVHCR_EL2, undef_access, reset_val, 0,
+ nvhcr_el2_visibility),
EL2_REG(TTBR0_EL2, access_rw, reset_val, 0),
EL2_REG(TTBR1_EL2, access_rw, reset_val, 0),
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 22/29] KVM: arm64: Add routing for NVHCR_EL2 trap
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (20 preceding siblings ...)
2026-07-30 7:10 ` [PATCH v4 21/29] KVM: arm64: Add NVHCR_EL2 handling to the sysreg array Marc Zyngier
@ 2026-07-30 7:10 ` Marc Zyngier
2026-07-30 7:10 ` [PATCH v4 23/29] KVM: arm64: Add NVHCR_EL2 context switching Marc Zyngier
` (6 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:10 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
NVHCR_EL2 accesses from EL1 are taken to EL2 when HCRX_EL2.NVTGE==0
and HCR_EL2.NV==1. Describe this in the exception routing tables.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/emulate-nested.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c
index b7f3d86a94031..625604019fb32 100644
--- a/arch/arm64/kvm/emulate-nested.c
+++ b/arch/arm64/kvm/emulate-nested.c
@@ -136,6 +136,8 @@ enum cgt_group_id {
CGT_CPTR_TTA,
CGT_MDCR_HPMN,
+ CGT_HCR_NV_HCRX_nNVTGE,
+
/* Must be last */
__NR_CGT_GROUP_IDS__
};
@@ -588,6 +590,15 @@ static enum trap_behaviour check_mdcr_hpmn(struct kvm_vcpu *vcpu)
return BEHAVE_HANDLE_LOCALLY;
}
+static enum trap_behaviour check_hcr_nv_hcrx_nnvtge(struct kvm_vcpu *vcpu)
+{
+ if ((__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_EL2_NV) &&
+ !(__vcpu_sys_reg(vcpu, HCRX_EL2) & HCRX_EL2_NVTGE))
+ return BEHAVE_FORWARD_RW;
+
+ return BEHAVE_HANDLE_LOCALLY;
+}
+
#define CCC(id, fn) \
[id - __COMPLEX_CONDITIONS__] = fn
@@ -598,6 +609,7 @@ static const complex_condition_check ccc[] = {
CCC(CGT_CNTHCTL_EL1NVVCT, check_cnthctl_el1nvvct),
CCC(CGT_CPTR_TTA, check_cptr_tta),
CCC(CGT_MDCR_HPMN, check_mdcr_hpmn),
+ CCC(CGT_HCR_NV_HCRX_nNVTGE, check_hcr_nv_hcrx_nnvtge),
};
/*
@@ -853,6 +865,7 @@ static const struct encoding_to_trap_config encoding_to_cgt[] __initconst = {
SR_TRAP(SYS_SCTLR2_EL2, CGT_HCR_NV),
SR_RANGE_TRAP(SYS_HCR_EL2,
SYS_HCRX_EL2, CGT_HCR_NV),
+ SR_TRAP(SYS_NVHCR_EL2, CGT_HCR_NV_HCRX_nNVTGE),
SR_TRAP(SYS_SMPRIMAP_EL2, CGT_HCR_NV),
SR_TRAP(SYS_SMCR_EL2, CGT_HCR_NV),
SR_RANGE_TRAP(SYS_TTBR0_EL2,
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 23/29] KVM: arm64: Add NVHCR_EL2 context switching
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (21 preceding siblings ...)
2026-07-30 7:10 ` [PATCH v4 22/29] KVM: arm64: Add routing for NVHCR_EL2 trap Marc Zyngier
@ 2026-07-30 7:10 ` Marc Zyngier
2026-07-30 7:10 ` [PATCH v4 24/29] KVM: arm64: Engage NV3 ERET trap elision Marc Zyngier
` (5 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:10 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
Since NVHCR_EL2 represents the HCR_EL2 state of the EL1 guest, it
must be dealt with in some particular way:
- for a guest in hyp context (an L1 by definition), NVHCR_EL2 directly
reflects HCR_EL2 as read and written by the guest itself. It must
therefore be eagerly synced back with the emulation code which only
knows about HCR_EL2. This is unconditional if NV3 is available on
the host.
- For an L2 guest, NVHCR_EL2 is controlled by the L1 guest, and we
just context switch it like any other EL1 register. Yes, EL1, as
that's where this thing runs from the PoV of L1. This is conditioned
on the guest using NV3.
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 11 +++++++++++
arch/arm64/kvm/hyp/vhe/switch.c | 10 ++++++++--
2 files changed, 19 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
index a17cbe7582de9..c382848d31947 100644
--- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
+++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
@@ -172,6 +172,10 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
if (ctxt_has_sctlr2(ctxt))
ctxt_sys_reg(ctxt, SCTLR2_EL1) = read_sysreg_el1(SYS_SCTLR2);
+
+ /* Retrieve L2's HCR_EL2, and save it for future use */
+ if (is_nested_nv3_ctxt(ctxt_to_vcpu(ctxt)))
+ ctxt_sys_reg(ctxt, NVHCR_EL2) = read_sysreg_s(SYS_NVHCR_EL2);
}
static inline void __sysreg_save_el2_return_state(struct kvm_cpu_context *ctxt)
@@ -285,6 +289,13 @@ static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt,
if (ctxt_has_sctlr2(ctxt))
write_sysreg_el1(ctxt_sys_reg(ctxt, SCTLR2_EL1), SYS_SCTLR2);
+
+ /*
+ * Publish the L2 view of HCR_EL2 to the HW if L1 is using NV3.
+ * Otherwise, the data is already in place in the L1's own VNCR.
+ */
+ if (is_nested_nv3_ctxt(ctxt_to_vcpu(ctxt)))
+ write_sysreg_s(ctxt_sys_reg(ctxt, NVHCR_EL2), SYS_NVHCR_EL2);
}
/* Read the VCPU state's PSTATE, but translate (v)EL2 to EL1. */
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index e5a51dc3cbe5e..cbda9e087db39 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -71,7 +71,10 @@ static u64 __compute_hcr(struct kvm_vcpu *vcpu)
hcr |= HCR_NV1;
/* Publish the guest's view of HCR_EL2 to the HW */
- __vcpu_assign_sys_reg(vcpu, NVHCR_EL2, __vcpu_sys_reg(vcpu, HCR_EL2));
+ if (cpus_have_final_cap(ARM64_HAS_NV3) && vcpu_el2_e2h_is_set(vcpu))
+ write_sysreg_s(__vcpu_sys_reg(vcpu, HCR_EL2), SYS_NVHCR_EL2);
+ else
+ __vcpu_assign_sys_reg(vcpu, NVHCR_EL2, __vcpu_sys_reg(vcpu, HCR_EL2));
/*
* Nothing in HCR_EL2 should impact running in hypervisor
@@ -567,7 +570,10 @@ static void fixup_nv_guest_exit(struct kvm_vcpu *vcpu)
*vcpu_cpsr(vcpu) |= mode;
/* Publish the latest HCR_EL2 to the emulation */
- hcr = __vcpu_sys_reg(vcpu, NVHCR_EL2);
+ hcr = (cpus_have_final_cap(ARM64_HAS_NV3) &&
+ vcpu_el2_e2h_is_set(vcpu)) ?
+ read_sysreg_s(SYS_NVHCR_EL2) :
+ __vcpu_sys_reg(vcpu, NVHCR_EL2);
__vcpu_assign_sys_reg(vcpu, HCR_EL2, hcr);
}
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 24/29] KVM: arm64: Engage NV3 ERET trap elision
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (22 preceding siblings ...)
2026-07-30 7:10 ` [PATCH v4 23/29] KVM: arm64: Add NVHCR_EL2 context switching Marc Zyngier
@ 2026-07-30 7:10 ` Marc Zyngier
2026-07-30 7:10 ` [PATCH v4 25/29] KVM: arm64: Engage NV3 TLBI " Marc Zyngier
` (4 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:10 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
When running on NV3 HW, always engage ERET trap elision when running
the L1 context, as there is no benefit in not doing so.
An L1 can itself engage trap elision by setting its own view of
HCRX_EL2.NVTGE==1, which will subsequently be honored.
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_emulate.h | 10 ++++++++++
arch/arm64/kvm/hyp/vhe/switch.c | 4 ++++
2 files changed, 14 insertions(+)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index bdb7dfaa271cd..ee340d3294478 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -705,6 +705,16 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu)
if (kvm_has_feat(kvm, ID_AA64ISAR1_EL1, LS64, LS64_V))
vcpu->arch.hcrx_el2 |= HCRX_EL2_EnASR;
+
+ /*
+ * NV3 is a host-specific extension, and we always use
+ * it when present and that the guest uses NV. It may
+ * be hidden from the guest though.
+ */
+ if (cpus_have_final_cap(ARM64_HAS_NV3) &&
+ vcpu_has_nv(vcpu) && vcpu_el2_e2h_is_set(vcpu)) {
+ vcpu->arch.hcrx_el2 |= HCRX_EL2_NVTGE;
+ }
}
}
#endif /* __ARM64_KVM_EMULATE_H__ */
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index cbda9e087db39..4ed745bdde418 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -345,6 +345,10 @@ static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code)
u64 esr = kvm_vcpu_get_esr(vcpu);
u64 spsr, elr, mode;
+ /* With NV3, the fast path is handled in HW */
+ if (cpus_have_final_cap(ARM64_HAS_NV3) && vcpu_el2_e2h_is_set(vcpu))
+ return false;
+
/*
* Going through the whole put/load motions is a waste of time
* if this is a VHE guest hypervisor returning to its own
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 25/29] KVM: arm64: Engage NV3 TLBI trap elision
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (23 preceding siblings ...)
2026-07-30 7:10 ` [PATCH v4 24/29] KVM: arm64: Engage NV3 ERET trap elision Marc Zyngier
@ 2026-07-30 7:10 ` Marc Zyngier
2026-07-30 7:10 ` [PATCH v4 26/29] KVM: arm64: Add FEAT_NV3 detection Marc Zyngier
` (3 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:10 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
Similarly to the ERET elision mechanism, FEAT_NV3 can elide TLBIs
that only affects the guest's S1 translation. Enable this, with the
express condition that the guest isn't NV2 aware, as we otherwise
need to trap these TLBIs to deal with VNCR mappings.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/kvm_emulate.h | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index ee340d3294478..67ad4523a1e67 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -714,6 +714,19 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu)
if (cpus_have_final_cap(ARM64_HAS_NV3) &&
vcpu_has_nv(vcpu) && vcpu_el2_e2h_is_set(vcpu)) {
vcpu->arch.hcrx_el2 |= HCRX_EL2_NVTGE;
+
+ /*
+ * If the guest is NV2-capable, then we need to see
+ * all the TLBIs, as configured in HCR_EL2.
+ * Otherwise, relax the TLBI traps to only TGE=0.
+ */
+ if (!kvm_has_nv2(vcpu->kvm)) {
+ vcpu->arch.hcrx_el2 |= (HCRX_EL2_NVnTTLB |
+ HCRX_EL2_NVnTTLBIS);
+
+ if (kvm_has_feat(kvm, ID_AA64ISAR0_EL1, TLB, OS))
+ vcpu->arch.hcrx_el2 |= HCRX_EL2_NVnTTLBOS;
+ }
}
}
}
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 26/29] KVM: arm64: Add FEAT_NV3 detection
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (24 preceding siblings ...)
2026-07-30 7:10 ` [PATCH v4 25/29] KVM: arm64: Engage NV3 TLBI " Marc Zyngier
@ 2026-07-30 7:10 ` Marc Zyngier
2026-07-30 7:10 ` [PATCH v4 27/29] KVM: arm64: Expose FEAT_NV3 to guests Marc Zyngier
` (2 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:10 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
Now that everything is in place to engage the FEAT_NV3 fast-path,
add the detection code to cpufeature.c.
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kernel/cpufeature.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index c9c124b0ccc8e..6ae1c816e2010 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -2627,6 +2627,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.matches = has_cpuid_feature,
ARM64_CPUID_FIELDS(ID_AA64MMFR4_EL1, NV_frac, NV2P1)
},
+ {
+ .desc = "FEAT_NV3",
+ .capability = ARM64_HAS_NV3,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
+ .matches = has_cpuid_feature,
+ ARM64_CPUID_FIELDS(ID_AA64MMFR4_EL1, NV_frac, NV3)
+ },
{
.capability = ARM64_HAS_32BIT_EL0_DO_NOT_USE,
.type = ARM64_CPUCAP_SYSTEM_FEATURE,
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 27/29] KVM: arm64: Expose FEAT_NV3 to guests
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (25 preceding siblings ...)
2026-07-30 7:10 ` [PATCH v4 26/29] KVM: arm64: Add FEAT_NV3 detection Marc Zyngier
@ 2026-07-30 7:10 ` Marc Zyngier
2026-07-30 7:10 ` [PATCH v4 28/29] KVM: arm64: selftest: Add NVHCR_EL2 to get-reg-list Marc Zyngier
2026-07-30 7:10 ` [PATCH v4 29/29] arm64: Add override for ID_AA64MMFR4_EL1.NV_frac Marc Zyngier
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:10 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
Further enable FEAT_NV3 by making it visible to NV guests.
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/nested.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index f646894fec9a9..70d605e60cfb3 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -1728,7 +1728,7 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)
* You get EITHER
*
* - FEAT_VHE without FEAT_E2H0
- * - FEAT_NV limited to FEAT_NV2(p1)
+ * - FEAT_NV limited to FEAT_NV2(p1)/NV3
* - HCR_EL2.NV1 being RES0
*
* OR
@@ -1741,7 +1741,9 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)
val = 0;
} else {
val &= ID_AA64MMFR4_EL1_NV_frac;
- if (cpus_have_final_cap(ARM64_HAS_NV2P1))
+ if (cpus_have_final_cap(ARM64_HAS_NV3))
+ val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64MMFR4_EL1, NV_frac, NV3);
+ else if (cpus_have_final_cap(ARM64_HAS_NV2P1))
val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64MMFR4_EL1, NV_frac, NV2P1);
else
val = SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY);
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 28/29] KVM: arm64: selftest: Add NVHCR_EL2 to get-reg-list
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (26 preceding siblings ...)
2026-07-30 7:10 ` [PATCH v4 27/29] KVM: arm64: Expose FEAT_NV3 to guests Marc Zyngier
@ 2026-07-30 7:10 ` Marc Zyngier
2026-07-30 7:10 ` [PATCH v4 29/29] arm64: Add override for ID_AA64MMFR4_EL1.NV_frac Marc Zyngier
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:10 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
Since we now expose a new system register on NV3-capable HW when
the guest is exposed to it, add it to the existing checker. FWIW.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
tools/testing/selftests/kvm/arm64/get-reg-list.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/tools/testing/selftests/kvm/arm64/get-reg-list.c b/tools/testing/selftests/kvm/arm64/get-reg-list.c
index 0a3a94c4cca16..3cecc6d5dbd51 100644
--- a/tools/testing/selftests/kvm/arm64/get-reg-list.c
+++ b/tools/testing/selftests/kvm/arm64/get-reg-list.c
@@ -67,6 +67,7 @@ static struct feature_id_reg feat_id_regs[] = {
REG_FEAT(VDISR_EL2, ID_AA64PFR0_EL1, RAS, IMP),
REG_FEAT(VSESR_EL2, ID_AA64PFR0_EL1, RAS, IMP),
REG_FEAT(VNCR_EL2, ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY),
+ REG_FEAT(NVHCR_EL2, ID_AA64MMFR4_EL1, NV_frac, NV3),
REG_FEAT(CNTHV_CTL_EL2, ID_AA64MMFR1_EL1, VH, IMP),
REG_FEAT(CNTHV_CVAL_EL2,ID_AA64MMFR1_EL1, VH, IMP),
REG_FEAT(ZCR_EL2, ID_AA64PFR0_EL1, SVE, IMP),
@@ -770,6 +771,7 @@ static __u64 el2_regs[] = {
SYS_REG(SP_EL2),
SYS_REG(VDISR_EL2),
SYS_REG(VSESR_EL2),
+ SYS_REG(NVHCR_EL2),
};
static __u64 el2_e2h0_regs[] = {
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 29/29] arm64: Add override for ID_AA64MMFR4_EL1.NV_frac
2026-07-30 7:09 [PATCH v4 00/29] KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3 Marc Zyngier
` (27 preceding siblings ...)
2026-07-30 7:10 ` [PATCH v4 28/29] KVM: arm64: selftest: Add NVHCR_EL2 to get-reg-list Marc Zyngier
@ 2026-07-30 7:10 ` Marc Zyngier
28 siblings, 0 replies; 32+ messages in thread
From: Marc Zyngier @ 2026-07-30 7:10 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel, kvm
Cc: Steffen Eiden, Joey Gouly, Suzuki K Poulose, Oliver Upton,
Zenghui Yu, Yao Yuan, Wei-Lin Chang, Vladimir Murzin
In a very unsurprising turn of events, there is a large class of
firmware that is totally unable to deal with FEAT_NV3, and doesn't
set the required SCR2_EL3.NV3En bit, leading to an UNDEF exception
or an unhandled trap to EL3, depending on the implementation.
Allow the unfortunate user to override ID_AA64MMFR4_EL1.NV_frac
and get a working system. Hopefully firmware will be fixed before
actually HW ships, but I have been there before... :-/
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/cpufeature.h | 1 +
arch/arm64/kernel/cpufeature.c | 4 +++-
arch/arm64/kernel/image-vars.h | 1 +
arch/arm64/kernel/pi/idreg-override.c | 10 ++++++++++
4 files changed, 15 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index a57870fa96db5..a42683af79fb5 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -968,6 +968,7 @@ struct arm64_ftr_reg *get_arm64_ftr_reg(u32 sys_id);
extern struct arm64_ftr_override id_aa64mmfr0_override;
extern struct arm64_ftr_override id_aa64mmfr1_override;
extern struct arm64_ftr_override id_aa64mmfr2_override;
+extern struct arm64_ftr_override id_aa64mmfr4_override;
extern struct arm64_ftr_override id_aa64pfr0_override;
extern struct arm64_ftr_override id_aa64pfr1_override;
extern struct arm64_ftr_override id_aa64zfr0_override;
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 6ae1c816e2010..14fbfa8e6b7b5 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -785,6 +785,7 @@ static const struct arm64_ftr_bits ftr_raz[] = {
struct arm64_ftr_override __read_mostly id_aa64mmfr0_override;
struct arm64_ftr_override __read_mostly id_aa64mmfr1_override;
struct arm64_ftr_override __read_mostly id_aa64mmfr2_override;
+struct arm64_ftr_override __read_mostly id_aa64mmfr4_override;
struct arm64_ftr_override __read_mostly id_aa64pfr0_override;
struct arm64_ftr_override __read_mostly id_aa64pfr1_override;
struct arm64_ftr_override __read_mostly id_aa64zfr0_override;
@@ -858,7 +859,8 @@ static const struct __ftr_reg_entry {
ARM64_FTR_REG_OVERRIDE(SYS_ID_AA64MMFR2_EL1, ftr_id_aa64mmfr2,
&id_aa64mmfr2_override),
ARM64_FTR_REG(SYS_ID_AA64MMFR3_EL1, ftr_id_aa64mmfr3),
- ARM64_FTR_REG(SYS_ID_AA64MMFR4_EL1, ftr_id_aa64mmfr4),
+ ARM64_FTR_REG_OVERRIDE(SYS_ID_AA64MMFR4_EL1, ftr_id_aa64mmfr4,
+ &id_aa64mmfr4_override),
/* Op1 = 0, CRn = 10, CRm = 4 */
ARM64_FTR_REG(SYS_MPAMIDR_EL1, ftr_mpamidr),
diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h
index d4c7d45ae6bc8..d15c2cb1b0f28 100644
--- a/arch/arm64/kernel/image-vars.h
+++ b/arch/arm64/kernel/image-vars.h
@@ -51,6 +51,7 @@ PI_EXPORT_SYM(id_aa64isar2_override);
PI_EXPORT_SYM(id_aa64mmfr0_override);
PI_EXPORT_SYM(id_aa64mmfr1_override);
PI_EXPORT_SYM(id_aa64mmfr2_override);
+PI_EXPORT_SYM(id_aa64mmfr4_override);
PI_EXPORT_SYM(id_aa64pfr0_override);
PI_EXPORT_SYM(id_aa64pfr1_override);
PI_EXPORT_SYM(id_aa64smfr0_override);
diff --git a/arch/arm64/kernel/pi/idreg-override.c b/arch/arm64/kernel/pi/idreg-override.c
index bc57b290e5e7b..4e47616bcac23 100644
--- a/arch/arm64/kernel/pi/idreg-override.c
+++ b/arch/arm64/kernel/pi/idreg-override.c
@@ -106,6 +106,15 @@ static const struct ftr_set_desc mmfr2 __prel64_initconst = {
},
};
+static const struct ftr_set_desc mmfr4 __prel64_initconst = {
+ .name = "id_aa64mmfr4",
+ .override = &id_aa64mmfr4_override,
+ .fields = {
+ FIELD("nv_frac", ID_AA64MMFR4_EL1_NV_frac_SHIFT, NULL),
+ {}
+ },
+};
+
static bool __init pfr0_sve_filter(u64 val)
{
/*
@@ -220,6 +229,7 @@ PREL64(const struct ftr_set_desc, reg) regs[] __prel64_initconst = {
{ &mmfr0 },
{ &mmfr1 },
{ &mmfr2 },
+ { &mmfr4 },
{ &pfr0 },
{ &pfr1 },
{ &isar1 },
--
2.47.3
^ permalink raw reply related [flat|nested] 32+ messages in thread