linux-arm-kernel.lists.infradead.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 0/3] KVM: arm64: Work around Ampere1 erratum AC03_CPU_38
@ 2023-06-09 22:01 Oliver Upton
  2023-06-09 22:01 ` [PATCH 1/3] arm64: errata: Mitigate Ampere1 erratum AC03_CPU_38 at stage-2 Oliver Upton
                   ` (5 more replies)
  0 siblings, 6 replies; 11+ messages in thread
From: Oliver Upton @ 2023-06-09 22:01 UTC (permalink / raw)
  To: kvmarm
  Cc: Marc Zyngier, James Morse, Suzuki K Poulose, Zenghui Yu,
	Will Deacon, Catalin Marinas, linux-arm-kernel, Darren Hart,
	D Scott Phillips, Oliver Upton

Hi folks,

Small series to work around a CPU erratum on AmpereOne. While the
implementation does not advertise support for FEAT_HAFDBS (due to
another erratum), the associated control bits do not have RES0 behavior
as required by the architecture.

Usage of HAFDBS at stage-1 is unaffected, since HA and HD are only
enabled on implementations that advertise the feature. However, KVM
relies on HA having RES0 semantics if the feature isn't implemented. The
end result is that KVM enables a broken hardware access flag
implementation that could lead to correctness issues.

Applies to 6.4-rc1. Tested with access_tracking_perf_test, verifying
that KVM is indeed taking Access Flag faults.

Oliver Upton (3):
  arm64: errata: Mitigate Ampere1 erratum AC03_CPU_38 at stage-2
  KVM: arm64: Refactor HFGxTR configuration into separate helpers
  KVM: arm64: Prevent guests from enabling HA/HD on Ampere1

 Documentation/arm64/silicon-errata.rst  |  3 +
 arch/arm64/Kconfig                      | 17 +++++
 arch/arm64/kernel/cpu_errata.c          |  7 ++
 arch/arm64/kvm/hyp/include/hyp/switch.h | 99 ++++++++++++++++++++-----
 arch/arm64/kvm/hyp/pgtable.c            | 14 +++-
 arch/arm64/tools/cpucaps                |  1 +
 6 files changed, 120 insertions(+), 21 deletions(-)


base-commit: ac9a78681b921877518763ba0e89202254349d1b
-- 
2.41.0.162.gfafddb0af9-goog


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH 1/3] arm64: errata: Mitigate Ampere1 erratum AC03_CPU_38 at stage-2
  2023-06-09 22:01 [PATCH 0/3] KVM: arm64: Work around Ampere1 erratum AC03_CPU_38 Oliver Upton
@ 2023-06-09 22:01 ` Oliver Upton
  2023-06-14 16:58   ` Catalin Marinas
  2023-06-14 17:15   ` D Scott Phillips
  2023-06-09 22:01 ` [PATCH 2/3] KVM: arm64: Refactor HFGxTR configuration into separate helpers Oliver Upton
                   ` (4 subsequent siblings)
  5 siblings, 2 replies; 11+ messages in thread
From: Oliver Upton @ 2023-06-09 22:01 UTC (permalink / raw)
  To: kvmarm
  Cc: Marc Zyngier, James Morse, Suzuki K Poulose, Zenghui Yu,
	Will Deacon, Catalin Marinas, linux-arm-kernel, Darren Hart,
	D Scott Phillips, Oliver Upton, stable

AmpereOne has an erratum in its implementation of FEAT_HAFDBS that
required disabling the feature on the design. This was done by reporting
the feature as not implemented in the ID register, although the
corresponding control bits were not actually RES0. This does not align
well with the requirements of the architecture, which mandates these
bits be RES0 if HAFDBS isn't implemented.

The kernel's use of stage-1 is unaffected, as the HA and HD bits are
only set if HAFDBS is detected in the ID register. KVM, on the other
hand, relies on the RES0 behavior at stage-2 to use the same value for
VTCR_EL2 on any cpu in the system. Mitigate the non-RES0 behavior by
leaving VTCR_EL2.HA clear on affected systems.

Cc: stable@vger.kernel.org
Cc: D Scott Phillips <scott@os.amperecomputing.com>
Cc: Darren Hart <darren@os.amperecomputing.com>
Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
---
 Documentation/arm64/silicon-errata.rst |  3 +++
 arch/arm64/Kconfig                     | 17 +++++++++++++++++
 arch/arm64/kernel/cpu_errata.c         |  7 +++++++
 arch/arm64/kvm/hyp/pgtable.c           | 14 +++++++++++---
 arch/arm64/tools/cpucaps               |  1 +
 5 files changed, 39 insertions(+), 3 deletions(-)

diff --git a/Documentation/arm64/silicon-errata.rst b/Documentation/arm64/silicon-errata.rst
index 9e311bc43e05..cd46e2b20a81 100644
--- a/Documentation/arm64/silicon-errata.rst
+++ b/Documentation/arm64/silicon-errata.rst
@@ -52,6 +52,9 @@ stable kernels.
 | Allwinner      | A64/R18         | UNKNOWN1        | SUN50I_ERRATUM_UNKNOWN1     |
 +----------------+-----------------+-----------------+-----------------------------+
 +----------------+-----------------+-----------------+-----------------------------+
+| Ampere         | AmpereOne       | AC03_CPU_38     | AMPERE_ERRATUM_AC03_CPU_38  |
++----------------+-----------------+-----------------+-----------------------------+
++----------------+-----------------+-----------------+-----------------------------+
 | ARM            | Cortex-A510     | #2457168        | ARM64_ERRATUM_2457168       |
 +----------------+-----------------+-----------------+-----------------------------+
 | ARM            | Cortex-A510     | #2064142        | ARM64_ERRATUM_2064142       |
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index b1201d25a8a4..f853af10142b 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -406,6 +406,23 @@ menu "Kernel Features"
 
 menu "ARM errata workarounds via the alternatives framework"
 
+config AMPERE_ERRATUM_AC03_CPU_38
+        bool "AmpereOne: AC03_CPU_38: Certain bits in the Virtualization Translation Control Register and Translation Control Registers do not follow RES0 semantics"
+	default y
+	help
+	  This option adds an alternative code sequence to work around Ampere
+	  erratum AC03_CPU_38 on AmpereOne.
+
+	  The affected design reports FEAT_HAFDBS as not implemented in
+	  ID_AA64MMFR1_EL1.HAFDBS, but (V)TCR_ELx.{HA,HD} are not RES0
+	  as required by the architecture.
+
+	  The workaround forces KVM to explicitly set VTCR_EL2.HA to 0,
+	  which avoids enabling unadvertised hardware Access Flag management
+	  at stage-2.
+
+	  If unsure, say Y.
+
 config ARM64_WORKAROUND_CLEAN_CACHE
 	bool
 
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 307faa2b4395..be66e94a21bd 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -729,6 +729,13 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
 		MIDR_FIXED(MIDR_CPU_VAR_REV(1,1), BIT(25)),
 		.cpu_enable = cpu_clear_bf16_from_user_emulation,
 	},
+#endif
+#ifdef CONFIG_AMPERE_ERRATUM_AC03_CPU_38
+	{
+		.desc = "AmpereOne erratum AC03_CPU_38",
+		.capability = ARM64_WORKAROUND_AMPERE_AC03_CPU_38,
+		ERRATA_MIDR_ALL_VERSIONS(MIDR_AMPERE1),
+	},
 #endif
 	{
 	}
diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c
index 3d61bd3e591d..9b5c8e6c08a0 100644
--- a/arch/arm64/kvm/hyp/pgtable.c
+++ b/arch/arm64/kvm/hyp/pgtable.c
@@ -609,10 +609,18 @@ u64 kvm_get_vtcr(u64 mmfr0, u64 mmfr1, u32 phys_shift)
 #ifdef CONFIG_ARM64_HW_AFDBM
 	/*
 	 * Enable the Hardware Access Flag management, unconditionally
-	 * on all CPUs. The features is RES0 on CPUs without the support
-	 * and must be ignored by the CPUs.
+	 * on all CPUs. In systems that have asymmetric support for the feature
+	 * this allows KVM to leverage hardware support on the subset of cores
+	 * that implement the feature.
+	 *
+	 * The architecture requires VTCR_EL2.HA to be RES0 (thus ignored by
+	 * hardware) on implementations that do not advertise support for the
+	 * feature. As such, setting HA unconditionally is safe, unless you
+	 * happen to be running on a design that has unadvertised support for
+	 * HAFDBS. Here be dragons.
 	 */
-	vtcr |= VTCR_EL2_HA;
+	if (!cpus_have_final_cap(ARM64_WORKAROUND_AMPERE_AC03_CPU_38))
+		vtcr |= VTCR_EL2_HA;
 #endif /* CONFIG_ARM64_HW_AFDBM */
 
 	/* Set the vmid bits */
diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps
index 40ba95472594..9f9a2d6652eb 100644
--- a/arch/arm64/tools/cpucaps
+++ b/arch/arm64/tools/cpucaps
@@ -77,6 +77,7 @@ WORKAROUND_2077057
 WORKAROUND_2457168
 WORKAROUND_2645198
 WORKAROUND_2658417
+WORKAROUND_AMPERE_AC03_CPU_38
 WORKAROUND_TRBE_OVERWRITE_FILL_MODE
 WORKAROUND_TSB_FLUSH_FAILURE
 WORKAROUND_TRBE_WRITE_OUT_OF_RANGE
-- 
2.41.0.162.gfafddb0af9-goog


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 2/3] KVM: arm64: Refactor HFGxTR configuration into separate helpers
  2023-06-09 22:01 [PATCH 0/3] KVM: arm64: Work around Ampere1 erratum AC03_CPU_38 Oliver Upton
  2023-06-09 22:01 ` [PATCH 1/3] arm64: errata: Mitigate Ampere1 erratum AC03_CPU_38 at stage-2 Oliver Upton
@ 2023-06-09 22:01 ` Oliver Upton
  2023-06-09 22:01 ` [PATCH 3/3] KVM: arm64: Prevent guests from enabling HA/HD on Ampere1 Oliver Upton
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 11+ messages in thread
From: Oliver Upton @ 2023-06-09 22:01 UTC (permalink / raw)
  To: kvmarm
  Cc: Marc Zyngier, James Morse, Suzuki K Poulose, Zenghui Yu,
	Will Deacon, Catalin Marinas, linux-arm-kernel, Darren Hart,
	D Scott Phillips, Oliver Upton

A subsequent change will need to flip more trap bits in HFGWTR_EL2. Make
room for this by factoring out the programming of the HFGxTR registers
into helpers and using locals to build the set/clear masks.

Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
---
 arch/arm64/kvm/hyp/include/hyp/switch.h | 60 +++++++++++++++++--------
 1 file changed, 42 insertions(+), 18 deletions(-)

diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index c41166f1a1dd..e5702c27a8b2 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -70,6 +70,44 @@ static inline void __activate_traps_fpsimd32(struct kvm_vcpu *vcpu)
 	}
 }
 
+static inline bool __hfgxtr_traps_required(void)
+{
+	if (cpus_have_final_cap(ARM64_SME))
+		return true;
+
+	return false;
+}
+
+static inline void __activate_traps_hfgxtr(void)
+{
+	u64 r_clr = 0, w_clr = 0, r_set = 0, w_set = 0, tmp;
+
+	if (cpus_have_final_cap(ARM64_SME)) {
+		tmp = HFGxTR_EL2_nSMPRI_EL1_MASK | HFGxTR_EL2_nTPIDR2_EL0_MASK;
+
+		r_clr |= tmp;
+		w_clr |= tmp;
+	}
+
+	sysreg_clear_set_s(SYS_HFGRTR_EL2, r_clr, r_set);
+	sysreg_clear_set_s(SYS_HFGWTR_EL2, w_clr, w_set);
+}
+
+static inline void __deactivate_traps_hfgxtr(void)
+{
+	u64 r_clr = 0, w_clr = 0, r_set = 0, w_set = 0, tmp;
+
+	if (cpus_have_final_cap(ARM64_SME)) {
+		tmp = HFGxTR_EL2_nSMPRI_EL1_MASK | HFGxTR_EL2_nTPIDR2_EL0_MASK;
+
+		r_set |= tmp;
+		w_set |= tmp;
+	}
+
+	sysreg_clear_set_s(SYS_HFGRTR_EL2, r_clr, r_set);
+	sysreg_clear_set_s(SYS_HFGWTR_EL2, w_clr, w_set);
+}
+
 static inline void __activate_traps_common(struct kvm_vcpu *vcpu)
 {
 	/* Trap on AArch32 cp15 c15 (impdef sysregs) accesses (EL1 or EL0) */
@@ -89,16 +127,8 @@ static inline void __activate_traps_common(struct kvm_vcpu *vcpu)
 	vcpu->arch.mdcr_el2_host = read_sysreg(mdcr_el2);
 	write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
 
-	if (cpus_have_final_cap(ARM64_SME)) {
-		sysreg_clear_set_s(SYS_HFGRTR_EL2,
-				   HFGxTR_EL2_nSMPRI_EL1_MASK |
-				   HFGxTR_EL2_nTPIDR2_EL0_MASK,
-				   0);
-		sysreg_clear_set_s(SYS_HFGWTR_EL2,
-				   HFGxTR_EL2_nSMPRI_EL1_MASK |
-				   HFGxTR_EL2_nTPIDR2_EL0_MASK,
-				   0);
-	}
+	if (__hfgxtr_traps_required())
+		__activate_traps_hfgxtr();
 }
 
 static inline void __deactivate_traps_common(struct kvm_vcpu *vcpu)
@@ -109,14 +139,8 @@ static inline void __deactivate_traps_common(struct kvm_vcpu *vcpu)
 	if (kvm_arm_support_pmu_v3())
 		write_sysreg(0, pmuserenr_el0);
 
-	if (cpus_have_final_cap(ARM64_SME)) {
-		sysreg_clear_set_s(SYS_HFGRTR_EL2, 0,
-				   HFGxTR_EL2_nSMPRI_EL1_MASK |
-				   HFGxTR_EL2_nTPIDR2_EL0_MASK);
-		sysreg_clear_set_s(SYS_HFGWTR_EL2, 0,
-				   HFGxTR_EL2_nSMPRI_EL1_MASK |
-				   HFGxTR_EL2_nTPIDR2_EL0_MASK);
-	}
+	if (__hfgxtr_traps_required())
+		__deactivate_traps_hfgxtr();
 }
 
 static inline void ___activate_traps(struct kvm_vcpu *vcpu)
-- 
2.41.0.162.gfafddb0af9-goog


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 3/3] KVM: arm64: Prevent guests from enabling HA/HD on Ampere1
  2023-06-09 22:01 [PATCH 0/3] KVM: arm64: Work around Ampere1 erratum AC03_CPU_38 Oliver Upton
  2023-06-09 22:01 ` [PATCH 1/3] arm64: errata: Mitigate Ampere1 erratum AC03_CPU_38 at stage-2 Oliver Upton
  2023-06-09 22:01 ` [PATCH 2/3] KVM: arm64: Refactor HFGxTR configuration into separate helpers Oliver Upton
@ 2023-06-09 22:01 ` Oliver Upton
  2023-06-14 16:57 ` [PATCH 0/3] KVM: arm64: Work around Ampere1 erratum AC03_CPU_38 Catalin Marinas
                   ` (2 subsequent siblings)
  5 siblings, 0 replies; 11+ messages in thread
From: Oliver Upton @ 2023-06-09 22:01 UTC (permalink / raw)
  To: kvmarm
  Cc: Marc Zyngier, James Morse, Suzuki K Poulose, Zenghui Yu,
	Will Deacon, Catalin Marinas, linux-arm-kernel, Darren Hart,
	D Scott Phillips, Oliver Upton

An erratum in the HAFDBS implementation in AmpereOne was addressed by
clearing the feature in the ID register, with the expectation that
software would not attempt to use the corresponding controls in TCR_EL1.
The architecture, on the other hand, takes a much more pedantic stance
on the subject, requiring the TCR bits behave as RES0.

Take an extremely conservative stance on the issue and leverage the
precise write trap afforded by FGT. Handle guest writes by clearing HA
and HD before writing the intended value to the EL1 register alias.

Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
---
 arch/arm64/kvm/hyp/include/hyp/switch.h | 39 +++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index e5702c27a8b2..17d6bcd321d8 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -75,6 +75,9 @@ static inline bool __hfgxtr_traps_required(void)
 	if (cpus_have_final_cap(ARM64_SME))
 		return true;
 
+	if (cpus_have_final_cap(ARM64_WORKAROUND_AMPERE_AC03_CPU_38))
+		return true;
+
 	return false;
 }
 
@@ -89,6 +92,12 @@ static inline void __activate_traps_hfgxtr(void)
 		w_clr |= tmp;
 	}
 
+	/*
+	 * Trap guest writes to TCR_EL1 to prevent it from enabling HA or HD.
+	 */
+	if (cpus_have_final_cap(ARM64_WORKAROUND_AMPERE_AC03_CPU_38))
+		w_set |= HFGxTR_EL2_TCR_EL1_MASK;
+
 	sysreg_clear_set_s(SYS_HFGRTR_EL2, r_clr, r_set);
 	sysreg_clear_set_s(SYS_HFGWTR_EL2, w_clr, w_set);
 }
@@ -104,6 +113,9 @@ static inline void __deactivate_traps_hfgxtr(void)
 		w_set |= tmp;
 	}
 
+	if (cpus_have_final_cap(ARM64_WORKAROUND_AMPERE_AC03_CPU_38))
+		w_clr |= HFGxTR_EL2_TCR_EL1_MASK;
+
 	sysreg_clear_set_s(SYS_HFGRTR_EL2, r_clr, r_set);
 	sysreg_clear_set_s(SYS_HFGWTR_EL2, w_clr, w_set);
 }
@@ -400,12 +412,39 @@ static bool kvm_hyp_handle_cntpct(struct kvm_vcpu *vcpu)
 	return true;
 }
 
+static bool handle_ampere1_tcr(struct kvm_vcpu *vcpu)
+{
+	u32 sysreg = esr_sys64_to_sysreg(kvm_vcpu_get_esr(vcpu));
+	int rt = kvm_vcpu_sys_get_rt(vcpu);
+	u64 val = vcpu_get_reg(vcpu, rt);
+
+	if (sysreg != SYS_TCR_EL1)
+		return false;
+
+	/*
+	 * Affected parts do not advertise support for hardware Access Flag /
+	 * Dirty state management in ID_AA64MMFR1_EL1.HAFDBS, but the underlying
+	 * control bits are still functional. The architecture requires these be
+	 * RES0 on systems that do not implement FEAT_HAFDBS.
+	 *
+	 * Uphold the requirements of the architecture by masking guest writes
+	 * to TCR_EL1.{HA,HD} here.
+	 */
+	val &= ~(TCR_HD | TCR_HA);
+	write_sysreg_el1(val, SYS_TCR);
+	return true;
+}
+
 static bool kvm_hyp_handle_sysreg(struct kvm_vcpu *vcpu, u64 *exit_code)
 {
 	if (cpus_have_final_cap(ARM64_WORKAROUND_CAVIUM_TX2_219_TVM) &&
 	    handle_tx2_tvm(vcpu))
 		return true;
 
+	if (cpus_have_final_cap(ARM64_WORKAROUND_AMPERE_AC03_CPU_38) &&
+	    handle_ampere1_tcr(vcpu))
+		return true;
+
 	if (static_branch_unlikely(&vgic_v3_cpuif_trap) &&
 	    __vgic_v3_perform_cpuif_access(vcpu) == 1)
 		return true;
-- 
2.41.0.162.gfafddb0af9-goog


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* Re: [PATCH 0/3] KVM: arm64: Work around Ampere1 erratum AC03_CPU_38
  2023-06-09 22:01 [PATCH 0/3] KVM: arm64: Work around Ampere1 erratum AC03_CPU_38 Oliver Upton
                   ` (2 preceding siblings ...)
  2023-06-09 22:01 ` [PATCH 3/3] KVM: arm64: Prevent guests from enabling HA/HD on Ampere1 Oliver Upton
@ 2023-06-14 16:57 ` Catalin Marinas
  2023-06-14 23:06   ` Oliver Upton
  2023-06-15  9:51 ` Marc Zyngier
  2023-06-20 13:15 ` Oliver Upton
  5 siblings, 1 reply; 11+ messages in thread
From: Catalin Marinas @ 2023-06-14 16:57 UTC (permalink / raw)
  To: Oliver Upton
  Cc: kvmarm, Marc Zyngier, James Morse, Suzuki K Poulose, Zenghui Yu,
	Will Deacon, linux-arm-kernel, Darren Hart, D Scott Phillips

On Fri, Jun 09, 2023 at 10:01:01PM +0000, Oliver Upton wrote:
> Small series to work around a CPU erratum on AmpereOne. While the
> implementation does not advertise support for FEAT_HAFDBS (due to
> another erratum), the associated control bits do not have RES0 behavior
> as required by the architecture.
> 
> Usage of HAFDBS at stage-1 is unaffected, since HA and HD are only
> enabled on implementations that advertise the feature. However, KVM
> relies on HA having RES0 semantics if the feature isn't implemented. The
> end result is that KVM enables a broken hardware access flag
> implementation that could lead to correctness issues.

Just curious, what's the correctness issue here? The access flag is
mostly indicative of which pages are old for swapping out/discarding.
It's not like the dirty state which would be dangerous if we get wrong.

-- 
Catalin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH 1/3] arm64: errata: Mitigate Ampere1 erratum AC03_CPU_38 at stage-2
  2023-06-09 22:01 ` [PATCH 1/3] arm64: errata: Mitigate Ampere1 erratum AC03_CPU_38 at stage-2 Oliver Upton
@ 2023-06-14 16:58   ` Catalin Marinas
  2023-06-14 17:15   ` D Scott Phillips
  1 sibling, 0 replies; 11+ messages in thread
From: Catalin Marinas @ 2023-06-14 16:58 UTC (permalink / raw)
  To: Oliver Upton
  Cc: kvmarm, Marc Zyngier, James Morse, Suzuki K Poulose, Zenghui Yu,
	Will Deacon, linux-arm-kernel, Darren Hart, D Scott Phillips,
	stable

On Fri, Jun 09, 2023 at 10:01:02PM +0000, Oliver Upton wrote:
> AmpereOne has an erratum in its implementation of FEAT_HAFDBS that
> required disabling the feature on the design. This was done by reporting
> the feature as not implemented in the ID register, although the
> corresponding control bits were not actually RES0. This does not align
> well with the requirements of the architecture, which mandates these
> bits be RES0 if HAFDBS isn't implemented.
> 
> The kernel's use of stage-1 is unaffected, as the HA and HD bits are
> only set if HAFDBS is detected in the ID register. KVM, on the other
> hand, relies on the RES0 behavior at stage-2 to use the same value for
> VTCR_EL2 on any cpu in the system. Mitigate the non-RES0 behavior by
> leaving VTCR_EL2.HA clear on affected systems.
> 
> Cc: stable@vger.kernel.org
> Cc: D Scott Phillips <scott@os.amperecomputing.com>
> Cc: Darren Hart <darren@os.amperecomputing.com>
> Signed-off-by: Oliver Upton <oliver.upton@linux.dev>

For the non-KVM bits in here:

Acked-by: Catalin Marinas <catalin.marinas@arm.com>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH 1/3] arm64: errata: Mitigate Ampere1 erratum AC03_CPU_38 at stage-2
  2023-06-09 22:01 ` [PATCH 1/3] arm64: errata: Mitigate Ampere1 erratum AC03_CPU_38 at stage-2 Oliver Upton
  2023-06-14 16:58   ` Catalin Marinas
@ 2023-06-14 17:15   ` D Scott Phillips
  1 sibling, 0 replies; 11+ messages in thread
From: D Scott Phillips @ 2023-06-14 17:15 UTC (permalink / raw)
  To: Oliver Upton, kvmarm
  Cc: Marc Zyngier, James Morse, Suzuki K Poulose, Zenghui Yu,
	Will Deacon, Catalin Marinas, linux-arm-kernel, Darren Hart,
	Oliver Upton, stable

Oliver Upton <oliver.upton@linux.dev> writes:

> AmpereOne has an erratum in its implementation of FEAT_HAFDBS that
> required disabling the feature on the design. This was done by reporting
> the feature as not implemented in the ID register, although the
> corresponding control bits were not actually RES0. This does not align
> well with the requirements of the architecture, which mandates these
> bits be RES0 if HAFDBS isn't implemented.
>
> The kernel's use of stage-1 is unaffected, as the HA and HD bits are
> only set if HAFDBS is detected in the ID register. KVM, on the other
> hand, relies on the RES0 behavior at stage-2 to use the same value for
> VTCR_EL2 on any cpu in the system. Mitigate the non-RES0 behavior by
> leaving VTCR_EL2.HA clear on affected systems.
>
> Cc: stable@vger.kernel.org
> Cc: D Scott Phillips <scott@os.amperecomputing.com>
> Cc: Darren Hart <darren@os.amperecomputing.com>
> Signed-off-by: Oliver Upton <oliver.upton@linux.dev>

Acked-by: D Scott Phillips <scott@os.amperecomputing.com>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH 0/3] KVM: arm64: Work around Ampere1 erratum AC03_CPU_38
  2023-06-14 16:57 ` [PATCH 0/3] KVM: arm64: Work around Ampere1 erratum AC03_CPU_38 Catalin Marinas
@ 2023-06-14 23:06   ` Oliver Upton
  2023-06-15  8:36     ` Catalin Marinas
  0 siblings, 1 reply; 11+ messages in thread
From: Oliver Upton @ 2023-06-14 23:06 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: kvmarm, Marc Zyngier, James Morse, Suzuki K Poulose, Zenghui Yu,
	Will Deacon, linux-arm-kernel, Darren Hart, D Scott Phillips

Hey Catalin,

On Wed, Jun 14, 2023 at 05:57:55PM +0100, Catalin Marinas wrote:
> On Fri, Jun 09, 2023 at 10:01:01PM +0000, Oliver Upton wrote:
> > Small series to work around a CPU erratum on AmpereOne. While the
> > implementation does not advertise support for FEAT_HAFDBS (due to
> > another erratum), the associated control bits do not have RES0 behavior
> > as required by the architecture.
> > 
> > Usage of HAFDBS at stage-1 is unaffected, since HA and HD are only
> > enabled on implementations that advertise the feature. However, KVM
> > relies on HA having RES0 semantics if the feature isn't implemented. The
> > end result is that KVM enables a broken hardware access flag
> > implementation that could lead to correctness issues.
> 
> Just curious, what's the correctness issue here? The access flag is
> mostly indicative of which pages are old for swapping out/discarding.
> It's not like the dirty state which would be dangerous if we get wrong.

I probably could have helped out by giving the full context.

The software-observable behavior on this system is that the A or D
updates could arrive after a PTE has been marked as invalid, which could
corrupt software metadata stuffed into the page tables. We do exactly
that at stage-2 in KVM for parallel fault handling, where a magic value
indicates a PTE is being updated by another thread.

-- 
Thanks,
Oliver

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH 0/3] KVM: arm64: Work around Ampere1 erratum AC03_CPU_38
  2023-06-14 23:06   ` Oliver Upton
@ 2023-06-15  8:36     ` Catalin Marinas
  0 siblings, 0 replies; 11+ messages in thread
From: Catalin Marinas @ 2023-06-15  8:36 UTC (permalink / raw)
  To: Oliver Upton
  Cc: kvmarm, Marc Zyngier, James Morse, Suzuki K Poulose, Zenghui Yu,
	Will Deacon, linux-arm-kernel, Darren Hart, D Scott Phillips

On Wed, Jun 14, 2023 at 11:06:40PM +0000, Oliver Upton wrote:
> Hey Catalin,
> 
> On Wed, Jun 14, 2023 at 05:57:55PM +0100, Catalin Marinas wrote:
> > On Fri, Jun 09, 2023 at 10:01:01PM +0000, Oliver Upton wrote:
> > > Small series to work around a CPU erratum on AmpereOne. While the
> > > implementation does not advertise support for FEAT_HAFDBS (due to
> > > another erratum), the associated control bits do not have RES0 behavior
> > > as required by the architecture.
> > > 
> > > Usage of HAFDBS at stage-1 is unaffected, since HA and HD are only
> > > enabled on implementations that advertise the feature. However, KVM
> > > relies on HA having RES0 semantics if the feature isn't implemented. The
> > > end result is that KVM enables a broken hardware access flag
> > > implementation that could lead to correctness issues.
> > 
> > Just curious, what's the correctness issue here? The access flag is
> > mostly indicative of which pages are old for swapping out/discarding.
> > It's not like the dirty state which would be dangerous if we get wrong.
> 
> I probably could have helped out by giving the full context.
> 
> The software-observable behavior on this system is that the A or D
> updates could arrive after a PTE has been marked as invalid, which could
> corrupt software metadata stuffed into the page tables. We do exactly
> that at stage-2 in KVM for parallel fault handling, where a magic value
> indicates a PTE is being updated by another thread.

Ah, ok, that's dangerous indeed. Thanks for the details (you may want to
add them in the patch description or the erratum kconfig entry).

-- 
Catalin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH 0/3] KVM: arm64: Work around Ampere1 erratum AC03_CPU_38
  2023-06-09 22:01 [PATCH 0/3] KVM: arm64: Work around Ampere1 erratum AC03_CPU_38 Oliver Upton
                   ` (3 preceding siblings ...)
  2023-06-14 16:57 ` [PATCH 0/3] KVM: arm64: Work around Ampere1 erratum AC03_CPU_38 Catalin Marinas
@ 2023-06-15  9:51 ` Marc Zyngier
  2023-06-20 13:15 ` Oliver Upton
  5 siblings, 0 replies; 11+ messages in thread
From: Marc Zyngier @ 2023-06-15  9:51 UTC (permalink / raw)
  To: Oliver Upton
  Cc: kvmarm, James Morse, Suzuki K Poulose, Zenghui Yu, Will Deacon,
	Catalin Marinas, linux-arm-kernel, Darren Hart, D Scott Phillips

On Fri, 09 Jun 2023 23:01:01 +0100,
Oliver Upton <oliver.upton@linux.dev> wrote:
> 
> Hi folks,
> 
> Small series to work around a CPU erratum on AmpereOne. While the
> implementation does not advertise support for FEAT_HAFDBS (due to
> another erratum), the associated control bits do not have RES0 behavior
> as required by the architecture.
> 
> Usage of HAFDBS at stage-1 is unaffected, since HA and HD are only
> enabled on implementations that advertise the feature. However, KVM
> relies on HA having RES0 semantics if the feature isn't implemented. The
> end result is that KVM enables a broken hardware access flag
> implementation that could lead to correctness issues.
> 
> Applies to 6.4-rc1. Tested with access_tracking_perf_test, verifying
> that KVM is indeed taking Access Flag faults.

For the series:

Reviewed-by: Marc Zyngier <maz@kernel.org>

	M.

-- 
Without deviation from the norm, progress is not possible.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH 0/3] KVM: arm64: Work around Ampere1 erratum AC03_CPU_38
  2023-06-09 22:01 [PATCH 0/3] KVM: arm64: Work around Ampere1 erratum AC03_CPU_38 Oliver Upton
                   ` (4 preceding siblings ...)
  2023-06-15  9:51 ` Marc Zyngier
@ 2023-06-20 13:15 ` Oliver Upton
  5 siblings, 0 replies; 11+ messages in thread
From: Oliver Upton @ 2023-06-20 13:15 UTC (permalink / raw)
  To: Oliver Upton, kvmarm
  Cc: Darren Hart, James Morse, D Scott Phillips, Marc Zyngier,
	Will Deacon, Zenghui Yu, Suzuki K Poulose, Catalin Marinas,
	linux-arm-kernel

On Fri, 9 Jun 2023 22:01:01 +0000, Oliver Upton wrote:
> Small series to work around a CPU erratum on AmpereOne. While the
> implementation does not advertise support for FEAT_HAFDBS (due to
> another erratum), the associated control bits do not have RES0 behavior
> as required by the architecture.
> 
> Usage of HAFDBS at stage-1 is unaffected, since HA and HD are only
> enabled on implementations that advertise the feature. However, KVM
> relies on HA having RES0 semantics if the feature isn't implemented. The
> end result is that KVM enables a broken hardware access flag
> implementation that could lead to correctness issues.
> 
> [...]

Applied w/ an expanded description of what's wrong with the unadvertised
HAFDBS implementation, per Catalin's suggestion.

[1/3] arm64: errata: Mitigate Ampere1 erratum AC03_CPU_38 at stage-2
      https://git.kernel.org/kvmarm/kvmarm/c/6df696cd9bc1
[2/3] KVM: arm64: Refactor HFGxTR configuration into separate helpers
      https://git.kernel.org/kvmarm/kvmarm/c/ce4a36225753
[3/3] KVM: arm64: Prevent guests from enabling HA/HD on Ampere1
      https://git.kernel.org/kvmarm/kvmarm/c/082fdfd13841

--
Best,
Oliver

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2023-06-20 13:16 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-06-09 22:01 [PATCH 0/3] KVM: arm64: Work around Ampere1 erratum AC03_CPU_38 Oliver Upton
2023-06-09 22:01 ` [PATCH 1/3] arm64: errata: Mitigate Ampere1 erratum AC03_CPU_38 at stage-2 Oliver Upton
2023-06-14 16:58   ` Catalin Marinas
2023-06-14 17:15   ` D Scott Phillips
2023-06-09 22:01 ` [PATCH 2/3] KVM: arm64: Refactor HFGxTR configuration into separate helpers Oliver Upton
2023-06-09 22:01 ` [PATCH 3/3] KVM: arm64: Prevent guests from enabling HA/HD on Ampere1 Oliver Upton
2023-06-14 16:57 ` [PATCH 0/3] KVM: arm64: Work around Ampere1 erratum AC03_CPU_38 Catalin Marinas
2023-06-14 23:06   ` Oliver Upton
2023-06-15  8:36     ` Catalin Marinas
2023-06-15  9:51 ` Marc Zyngier
2023-06-20 13:15 ` Oliver Upton

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).