[PATCH v2 0/2] KVM: arm64: Support for Arm v8.8 memcpy instructions in KVM guests

[PATCH v2 0/2] KVM: arm64: Support for Arm v8.8 memcpy instructions in KVM guests

[Kristina Martsenko]

Hi,

This is v2 of the series to allow using the new Arm memory copy instructions
in KVM guests. See v1 for more information [1].

Changes in v2:
 - Dropped HCRX_EL2 vcpu field
 - Rebased onto v6.6-rc2

Thanks,
Kristina

[1] https://lore.kernel.org/kvmarm/20230915124840.474888-1-kristina.martsenko@arm.com/

Kristina Martsenko (2):
  KVM: arm64: Add handler for MOPS exceptions
  KVM: arm64: Expose MOPS instructions to guests

 arch/arm64/include/asm/kvm_arm.h              |  4 +-
 arch/arm64/include/asm/traps.h                | 54 ++++++++++++++++++-
 arch/arm64/kernel/traps.c                     | 48 +----------------
 arch/arm64/kvm/hyp/include/hyp/switch.h       | 17 ++++++
 .../arm64/kvm/hyp/include/nvhe/fixed_config.h |  3 +-
 arch/arm64/kvm/hyp/nvhe/switch.c              |  2 +
 arch/arm64/kvm/hyp/vhe/switch.c               |  1 +
 arch/arm64/kvm/sys_regs.c                     |  1 -
 8 files changed, 78 insertions(+), 52 deletions(-)


base-commit: ce9ecca0238b140b88f43859b211c9fdfd8e5b70
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v2 1/2] KVM: arm64: Add handler for MOPS exceptions

[Kristina Martsenko]

An Armv8.8 FEAT_MOPS main or epilogue instruction will take an exception
if executed on a CPU with a different MOPS implementation option (A or
B) than the CPU where the preceding prologue instruction ran. In this
case the OS exception handler is expected to reset the registers and
restart execution from the prologue instruction.

A KVM guest may use the instructions at EL1 at times when the guest is
not able to handle the exception, expecting that the instructions will
only run on one CPU (e.g. when running UEFI boot services in the guest).
As KVM may reschedule the guest between different types of CPUs at any
time (on an asymmetric system), it needs to also handle the resulting
exception itself in case the guest is not able to. A similar situation
will also occur in the future when live migrating a guest from one type
of CPU to another.

Add handling for the MOPS exception to KVM. The handling can be shared
with the EL0 exception handler, as the logic and register layouts are
the same. The exception can be handled right after exiting a guest,
which avoids the cost of returning to the host exit handler.

Similarly to the EL0 exception handler, in case the main or epilogue
instruction is being single stepped, it makes sense to finish the step
before executing the prologue instruction, so advance the single step
state machine.

Signed-off-by: Kristina Martsenko <kristina.martsenko@arm.com>
---
 arch/arm64/include/asm/traps.h          | 54 ++++++++++++++++++++++++-
 arch/arm64/kernel/traps.c               | 48 +---------------------
 arch/arm64/kvm/hyp/include/hyp/switch.h | 17 ++++++++
 arch/arm64/kvm/hyp/nvhe/switch.c        |  2 +
 arch/arm64/kvm/hyp/vhe/switch.c         |  1 +
 5 files changed, 73 insertions(+), 49 deletions(-)

diff --git a/arch/arm64/include/asm/traps.h b/arch/arm64/include/asm/traps.h
index d66dfb3a72dd..eefe766d6161 100644
--- a/arch/arm64/include/asm/traps.h
+++ b/arch/arm64/include/asm/traps.h
@@ -9,10 +9,9 @@
 
 #include <linux/list.h>
 #include <asm/esr.h>
+#include <asm/ptrace.h>
 #include <asm/sections.h>
 
-struct pt_regs;
-
 #ifdef CONFIG_ARMV8_DEPRECATED
 bool try_emulate_armv8_deprecated(struct pt_regs *regs, u32 insn);
 #else
@@ -101,4 +100,55 @@ static inline unsigned long arm64_ras_serror_get_severity(unsigned long esr)
 
 bool arm64_is_fatal_ras_serror(struct pt_regs *regs, unsigned long esr);
 void __noreturn arm64_serror_panic(struct pt_regs *regs, unsigned long esr);
+
+static inline void arm64_mops_reset_regs(struct user_pt_regs *regs, unsigned long esr)
+{
+	bool wrong_option = esr & ESR_ELx_MOPS_ISS_WRONG_OPTION;
+	bool option_a = esr & ESR_ELx_MOPS_ISS_OPTION_A;
+	int dstreg = ESR_ELx_MOPS_ISS_DESTREG(esr);
+	int srcreg = ESR_ELx_MOPS_ISS_SRCREG(esr);
+	int sizereg = ESR_ELx_MOPS_ISS_SIZEREG(esr);
+	unsigned long dst, src, size;
+
+	dst = regs->regs[dstreg];
+	src = regs->regs[srcreg];
+	size = regs->regs[sizereg];
+
+	/*
+	 * Put the registers back in the original format suitable for a
+	 * prologue instruction, using the generic return routine from the
+	 * Arm ARM (DDI 0487I.a) rules CNTMJ and MWFQH.
+	 */
+	if (esr & ESR_ELx_MOPS_ISS_MEM_INST) {
+		/* SET* instruction */
+		if (option_a ^ wrong_option) {
+			/* Format is from Option A; forward set */
+			regs->regs[dstreg] = dst + size;
+			regs->regs[sizereg] = -size;
+		}
+	} else {
+		/* CPY* instruction */
+		if (!(option_a ^ wrong_option)) {
+			/* Format is from Option B */
+			if (regs->pstate & PSR_N_BIT) {
+				/* Backward copy */
+				regs->regs[dstreg] = dst - size;
+				regs->regs[srcreg] = src - size;
+			}
+		} else {
+			/* Format is from Option A */
+			if (size & BIT(63)) {
+				/* Forward copy */
+				regs->regs[dstreg] = dst + size;
+				regs->regs[srcreg] = src + size;
+				regs->regs[sizereg] = -size;
+			}
+		}
+	}
+
+	if (esr & ESR_ELx_MOPS_ISS_FROM_EPILOGUE)
+		regs->pc -= 8;
+	else
+		regs->pc -= 4;
+}
 #endif
diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
index 8b70759cdbb9..ede65a20e7dc 100644
--- a/arch/arm64/kernel/traps.c
+++ b/arch/arm64/kernel/traps.c
@@ -516,53 +516,7 @@ void do_el1_fpac(struct pt_regs *regs, unsigned long esr)
 
 void do_el0_mops(struct pt_regs *regs, unsigned long esr)
 {
-	bool wrong_option = esr & ESR_ELx_MOPS_ISS_WRONG_OPTION;
-	bool option_a = esr & ESR_ELx_MOPS_ISS_OPTION_A;
-	int dstreg = ESR_ELx_MOPS_ISS_DESTREG(esr);
-	int srcreg = ESR_ELx_MOPS_ISS_SRCREG(esr);
-	int sizereg = ESR_ELx_MOPS_ISS_SIZEREG(esr);
-	unsigned long dst, src, size;
-
-	dst = pt_regs_read_reg(regs, dstreg);
-	src = pt_regs_read_reg(regs, srcreg);
-	size = pt_regs_read_reg(regs, sizereg);
-
-	/*
-	 * Put the registers back in the original format suitable for a
-	 * prologue instruction, using the generic return routine from the
-	 * Arm ARM (DDI 0487I.a) rules CNTMJ and MWFQH.
-	 */
-	if (esr & ESR_ELx_MOPS_ISS_MEM_INST) {
-		/* SET* instruction */
-		if (option_a ^ wrong_option) {
-			/* Format is from Option A; forward set */
-			pt_regs_write_reg(regs, dstreg, dst + size);
-			pt_regs_write_reg(regs, sizereg, -size);
-		}
-	} else {
-		/* CPY* instruction */
-		if (!(option_a ^ wrong_option)) {
-			/* Format is from Option B */
-			if (regs->pstate & PSR_N_BIT) {
-				/* Backward copy */
-				pt_regs_write_reg(regs, dstreg, dst - size);
-				pt_regs_write_reg(regs, srcreg, src - size);
-			}
-		} else {
-			/* Format is from Option A */
-			if (size & BIT(63)) {
-				/* Forward copy */
-				pt_regs_write_reg(regs, dstreg, dst + size);
-				pt_regs_write_reg(regs, srcreg, src + size);
-				pt_regs_write_reg(regs, sizereg, -size);
-			}
-		}
-	}
-
-	if (esr & ESR_ELx_MOPS_ISS_FROM_EPILOGUE)
-		regs->pc -= 8;
-	else
-		regs->pc -= 4;
+	arm64_mops_reset_regs(&regs->user_regs, esr);
 
 	/*
 	 * If single stepping then finish the step before executing the
diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index 9cfe6bd1dbe4..f99d8af0b9af 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -30,6 +30,7 @@
 #include <asm/fpsimd.h>
 #include <asm/debug-monitors.h>
 #include <asm/processor.h>
+#include <asm/traps.h>
 
 struct kvm_exception_table_entry {
 	int insn, fixup;
@@ -265,6 +266,22 @@ static inline bool __populate_fault_info(struct kvm_vcpu *vcpu)
 	return __get_fault_info(vcpu->arch.fault.esr_el2, &vcpu->arch.fault);
 }
 
+static bool kvm_hyp_handle_mops(struct kvm_vcpu *vcpu, u64 *exit_code)
+{
+	*vcpu_pc(vcpu) = read_sysreg_el2(SYS_ELR);
+	arm64_mops_reset_regs(vcpu_gp_regs(vcpu), vcpu->arch.fault.esr_el2);
+	write_sysreg_el2(*vcpu_pc(vcpu), SYS_ELR);
+
+	/*
+	 * Finish potential single step before executing the prologue
+	 * instruction.
+	 */
+	*vcpu_cpsr(vcpu) &= ~DBG_SPSR_SS;
+	write_sysreg_el2(*vcpu_cpsr(vcpu), SYS_SPSR);
+
+	return true;
+}
+
 static inline void __hyp_sve_restore_guest(struct kvm_vcpu *vcpu)
 {
 	sve_cond_update_zcr_vq(vcpu_sve_max_vq(vcpu) - 1, SYS_ZCR_EL2);
diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c
index c353a06ee7e6..c50f8459e4fc 100644
--- a/arch/arm64/kvm/hyp/nvhe/switch.c
+++ b/arch/arm64/kvm/hyp/nvhe/switch.c
@@ -192,6 +192,7 @@ static const exit_handler_fn hyp_exit_handlers[] = {
 	[ESR_ELx_EC_DABT_LOW]		= kvm_hyp_handle_dabt_low,
 	[ESR_ELx_EC_WATCHPT_LOW]	= kvm_hyp_handle_watchpt_low,
 	[ESR_ELx_EC_PAC]		= kvm_hyp_handle_ptrauth,
+	[ESR_ELx_EC_MOPS]		= kvm_hyp_handle_mops,
 };
 
 static const exit_handler_fn pvm_exit_handlers[] = {
@@ -203,6 +204,7 @@ static const exit_handler_fn pvm_exit_handlers[] = {
 	[ESR_ELx_EC_DABT_LOW]		= kvm_hyp_handle_dabt_low,
 	[ESR_ELx_EC_WATCHPT_LOW]	= kvm_hyp_handle_watchpt_low,
 	[ESR_ELx_EC_PAC]		= kvm_hyp_handle_ptrauth,
+	[ESR_ELx_EC_MOPS]		= kvm_hyp_handle_mops,
 };
 
 static const exit_handler_fn *kvm_get_exit_handler_array(struct kvm_vcpu *vcpu)
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 6537f58b1a8c..796202f2e08f 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -126,6 +126,7 @@ static const exit_handler_fn hyp_exit_handlers[] = {
 	[ESR_ELx_EC_DABT_LOW]		= kvm_hyp_handle_dabt_low,
 	[ESR_ELx_EC_WATCHPT_LOW]	= kvm_hyp_handle_watchpt_low,
 	[ESR_ELx_EC_PAC]		= kvm_hyp_handle_ptrauth,
+	[ESR_ELx_EC_MOPS]		= kvm_hyp_handle_mops,
 };
 
 static const exit_handler_fn *kvm_get_exit_handler_array(struct kvm_vcpu *vcpu)
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v2 2/2] KVM: arm64: Expose MOPS instructions to guests

[Kristina Martsenko]

Expose the Armv8.8 FEAT_MOPS feature to guests in the ID register and
allow the MOPS instructions to be run in a guest. Only expose MOPS if
the whole system supports it.

Note, it is expected that guests do not use these instructions on MMIO,
similarly to other instructions where ESR_EL2.ISV==0 such as LDP/STP.

Signed-off-by: Kristina Martsenko <kristina.martsenko@arm.com>
---
 arch/arm64/include/asm/kvm_arm.h               | 4 +++-
 arch/arm64/kvm/hyp/include/nvhe/fixed_config.h | 3 ++-
 arch/arm64/kvm/sys_regs.c                      | 1 -
 3 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 5882b2415596..2186927b1d21 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -102,7 +102,9 @@
 #define HCR_HOST_NVHE_PROTECTED_FLAGS (HCR_HOST_NVHE_FLAGS | HCR_TSC)
 #define HCR_HOST_VHE_FLAGS (HCR_RW | HCR_TGE | HCR_E2H)
 
-#define HCRX_GUEST_FLAGS (HCRX_EL2_SMPME | HCRX_EL2_TCR2En)
+#define HCRX_GUEST_FLAGS \
+	(HCRX_EL2_SMPME | HCRX_EL2_TCR2En | \
+	 (cpus_have_final_cap(ARM64_HAS_MOPS) ? (HCRX_EL2_MSCEn | HCRX_EL2_MCE2) : 0))
 #define HCRX_HOST_FLAGS (HCRX_EL2_MSCEn | HCRX_EL2_TCR2En)
 
 /* TCR_EL2 Registers bits */
diff --git a/arch/arm64/kvm/hyp/include/nvhe/fixed_config.h b/arch/arm64/kvm/hyp/include/nvhe/fixed_config.h
index 37440e1dda93..e91922daa8ca 100644
--- a/arch/arm64/kvm/hyp/include/nvhe/fixed_config.h
+++ b/arch/arm64/kvm/hyp/include/nvhe/fixed_config.h
@@ -197,7 +197,8 @@
 
 #define PVM_ID_AA64ISAR2_ALLOW (\
 	ARM64_FEATURE_MASK(ID_AA64ISAR2_EL1_GPA3) | \
-	ARM64_FEATURE_MASK(ID_AA64ISAR2_EL1_APA3) \
+	ARM64_FEATURE_MASK(ID_AA64ISAR2_EL1_APA3) | \
+	ARM64_FEATURE_MASK(ID_AA64ISAR2_EL1_MOPS) \
 	)
 
 u64 pvm_read_id_reg(const struct kvm_vcpu *vcpu, u32 id);
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index e92ec810d449..153baf2f72cb 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1338,7 +1338,6 @@ static u64 __kvm_read_sanitised_id_reg(const struct kvm_vcpu *vcpu,
 				 ARM64_FEATURE_MASK(ID_AA64ISAR2_EL1_GPA3));
 		if (!cpus_have_final_cap(ARM64_HAS_WFXT))
 			val &= ~ARM64_FEATURE_MASK(ID_AA64ISAR2_EL1_WFxT);
-		val &= ~ARM64_FEATURE_MASK(ID_AA64ISAR2_EL1_MOPS);
 		break;
 	case SYS_ID_AA64MMFR2_EL1:
 		val &= ~ID_AA64MMFR2_EL1_CCIDX_MASK;
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 1/2] KVM: arm64: Add handler for MOPS exceptions

[Marc Zyngier]

Hi Kristina,

On Fri, 22 Sep 2023 12:25:07 +0100,
Kristina Martsenko <kristina.martsenko@arm.com> wrote:
> 
> An Armv8.8 FEAT_MOPS main or epilogue instruction will take an exception
> if executed on a CPU with a different MOPS implementation option (A or
> B) than the CPU where the preceding prologue instruction ran. In this
> case the OS exception handler is expected to reset the registers and
> restart execution from the prologue instruction.
> 
> A KVM guest may use the instructions at EL1 at times when the guest is
> not able to handle the exception, expecting that the instructions will
> only run on one CPU (e.g. when running UEFI boot services in the guest).
> As KVM may reschedule the guest between different types of CPUs at any
> time (on an asymmetric system), it needs to also handle the resulting
> exception itself in case the guest is not able to. A similar situation
> will also occur in the future when live migrating a guest from one type
> of CPU to another.
>
> Add handling for the MOPS exception to KVM. The handling can be shared
> with the EL0 exception handler, as the logic and register layouts are
> the same. The exception can be handled right after exiting a guest,
> which avoids the cost of returning to the host exit handler.
> 
> Similarly to the EL0 exception handler, in case the main or epilogue
> instruction is being single stepped, it makes sense to finish the step
> before executing the prologue instruction, so advance the single step
> state machine.

What is the rationale for advancing the state machine? Shouldn't we
instead return to the guest and immediately get the SS exception,
which in turn gets reported to userspace? Is it because we rollback
the PC to a previous instruction?

In the latter case, won't userspace see multiple SS exceptions for the
middle instruction if trapping from the epilogue? This would be a bit
surprising, to say the least.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 1/2] KVM: arm64: Add handler for MOPS exceptions

[Kristina Martsenko]

On 24/09/2023 15:48, Marc Zyngier wrote:
> Hi Kristina,

Hi Marc,

> On Fri, 22 Sep 2023 12:25:07 +0100,
> Kristina Martsenko <kristina.martsenko@arm.com> wrote:
>>
>> An Armv8.8 FEAT_MOPS main or epilogue instruction will take an exception
>> if executed on a CPU with a different MOPS implementation option (A or
>> B) than the CPU where the preceding prologue instruction ran. In this
>> case the OS exception handler is expected to reset the registers and
>> restart execution from the prologue instruction.
>>
>> A KVM guest may use the instructions at EL1 at times when the guest is
>> not able to handle the exception, expecting that the instructions will
>> only run on one CPU (e.g. when running UEFI boot services in the guest).
>> As KVM may reschedule the guest between different types of CPUs at any
>> time (on an asymmetric system), it needs to also handle the resulting
>> exception itself in case the guest is not able to. A similar situation
>> will also occur in the future when live migrating a guest from one type
>> of CPU to another.
>>
>> Add handling for the MOPS exception to KVM. The handling can be shared
>> with the EL0 exception handler, as the logic and register layouts are
>> the same. The exception can be handled right after exiting a guest,
>> which avoids the cost of returning to the host exit handler.
>>
>> Similarly to the EL0 exception handler, in case the main or epilogue
>> instruction is being single stepped, it makes sense to finish the step
>> before executing the prologue instruction, so advance the single step
>> state machine.
> 
> What is the rationale for advancing the state machine? Shouldn't we
> instead return to the guest and immediately get the SS exception,
> which in turn gets reported to userspace? Is it because we rollback
> the PC to a previous instruction?

Yes, because we rollback the PC to the prologue instruction. We advance the
state machine so that the SS exception is taken immediately upon returning to
the guest at the prologue instruction. If we didn't advance it then we would
return to the guest, execute the prologue instruction, and then take the SS
exception on the middle instruction. Which would be surprising as userspace
would see the middle and epilogue instructions executed multiple times but not
the prologue.

> In the latter case, won't userspace see multiple SS exceptions for the
> middle instruction if trapping from the epilogue? This would be a bit
> surprising, to say the least.

Not sure I follow. Do you mean multiple in a row or multiple in total? Not in a
row (we step the prologue instruction in between), but yes in total (every time
we start executing the middle instruction). And this happens when trapping from
the middle instruction too, not just the epilogue. Do you see a better way of
handling it?

Here is an example of what GDB sees when single stepping a guest while the
guest executes these instructions ("mops ex" are debug prints in kvm; I've
added prologue/main/epilogue comments):

Breakpoint 2, 0xffff80008051b6a4 in ?? ()
0xffff80008051b6a8 in ?? () # prologue
0xffff80008051b6ac in ?? () # main
[   33.615305] mops ex: memcpy: B->A: fwd: main
0xffff80008051b6a8 in ?? () # prologue
0xffff80008051b6ac in ?? () # main
0xffff80008051b6b0 in ?? () # epilogue
[   34.141251] mops ex: memcpy: A->B: fwd: epi
0xffff80008051b6a8 in ?? () # prologue
0xffff80008051b6ac in ?? () # main
0xffff80008051b6b0 in ?? () # epilogue
[   34.209822] mops ex: memcpy: B->A: fwd: epi
0xffff80008051b6a8 in ?? () # prologue
0xffff80008051b6ac in ?? () # main
0xffff80008051b6b0 in ?? () # epilogue
0xffff80008051b6b4 in ?? ()
[...]

Thanks,
Kristina


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 0/2] KVM: arm64: Support for Arm v8.8 memcpy instructions in KVM guests

[Oliver Upton]

Hi Kristina,

On Fri, Sep 22, 2023 at 12:25:06PM +0100, Kristina Martsenko wrote:
> Hi,
> 
> This is v2 of the series to allow using the new Arm memory copy instructions
> in KVM guests. See v1 for more information [1].


Thanks for sending out the series. I've been thinking about what the
architecture says for MOPS, and I wonder if what's currently in the
Arm ARM is clear enough for EL1 software to be written robustly.

While HCRX_EL2.MCE2 allows the hypervisor to intervene on MOPS
exceptions from EL1, there's no such control for EL0. So when vCPU
migration occurs EL1 could get an unexpected MOPS exception, even for a
process that was pinned to a single (virtual) CPU implementation.

Additionally, the wording of I_NXHPS seems to suggest that EL2 handling
of MOPS exceptions is only expected in certain circumstances where EL1 is
incapable of handling an exception. Is the unwritten expectation then
that EL1 software should tolerate 'unexpected' MOPS exceptions from EL1
and EL0, even if EL1 did not migrate the PE context?

Perhaps I'm being pedantic, but I'd really like for there to be some
documentation that suggests MOPS exceptions can happen due to context
migration done by a higher EL as that is the only option in the context
of virtualization.

-- 
Thanks,
Oliver

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 1/2] KVM: arm64: Add handler for MOPS exceptions

[Oliver Upton]

On Mon, Sep 25, 2023 at 04:16:06PM +0100, Kristina Martsenko wrote:

[...]

> > What is the rationale for advancing the state machine? Shouldn't we
> > instead return to the guest and immediately get the SS exception,
> > which in turn gets reported to userspace? Is it because we rollback
> > the PC to a previous instruction?
> 
> Yes, because we rollback the PC to the prologue instruction. We advance the
> state machine so that the SS exception is taken immediately upon returning to
> the guest at the prologue instruction. If we didn't advance it then we would
> return to the guest, execute the prologue instruction, and then take the SS
> exception on the middle instruction. Which would be surprising as userspace
> would see the middle and epilogue instructions executed multiple times but not
> the prologue.

I agree with Kristina that taking the SS exception on the prologue is
likely the best course of action. Especially since it matches the
behavior of single-stepping an EL0 MOPS sequence with an intervening CPU
migration.

This behavior might throw an EL1 that single-steps itself for a loop,
but I think it is impossible for a hypervisor to hide the consequences
of vCPU migration with MOPS in the first place.

Marc, I'm guessing you were most concerned about the former case where
the VMM was debugging the guest. Is there something you're concerned
about I missed?

-- 
Thanks,
Oliver

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 0/2] KVM: arm64: Support for Arm v8.8 memcpy instructions in KVM guests

[Kristina Martsenko]

On 27/09/2023 07:00, Oliver Upton wrote:
> Hi Kristina,

Hi Oliver,

> 
> On Fri, Sep 22, 2023 at 12:25:06PM +0100, Kristina Martsenko wrote:
>> Hi,
>>
>> This is v2 of the series to allow using the new Arm memory copy instructions
>> in KVM guests. See v1 for more information [1].
> 
> 
> Thanks for sending out the series. I've been thinking about what the
> architecture says for MOPS, and I wonder if what's currently in the
> Arm ARM is clear enough for EL1 software to be written robustly.
> 
> While HCRX_EL2.MCE2 allows the hypervisor to intervene on MOPS
> exceptions from EL1, there's no such control for EL0. So when vCPU
> migration occurs EL1 could get an unexpected MOPS exception, even for a
> process that was pinned to a single (virtual) CPU implementation.
> 
> Additionally, the wording of I_NXHPS seems to suggest that EL2 handling
> of MOPS exceptions is only expected in certain circumstances where EL1 is
> incapable of handling an exception. Is the unwritten expectation then
> that EL1 software should tolerate 'unexpected' MOPS exceptions from EL1
> and EL0, even if EL1 did not migrate the PE context?
> 
> Perhaps I'm being pedantic, but I'd really like for there to be some
> documentation that suggests MOPS exceptions can happen due to context
> migration done by a higher EL as that is the only option in the context
> of virtualization.

That's a good point. This shouldn't affect Linux guests as Linux is
always able to handle a MOPS exception coming from EL0. But it would
affect any non-Linux guest that pins all its EL0 tasks and doesn't
implement a handler. It's not clear to me what the expectation for
guests is, I'll ask the architects to clarify and get back to you.

Thanks for the feedback!

Kristina

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 0/2] KVM: arm64: Support for Arm v8.8 memcpy instructions in KVM guests

[Oliver Upton]

On Thu, Sep 28, 2023 at 05:55:39PM +0100, Kristina Martsenko wrote:

[...]

> > Perhaps I'm being pedantic, but I'd really like for there to be some
> > documentation that suggests MOPS exceptions can happen due to context
> > migration done by a higher EL as that is the only option in the context
> > of virtualization.
> 
> That's a good point. This shouldn't affect Linux guests as Linux is
> always able to handle a MOPS exception coming from EL0. But it would
> affect any non-Linux guest that pins all its EL0 tasks and doesn't
> implement a handler. It's not clear to me what the expectation for
> guests is, I'll ask the architects to clarify and get back to you.

That'd be excellent, thanks! All I'm looking for is something to point
folks at if/when they complain about MOPS behavior in KVM guests.

-- 
Thanks,
Oliver

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 1/2] KVM: arm64: Add handler for MOPS exceptions

[Marc Zyngier]

On Wed, 27 Sep 2023 09:28:20 +0100,
Oliver Upton <oliver.upton@linux.dev> wrote:
> 
> On Mon, Sep 25, 2023 at 04:16:06PM +0100, Kristina Martsenko wrote:
> 
> [...]
> 
> > > What is the rationale for advancing the state machine? Shouldn't we
> > > instead return to the guest and immediately get the SS exception,
> > > which in turn gets reported to userspace? Is it because we rollback
> > > the PC to a previous instruction?
> > 
> > Yes, because we rollback the PC to the prologue instruction. We advance the
> > state machine so that the SS exception is taken immediately upon returning to
> > the guest at the prologue instruction. If we didn't advance it then we would
> > return to the guest, execute the prologue instruction, and then take the SS
> > exception on the middle instruction. Which would be surprising as userspace
> > would see the middle and epilogue instructions executed multiple times but not
> > the prologue.
> 
> I agree with Kristina that taking the SS exception on the prologue is
> likely the best course of action. Especially since it matches the
> behavior of single-stepping an EL0 MOPS sequence with an intervening CPU
> migration.
> 
> This behavior might throw an EL1 that single-steps itself for a loop,
> but I think it is impossible for a hypervisor to hide the consequences
> of vCPU migration with MOPS in the first place.
> 
> Marc, I'm guessing you were most concerned about the former case where
> the VMM was debugging the guest. Is there something you're concerned
> about I missed?

My concern is not only the VMM, but any userspace that perform
single-stepping. Imagine the debugger tracks PC by itself, and simply
increments it by 4 on a non-branch, non-fault instruction.

Move the vcpu or the userspace around, rewind PC, and now the debugger
is out of whack with what is executing. While I agree that there is
not much a hypervisor can do about that, I'm a bit worried that we are
going to break existing SW with this.

Now the obvious solution is "don't do that"...

	M.

-- 
Without deviation from the norm, progress is not possible.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 0/2] KVM: arm64: Support for Arm v8.8 memcpy instructions in KVM guests

[Marc Zyngier]

On Thu, 28 Sep 2023 17:55:39 +0100,
Kristina Martsenko <kristina.martsenko@arm.com> wrote:
> 
> On 27/09/2023 07:00, Oliver Upton wrote:
> > Hi Kristina,
> 
> Hi Oliver,
> 
> > 
> > On Fri, Sep 22, 2023 at 12:25:06PM +0100, Kristina Martsenko wrote:
> >> Hi,
> >>
> >> This is v2 of the series to allow using the new Arm memory copy instructions
> >> in KVM guests. See v1 for more information [1].
> > 
> > 
> > Thanks for sending out the series. I've been thinking about what the
> > architecture says for MOPS, and I wonder if what's currently in the
> > Arm ARM is clear enough for EL1 software to be written robustly.
> > 
> > While HCRX_EL2.MCE2 allows the hypervisor to intervene on MOPS
> > exceptions from EL1, there's no such control for EL0. So when vCPU
> > migration occurs EL1 could get an unexpected MOPS exception, even for a
> > process that was pinned to a single (virtual) CPU implementation.
> > 
> > Additionally, the wording of I_NXHPS seems to suggest that EL2 handling
> > of MOPS exceptions is only expected in certain circumstances where EL1 is
> > incapable of handling an exception. Is the unwritten expectation then
> > that EL1 software should tolerate 'unexpected' MOPS exceptions from EL1
> > and EL0, even if EL1 did not migrate the PE context?
> > 
> > Perhaps I'm being pedantic, but I'd really like for there to be some
> > documentation that suggests MOPS exceptions can happen due to context
> > migration done by a higher EL as that is the only option in the context
> > of virtualization.
> 
> That's a good point. This shouldn't affect Linux guests as Linux is
> always able to handle a MOPS exception coming from EL0. But it would
> affect any non-Linux guest that pins all its EL0 tasks and doesn't
> implement a handler. It's not clear to me what the expectation for
> guests is, I'll ask the architects to clarify and get back to you.

My understanding is that MCE2 should always be set if the hypervisor
can migrate vcpus across implementations behind EL1's back, and that
in this context, EL1 never sees such an exception.

I guess the only case where we could let EL1 handle such exception is
by only setting MCE2 on the first entry into the guest after a vcpu
migration (and clear it after that). Is it worth the effort?
Absolutely not.

	M.

-- 
Without deviation from the norm, progress is not possible.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 0/2] KVM: arm64: Support for Arm v8.8 memcpy instructions in KVM guests

[Kristina Martsenko]

On 29/09/2023 10:29, Marc Zyngier wrote:
> On Thu, 28 Sep 2023 17:55:39 +0100,
> Kristina Martsenko <kristina.martsenko@arm.com> wrote:
>>
>> On 27/09/2023 07:00, Oliver Upton wrote:
>>>
>>> On Fri, Sep 22, 2023 at 12:25:06PM +0100, Kristina Martsenko wrote:
>>>> Hi,
>>>>
>>>> This is v2 of the series to allow using the new Arm memory copy instructions
>>>> in KVM guests. See v1 for more information [1].
>>>
>>>
>>> Thanks for sending out the series. I've been thinking about what the
>>> architecture says for MOPS, and I wonder if what's currently in the
>>> Arm ARM is clear enough for EL1 software to be written robustly.
>>>
>>> While HCRX_EL2.MCE2 allows the hypervisor to intervene on MOPS
>>> exceptions from EL1, there's no such control for EL0. So when vCPU
>>> migration occurs EL1 could get an unexpected MOPS exception, even for a
>>> process that was pinned to a single (virtual) CPU implementation.
>>>
>>> Additionally, the wording of I_NXHPS seems to suggest that EL2 handling
>>> of MOPS exceptions is only expected in certain circumstances where EL1 is
>>> incapable of handling an exception. Is the unwritten expectation then
>>> that EL1 software should tolerate 'unexpected' MOPS exceptions from EL1
>>> and EL0, even if EL1 did not migrate the PE context?
>>>
>>> Perhaps I'm being pedantic, but I'd really like for there to be some
>>> documentation that suggests MOPS exceptions can happen due to context
>>> migration done by a higher EL as that is the only option in the context
>>> of virtualization.
>>
>> That's a good point. This shouldn't affect Linux guests as Linux is
>> always able to handle a MOPS exception coming from EL0. But it would
>> affect any non-Linux guest that pins all its EL0 tasks and doesn't
>> implement a handler. It's not clear to me what the expectation for
>> guests is, I'll ask the architects to clarify and get back to you.
> 
> My understanding is that MCE2 should always be set if the hypervisor
> can migrate vcpus across implementations behind EL1's back, and that
> in this context, EL1 never sees such an exception.

Notice that MCE2 only traps exceptions from EL1, not from EL0.
Exceptions from EL0 always go to EL1. Even if MCE2 is always set, EL1
will see the exception when the hypervisor migrates the vcpu while the
vcpu is executing a MOPS instruction in EL0.

Thanks,
Kristina


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 1/2] KVM: arm64: Add handler for MOPS exceptions

[Kristina Martsenko]

On 29/09/2023 10:23, Marc Zyngier wrote:
> On Wed, 27 Sep 2023 09:28:20 +0100,
> Oliver Upton <oliver.upton@linux.dev> wrote:
>>
>> On Mon, Sep 25, 2023 at 04:16:06PM +0100, Kristina Martsenko wrote:
>>
>> [...]
>>
>>>> What is the rationale for advancing the state machine? Shouldn't we
>>>> instead return to the guest and immediately get the SS exception,
>>>> which in turn gets reported to userspace? Is it because we rollback
>>>> the PC to a previous instruction?
>>>
>>> Yes, because we rollback the PC to the prologue instruction. We advance the
>>> state machine so that the SS exception is taken immediately upon returning to
>>> the guest at the prologue instruction. If we didn't advance it then we would
>>> return to the guest, execute the prologue instruction, and then take the SS
>>> exception on the middle instruction. Which would be surprising as userspace
>>> would see the middle and epilogue instructions executed multiple times but not
>>> the prologue.
>>
>> I agree with Kristina that taking the SS exception on the prologue is
>> likely the best course of action. Especially since it matches the
>> behavior of single-stepping an EL0 MOPS sequence with an intervening CPU
>> migration.
>>
>> This behavior might throw an EL1 that single-steps itself for a loop,
>> but I think it is impossible for a hypervisor to hide the consequences
>> of vCPU migration with MOPS in the first place.
>>
>> Marc, I'm guessing you were most concerned about the former case where
>> the VMM was debugging the guest. Is there something you're concerned
>> about I missed?
> 
> My concern is not only the VMM, but any userspace that perform
> single-stepping. Imagine the debugger tracks PC by itself, and simply
> increments it by 4 on a non-branch, non-fault instruction.
> 
> Move the vcpu or the userspace around, rewind PC, and now the debugger
> is out of whack with what is executing. While I agree that there is
> not much a hypervisor can do about that, I'm a bit worried that we are
> going to break existing SW with this.
> 
> Now the obvious solution is "don't do that"...

If the debugger can handle the PC changing on branching or faulting
instructions, then why can't it handle it on MOPS instructions? Wouldn't
such a debugger need to be updated any time the architecture adds new
branching or faulting instructions? What's different here?

Confused,
Kristina

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 1/2] KVM: arm64: Add handler for MOPS exceptions

[Marc Zyngier]

On Mon, 02 Oct 2023 15:06:33 +0100,
Kristina Martsenko <kristina.martsenko@arm.com> wrote:
> 
> On 29/09/2023 10:23, Marc Zyngier wrote:
> > On Wed, 27 Sep 2023 09:28:20 +0100,
> > Oliver Upton <oliver.upton@linux.dev> wrote:
> >>
> >> On Mon, Sep 25, 2023 at 04:16:06PM +0100, Kristina Martsenko wrote:
> >>
> >> [...]
> >>
> >>>> What is the rationale for advancing the state machine? Shouldn't we
> >>>> instead return to the guest and immediately get the SS exception,
> >>>> which in turn gets reported to userspace? Is it because we rollback
> >>>> the PC to a previous instruction?
> >>>
> >>> Yes, because we rollback the PC to the prologue instruction. We advance the
> >>> state machine so that the SS exception is taken immediately upon returning to
> >>> the guest at the prologue instruction. If we didn't advance it then we would
> >>> return to the guest, execute the prologue instruction, and then take the SS
> >>> exception on the middle instruction. Which would be surprising as userspace
> >>> would see the middle and epilogue instructions executed multiple times but not
> >>> the prologue.
> >>
> >> I agree with Kristina that taking the SS exception on the prologue is
> >> likely the best course of action. Especially since it matches the
> >> behavior of single-stepping an EL0 MOPS sequence with an intervening CPU
> >> migration.
> >>
> >> This behavior might throw an EL1 that single-steps itself for a loop,
> >> but I think it is impossible for a hypervisor to hide the consequences
> >> of vCPU migration with MOPS in the first place.
> >>
> >> Marc, I'm guessing you were most concerned about the former case where
> >> the VMM was debugging the guest. Is there something you're concerned
> >> about I missed?
> > 
> > My concern is not only the VMM, but any userspace that perform
> > single-stepping. Imagine the debugger tracks PC by itself, and simply
> > increments it by 4 on a non-branch, non-fault instruction.
> > 
> > Move the vcpu or the userspace around, rewind PC, and now the debugger
> > is out of whack with what is executing. While I agree that there is
> > not much a hypervisor can do about that, I'm a bit worried that we are
> > going to break existing SW with this.
> > 
> > Now the obvious solution is "don't do that"...
> 
> If the debugger can handle the PC changing on branching or faulting
> instructions, then why can't it handle it on MOPS instructions? Wouldn't
> such a debugger need to be updated any time the architecture adds new
> branching or faulting instructions? What's different here?

What is different is that we *go back* in the instruction stream,
which is a first. I'm not saying that the debugger I describe above
would be a very clever piece of SW, quite the opposite. But the way
the architecture works results in some interesting side-effects, and
I'm willing to bet that some SW will break (rr?).

But again, asymmetric systems are such a bad idea that I can't say I
care.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 0/2] KVM: arm64: Support for Arm v8.8 memcpy instructions in KVM guests

[Marc Zyngier]

On Fri, 29 Sep 2023 15:51:32 +0100,
Kristina Martsenko <kristina.martsenko@arm.com> wrote:
> 
> On 29/09/2023 10:29, Marc Zyngier wrote:
> > On Thu, 28 Sep 2023 17:55:39 +0100,
> > Kristina Martsenko <kristina.martsenko@arm.com> wrote:
> >>
> >> On 27/09/2023 07:00, Oliver Upton wrote:
> >>>
> >>> On Fri, Sep 22, 2023 at 12:25:06PM +0100, Kristina Martsenko wrote:
> >>>> Hi,
> >>>>
> >>>> This is v2 of the series to allow using the new Arm memory copy instructions
> >>>> in KVM guests. See v1 for more information [1].
> >>>
> >>>
> >>> Thanks for sending out the series. I've been thinking about what the
> >>> architecture says for MOPS, and I wonder if what's currently in the
> >>> Arm ARM is clear enough for EL1 software to be written robustly.
> >>>
> >>> While HCRX_EL2.MCE2 allows the hypervisor to intervene on MOPS
> >>> exceptions from EL1, there's no such control for EL0. So when vCPU
> >>> migration occurs EL1 could get an unexpected MOPS exception, even for a
> >>> process that was pinned to a single (virtual) CPU implementation.
> >>>
> >>> Additionally, the wording of I_NXHPS seems to suggest that EL2 handling
> >>> of MOPS exceptions is only expected in certain circumstances where EL1 is
> >>> incapable of handling an exception. Is the unwritten expectation then
> >>> that EL1 software should tolerate 'unexpected' MOPS exceptions from EL1
> >>> and EL0, even if EL1 did not migrate the PE context?
> >>>
> >>> Perhaps I'm being pedantic, but I'd really like for there to be some
> >>> documentation that suggests MOPS exceptions can happen due to context
> >>> migration done by a higher EL as that is the only option in the context
> >>> of virtualization.
> >>
> >> That's a good point. This shouldn't affect Linux guests as Linux is
> >> always able to handle a MOPS exception coming from EL0. But it would
> >> affect any non-Linux guest that pins all its EL0 tasks and doesn't
> >> implement a handler. It's not clear to me what the expectation for
> >> guests is, I'll ask the architects to clarify and get back to you.
> > 
> > My understanding is that MCE2 should always be set if the hypervisor
> > can migrate vcpus across implementations behind EL1's back, and that
> > in this context, EL1 never sees such an exception.
> 
> Notice that MCE2 only traps exceptions from EL1, not from EL0.
> Exceptions from EL0 always go to EL1. Even if MCE2 is always set, EL1
> will see the exception when the hypervisor migrates the vcpu while the
> vcpu is executing a MOPS instruction in EL0.

Ah, good point. I stand corrected.

	M.

-- 
Without deviation from the norm, progress is not possible.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 1/2] KVM: arm64: Add handler for MOPS exceptions

[Catalin Marinas]

On Mon, Oct 02, 2023 at 03:55:33PM +0100, Marc Zyngier wrote:
> On Mon, 02 Oct 2023 15:06:33 +0100,
> Kristina Martsenko <kristina.martsenko@arm.com> wrote:
> > On 29/09/2023 10:23, Marc Zyngier wrote:
> > > On Wed, 27 Sep 2023 09:28:20 +0100,
> > > Oliver Upton <oliver.upton@linux.dev> wrote:
> > >> On Mon, Sep 25, 2023 at 04:16:06PM +0100, Kristina Martsenko wrote:
> > >>>> What is the rationale for advancing the state machine? Shouldn't we
> > >>>> instead return to the guest and immediately get the SS exception,
> > >>>> which in turn gets reported to userspace? Is it because we rollback
> > >>>> the PC to a previous instruction?
> > >>>
> > >>> Yes, because we rollback the PC to the prologue instruction. We advance the
> > >>> state machine so that the SS exception is taken immediately upon returning to
> > >>> the guest at the prologue instruction. If we didn't advance it then we would
> > >>> return to the guest, execute the prologue instruction, and then take the SS
> > >>> exception on the middle instruction. Which would be surprising as userspace
> > >>> would see the middle and epilogue instructions executed multiple times but not
> > >>> the prologue.
> > >>
> > >> I agree with Kristina that taking the SS exception on the prologue is
> > >> likely the best course of action. Especially since it matches the
> > >> behavior of single-stepping an EL0 MOPS sequence with an intervening CPU
> > >> migration.
> > >>
> > >> This behavior might throw an EL1 that single-steps itself for a loop,
> > >> but I think it is impossible for a hypervisor to hide the consequences
> > >> of vCPU migration with MOPS in the first place.
> > >>
> > >> Marc, I'm guessing you were most concerned about the former case where
> > >> the VMM was debugging the guest. Is there something you're concerned
> > >> about I missed?
> > > 
> > > My concern is not only the VMM, but any userspace that perform
> > > single-stepping. Imagine the debugger tracks PC by itself, and simply
> > > increments it by 4 on a non-branch, non-fault instruction.
> > > 
> > > Move the vcpu or the userspace around, rewind PC, and now the debugger
> > > is out of whack with what is executing. While I agree that there is
> > > not much a hypervisor can do about that, I'm a bit worried that we are
> > > going to break existing SW with this.
> > > 
> > > Now the obvious solution is "don't do that"...
> > 
> > If the debugger can handle the PC changing on branching or faulting
> > instructions, then why can't it handle it on MOPS instructions? Wouldn't
> > such a debugger need to be updated any time the architecture adds new
> > branching or faulting instructions? What's different here?
> 
> What is different is that we *go back* in the instruction stream,
> which is a first. I'm not saying that the debugger I describe above
> would be a very clever piece of SW, quite the opposite. But the way
> the architecture works results in some interesting side-effects, and
> I'm willing to bet that some SW will break (rr?).

The way the architecture works, either with or without Kristina's
single-step change, a debugger would get confused. At least for EL0, I
find the proposed (well, upstreamed) approach more predictable - it
always restarts from the prologue in case of migration between CPUs with
different MOPS implementation (which is not just theoretical AFAIK).
It's more like these three instructions are a bigger CISC one ;) (though
the CPU can step through its parts).

A more transparent approach would have been to fully emulate the
instructions in the kernel and advance the PC as expected but I don't
think that's even possible. An implementation may decide to leave some
bytes to be copied by the epilogue but we can't know that in software,
it's a microarchitecture thing.

There is the case of EL1 debugging itself (kgdb) and it triggers a MOPS
exception to EL2. It would look weird for the guest but I guess the only
other option is to disable MCE2 and let EL1 handle the mismatch MOPS
option itself (assuming it knows how to; it should be fine for Linux). I
think I still prefer Kristina's proposal for KVM as more generic, with
the downside of breaking less usual cases like the kernel
single-stepping itself.

-- 
Catalin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 1/2] KVM: arm64: Add handler for MOPS exceptions

[Marc Zyngier]

On Tue, 03 Oct 2023 15:29:42 +0100,
Catalin Marinas <catalin.marinas@arm.com> wrote:
> 
> The way the architecture works, either with or without Kristina's
> single-step change, a debugger would get confused. At least for EL0, I
> find the proposed (well, upstreamed) approach more predictable - it
> always restarts from the prologue in case of migration between CPUs with
> different MOPS implementation (which is not just theoretical AFAIK).
> It's more like these three instructions are a bigger CISC one ;) (though
> the CPU can step through its parts).
> 
> A more transparent approach would have been to fully emulate the
> instructions in the kernel and advance the PC as expected but I don't
> think that's even possible. An implementation may decide to leave some
> bytes to be copied by the epilogue but we can't know that in software,
> it's a microarchitecture thing.
> 
> There is the case of EL1 debugging itself (kgdb) and it triggers a MOPS
> exception to EL2. It would look weird for the guest but I guess the only
> other option is to disable MCE2 and let EL1 handle the mismatch MOPS
> option itself (assuming it knows how to; it should be fine for Linux). I
> think I still prefer Kristina's proposal for KVM as more generic, with
> the downside of breaking less usual cases like the kernel
> single-stepping itself.

I don't disagree at all.

My issue isn't with Kristina's patches, which are absolutely fine. It
has more to do with the shape of the FEAT_MOPS extension itself, which
exposes uarch details to SW instead of abstracting them.

But I've now ranted about it for close to two weeks, and it is time
for me to move on... ;-)

	M.

-- 
Without deviation from the norm, progress is not possible.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 0/2] KVM: arm64: Support for Arm v8.8 memcpy instructions in KVM guests

[Marc Zyngier]

On Fri, 22 Sep 2023 12:25:06 +0100,
Kristina Martsenko <kristina.martsenko@arm.com> wrote:
> 
> Hi,
> 
> This is v2 of the series to allow using the new Arm memory copy instructions
> in KVM guests. See v1 for more information [1].
> 
> Changes in v2:
>  - Dropped HCRX_EL2 vcpu field
>  - Rebased onto v6.6-rc2

FWIW, and despite my misgivings about the architecture:

Reviewed-by: Marc Zyngier <maz@kernel.org>

	M.

-- 
Without deviation from the norm, progress is not possible.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 0/2] KVM: arm64: Support for Arm v8.8 memcpy instructions in KVM guests

[Oliver Upton]

On Fri, 22 Sep 2023 12:25:06 +0100, Kristina Martsenko wrote:
> This is v2 of the series to allow using the new Arm memory copy instructions
> in KVM guests. See v1 for more information [1].
> 
> Changes in v2:
>  - Dropped HCRX_EL2 vcpu field
>  - Rebased onto v6.6-rc2
> 
> [...]

Applied to kvmarm/next, thanks!

[1/2] KVM: arm64: Add handler for MOPS exceptions
      https://git.kernel.org/kvmarm/kvmarm/c/17b8ac23488b
[2/2] KVM: arm64: Expose MOPS instructions to guests
      https://git.kernel.org/kvmarm/kvmarm/c/a24015b6cc66

--
Best,
Oliver

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel