* [PATCH] KVM: riscv: Power on secondary vCPUs from migration
@ 2025-09-15 12:23 Jinyu Tang
2025-09-15 14:19 ` Radim Krčmář
0 siblings, 1 reply; 3+ messages in thread
From: Jinyu Tang @ 2025-09-15 12:23 UTC (permalink / raw)
To: Anup Patel, Atish Patra, Andrew Jones, Conor Dooley,
Yong-Xuan Wang, Paul Walmsley, Nutty Liu, Tianshun Sun
Cc: kvm, kvm-riscv, linux-riscv, linux-kernel, Jinyu Tang
The current logic keeps all secondary VCPUs powered off on their
first run in kvm_arch_vcpu_postcreate(), relying on the boot VCPU
to wake them up by sbi call. This is correct for a fresh VM start,
where VCPUs begin execution at the bootaddress (0x80000000).
However, this behavior is not suitable for VCPUs that are being
restored from a state (e.g., during migration resume or snapshot
load). These VCPUs have a saved program counter (sepc). Forcing
them to wait for a wake-up from the boot VCPU, which may not
happen or may happen incorrectly, leaves them in a stuck state
when using Qemu to migration if smp is larger than one.
So check a cold start and a warm resumption by the value of the
guest's sepc register. If the VCPU is running for the first time
*and* its sepc is not the hardware boot address, it indicates a
resumed vCPU that must be powered on immediately to continue
execution from its saved context.
Signed-off-by: Jinyu Tang <tjytimi@163.com>
Tested-by: Tianshun Sun <stsmail163@163.com>
---
arch/riscv/kvm/vcpu.c | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
index 3ebcfffaa..86aeba886 100644
--- a/arch/riscv/kvm/vcpu.c
+++ b/arch/riscv/kvm/vcpu.c
@@ -867,8 +867,16 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
struct kvm_cpu_trap trap;
struct kvm_run *run = vcpu->run;
- if (!vcpu->arch.ran_atleast_once)
+ if (!vcpu->arch.ran_atleast_once) {
kvm_riscv_vcpu_setup_config(vcpu);
+ /*
+ * For VCPUs that are resuming (e.g., from migration)
+ * and not starting from the boot address, explicitly
+ * power them on.
+ */
+ if (vcpu->arch.guest_context.sepc != 0x80000000)
+ kvm_riscv_vcpu_power_on(vcpu);
+ }
/* Mark this VCPU ran at least once */
vcpu->arch.ran_atleast_once = true;
--
2.43.0
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 3+ messages in thread* Re: [PATCH] KVM: riscv: Power on secondary vCPUs from migration
2025-09-15 12:23 [PATCH] KVM: riscv: Power on secondary vCPUs from migration Jinyu Tang
@ 2025-09-15 14:19 ` Radim Krčmář
2025-09-15 18:54 ` Andrew Jones
0 siblings, 1 reply; 3+ messages in thread
From: Radim Krčmář @ 2025-09-15 14:19 UTC (permalink / raw)
To: Jinyu Tang, Anup Patel, Atish Patra, Andrew Jones, Conor Dooley,
Yong-Xuan Wang, Paul Walmsley, Nutty Liu, Tianshun Sun
Cc: kvm, kvm-riscv, linux-riscv, linux-kernel, linux-riscv
2025-09-15T20:23:34+08:00, Jinyu Tang <tjytimi@163.com>:
> The current logic keeps all secondary VCPUs powered off on their
> first run in kvm_arch_vcpu_postcreate(), relying on the boot VCPU
> to wake them up by sbi call. This is correct for a fresh VM start,
> where VCPUs begin execution at the bootaddress (0x80000000).
>
> However, this behavior is not suitable for VCPUs that are being
> restored from a state (e.g., during migration resume or snapshot
> load). These VCPUs have a saved program counter (sepc). Forcing
> them to wait for a wake-up from the boot VCPU, which may not
> happen or may happen incorrectly, leaves them in a stuck state
> when using Qemu to migration if smp is larger than one.
>
> So check a cold start and a warm resumption by the value of the
> guest's sepc register. If the VCPU is running for the first time
> *and* its sepc is not the hardware boot address, it indicates a
> resumed vCPU that must be powered on immediately to continue
> execution from its saved context.
>
> Signed-off-by: Jinyu Tang <tjytimi@163.com>
> Tested-by: Tianshun Sun <stsmail163@163.com>
> ---
I don't like this approach. Userspace controls the state of the VM, and
KVM shouldn't randomly change the state that userspace wants.
> diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
> @@ -867,8 +867,16 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
> struct kvm_cpu_trap trap;
> struct kvm_run *run = vcpu->run;
>
> - if (!vcpu->arch.ran_atleast_once)
> + if (!vcpu->arch.ran_atleast_once) {
> kvm_riscv_vcpu_setup_config(vcpu);
> + /*
> + * For VCPUs that are resuming (e.g., from migration)
> + * and not starting from the boot address, explicitly
> + * power them on.
> + */
> + if (vcpu->arch.guest_context.sepc != 0x80000000)
Offlined VCPUs are not guaranteed to have sepc == 0x80000000, so this
patch would incorrectly wake them up.
(Depending on vcpu->arch.ran_atleast_once is flaky at best as well.)
Please try to fix userspace instead,
Thanks.
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: [PATCH] KVM: riscv: Power on secondary vCPUs from migration
2025-09-15 14:19 ` Radim Krčmář
@ 2025-09-15 18:54 ` Andrew Jones
0 siblings, 0 replies; 3+ messages in thread
From: Andrew Jones @ 2025-09-15 18:54 UTC (permalink / raw)
To: Radim Krčmář
Cc: Jinyu Tang, Anup Patel, Atish Patra, Conor Dooley, Yong-Xuan Wang,
Paul Walmsley, Nutty Liu, Tianshun Sun, kvm, kvm-riscv,
linux-riscv, linux-kernel, linux-riscv
On Mon, Sep 15, 2025 at 04:19:21PM +0200, Radim Krčmář wrote:
> 2025-09-15T20:23:34+08:00, Jinyu Tang <tjytimi@163.com>:
> > The current logic keeps all secondary VCPUs powered off on their
> > first run in kvm_arch_vcpu_postcreate(), relying on the boot VCPU
> > to wake them up by sbi call. This is correct for a fresh VM start,
> > where VCPUs begin execution at the bootaddress (0x80000000).
> >
> > However, this behavior is not suitable for VCPUs that are being
> > restored from a state (e.g., during migration resume or snapshot
> > load). These VCPUs have a saved program counter (sepc). Forcing
> > them to wait for a wake-up from the boot VCPU, which may not
> > happen or may happen incorrectly, leaves them in a stuck state
> > when using Qemu to migration if smp is larger than one.
> >
> > So check a cold start and a warm resumption by the value of the
> > guest's sepc register. If the VCPU is running for the first time
> > *and* its sepc is not the hardware boot address, it indicates a
> > resumed vCPU that must be powered on immediately to continue
> > execution from its saved context.
> >
> > Signed-off-by: Jinyu Tang <tjytimi@163.com>
> > Tested-by: Tianshun Sun <stsmail163@163.com>
> > ---
>
> I don't like this approach. Userspace controls the state of the VM, and
> KVM shouldn't randomly change the state that userspace wants.
>
> > diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
> > @@ -867,8 +867,16 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
> > struct kvm_cpu_trap trap;
> > struct kvm_run *run = vcpu->run;
> >
> > - if (!vcpu->arch.ran_atleast_once)
> > + if (!vcpu->arch.ran_atleast_once) {
> > kvm_riscv_vcpu_setup_config(vcpu);
> > + /*
> > + * For VCPUs that are resuming (e.g., from migration)
> > + * and not starting from the boot address, explicitly
> > + * power them on.
> > + */
> > + if (vcpu->arch.guest_context.sepc != 0x80000000)
>
> Offlined VCPUs are not guaranteed to have sepc == 0x80000000, so this
> patch would incorrectly wake them up.
> (Depending on vcpu->arch.ran_atleast_once is flaky at best as well.)
>
> Please try to fix userspace instead,
Yes, and maybe it's already fixed
https://lore.kernel.org/all/20250915070811.3422578-1-xb@ultrarisc.com/
Thanks,
drew
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2025-09-15 18:54 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-09-15 12:23 [PATCH] KVM: riscv: Power on secondary vCPUs from migration Jinyu Tang
2025-09-15 14:19 ` Radim Krčmář
2025-09-15 18:54 ` Andrew Jones
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).