Re: [PATCH] KVM: riscv: Power on secondary vCPUs from migration

From: Andrew Jones <ajones@ventanamicro.com>
To: "Radim Krčmář" <rkrcmar@ventanamicro.com>
Cc: Jinyu Tang <tjytimi@163.com>, Anup Patel <anup@brainfault.org>,
	 Atish Patra <atish.patra@linux.dev>,
	Conor Dooley <conor.dooley@microchip.com>,
	 Yong-Xuan Wang <yongxuan.wang@sifive.com>,
	Paul Walmsley <paul.walmsley@sifive.com>,
	 Nutty Liu <nutty.liu@hotmail.com>,
	Tianshun Sun <stsmail163@163.com>,
	kvm@vger.kernel.org,  kvm-riscv@lists.infradead.org,
	linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org,
	 linux-riscv <linux-riscv-bounces@lists.infradead.org>
Subject: Re: [PATCH] KVM: riscv: Power on secondary vCPUs from migration
Date: Mon, 15 Sep 2025 13:54:25 -0500	[thread overview]
Message-ID: <20250915-23f31d3577fe91c7d9944b1f@orel> (raw)
In-Reply-To: <DCTFU1UCDSZZ.3J6L3T6TYTELM@ventanamicro.com>

On Mon, Sep 15, 2025 at 04:19:21PM +0200, Radim Krčmář wrote:
> 2025-09-15T20:23:34+08:00, Jinyu Tang <tjytimi@163.com>:
> > The current logic keeps all secondary VCPUs powered off on their
> > first run in kvm_arch_vcpu_postcreate(), relying on the boot VCPU 
> > to wake them up by sbi call. This is correct for a fresh VM start,
> > where VCPUs begin execution at the bootaddress (0x80000000).
> >
> > However, this behavior is not suitable for VCPUs that are being
> > restored from a state (e.g., during migration resume or snapshot
> > load). These VCPUs have a saved program counter (sepc). Forcing
> > them to wait for a wake-up from the boot VCPU, which may not
> > happen or may happen incorrectly, leaves them in a stuck state
> > when using Qemu to migration if smp is larger than one.
> >
> > So check a cold start and a warm resumption by the value of the 
> > guest's sepc register. If the VCPU is running for the first time 
> > *and* its sepc is not the hardware boot address, it indicates a 
> > resumed vCPU that must be powered on immediately to continue 
> > execution from its saved context.
> >
> > Signed-off-by: Jinyu Tang <tjytimi@163.com>
> > Tested-by: Tianshun Sun <stsmail163@163.com>
> > ---
> 
> I don't like this approach.  Userspace controls the state of the VM, and
> KVM shouldn't randomly change the state that userspace wants.
> 
> > diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
> > @@ -867,8 +867,16 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
> >  	struct kvm_cpu_trap trap;
> >  	struct kvm_run *run = vcpu->run;
> >  
> > -	if (!vcpu->arch.ran_atleast_once)
> > +	if (!vcpu->arch.ran_atleast_once) {
> >  		kvm_riscv_vcpu_setup_config(vcpu);
> > +		/*
> > +		 * For VCPUs that are resuming (e.g., from migration)
> > +		 * and not starting from the boot address, explicitly
> > +		 * power them on.
> > +		 */
> > +		if (vcpu->arch.guest_context.sepc != 0x80000000)
> 
> Offlined VCPUs are not guaranteed to have sepc == 0x80000000, so this
> patch would incorrectly wake them up.
> (Depending on vcpu->arch.ran_atleast_once is flaky at best as well.)
> 
> Please try to fix userspace instead,

Yes, and maybe it's already fixed
https://lore.kernel.org/all/20250915070811.3422578-1-xb@ultrarisc.com/

Thanks,
drew

-- 
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv