Re: [Qemu-devel] [Query] Live Migration between machines with different processor ids

["Dr. David Alan Gilbert" <dgilbert@redhat.com>]

* Jaggi, Manish (Manish.Jaggi@cavium.com) wrote:

> Just to add what happens in ARM64 case, qemu running on Machine A sends cpu state information to Machine B.
> This state contains MIDR value, and so Processor ID value is compared in KVM and not in qemu (correcting myself).
> 
> IIRC, Peter/Eric please point if there is something incorrect in the below flow...
> 
> (Machine B)
> target/arm/machine.c: cpu_post_load()
> 		- updates cpu->cpreg_values[i] : which includes MIDR (processor ID register)
> 
> 		- calls write_list_to_kvmstate(cpu, KVM_PUT_FULL_STATE)
> 
> 				target/arm/kvm.c: write_list_to_kvmstate
> 				- calls => kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, &r);
> 
> 					=> and it eventually lands up IIRC in Linux code in 
> 
> 							=> arch/arm64/kvm/sys_regs.c : set_invariant_sys_reg(u64 id, void __user *uaddr)
> 							 	/* This is what we mean by invariant: you can't change it. */
> 								if (r->val != val)
> 									return -EINVAL;
> 								Note: MIDR_EL1 is invariant register.
> result: Migration fails on Machine B.
> 
> A few points:
> - qemu on arm64 is invoked with -machine virt and -cpu as host. So we don't explicitly define which cpu. 

Note that even on x86 we don't guarantee as much about '-cpu host', what
we expect to work for migration is that if you pick a   '-cpu amodel'
and both hosts support the feature flags required by 'amodel' then it
should work.

> - In case Machine A and Machine B have almost same Core and the delta may-not have any effect on qemu operation, migration should work by just looking into whitelist.
> whitelist can be given as a parameter for qemu on machine B.
> 
> qemu-system-aarch64 -whitelist <ids separated by commas>
> 
> (This is my proposal)
> 
> - So in cpu_post_load (Machine B) qemu can lookup whitelist and replace the MIDR with the one at Machine B. 
> Sounds good?
> 
> - Juan raised a point about clock speed, I am not sure it will have any effect on arm since qemu is run with -cpu host param.
> I could be wrong here, Peter/Eric can you please correct me...

Clock speed is only really a problem for things like timestamp counters;
some cores let you scale them; for those that don't then yes it's a bit
odd.

Dave

> -Thanks
> Manish
> 
> 
> 
> > Dave
> >> - is providing a -mirate-compat-whitelist <file> option for arm only looks good?
> >> this option can be added in A1/A2 qemu command, so it would be upstream / downstream agnostic.
> > 
> >>> 
> >>> I am sorry to not be able to tell you that this is an easy problem.
> >>> 
> >>> Later, Juan.
> >> 
> > --
> > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK