Re: [Qemu-devel] [RFC PATCH] target-ppc: enable migration within the same CPU family

[Michael Mueller <mimu@linux.vnet.ibm.com>]

On Fri, 11 Apr 2014 02:03:29 +1000
Alexey Kardashevskiy <aik@ozlabs.ru> wrote:

> On 04/08/2014 08:32 PM, Michael Mueller wrote:
> > On Tue, 08 Apr 2014 20:04:42 +1000
> > Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> > 
> >> On 04/08/2014 07:47 PM, Michael Mueller wrote:
> >>> On Tue, 08 Apr 2014 11:23:14 +1000
> >>> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> >>>
> >>>> On 04/08/2014 04:53 AM, Andreas Färber wrote:
> >>>>> Am 07.04.2014 05:27, schrieb Alexey Kardashevskiy:
> >>>>>> On 04/04/2014 11:28 PM, Alexander Graf wrote:
> >>>>>>> On 04/04/2014 07:17 AM, Alexey Kardashevskiy wrote:
> >>>>>>>> On 03/24/2014 04:28 PM, Alexey Kardashevskiy wrote:
> >>>>>>>>> Currently only migration fails if CPU version is different even a bit.
> >>>>>>>>> For example, migration from POWER7 v2.0 to POWER7 v2.1 fails because of
> >>>>>>>>> that. Since there is no difference between CPU versions which could
> >>>>>>>>> affect migration stream, we can safely enable it.
> >>>>>>>>>
> >>>>>>>>> This adds a helper to find the closest POWERPC family class (i.e. first
> >>>>>>>>> abstract class in hierarchy).
> >>>>>>>>>
> >>>>>>>>> This replaces VMSTATE_UINTTL_EQUAL statement with a custom handler which
> >>>>>>>>> checks if the source and destination CPUs belong to the same family and
> >>>>>>>>> fails if they are not.
> >>>>>>>>>
> >>>>>>>>> This adds a PVR reset to the default value as it will be overwritten
> >>>>>>>>> by VMSTATE_UINTTL_ARRAY(env.spr, PowerPCCPU, 1024).
> >>>>>>>>>
> >>>>>>>>> Since the actual migration format is not changed by this patch,
> >>>>>>>>> @version_id of vmstate_ppc_cpu does not have to be changed either.
> >>>>>>>>>
> >>>>>>>>> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> >>>>>>>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> >>>>>>>>
> >>>>>>>> Ping?
> >>>>>>>
> >>>>>>> Can't we just always allow migration to succeed? It's a problem of the tool
> >>>>>>> stack above if it allows migration to an incompatible host, no?
> >>>>>>
> >>>>>> This is not how libvirt works. It simply sends the source XML, reconstructs
> >>>>>> a guest on the destination side and then migrates. hoping that the
> >>>>>> migration will fail is something (which only QEMU has knowledge of) is
> >>>>>> incompatible. The new guest will start with "-cpu host" (as the source) but
> >>>>>> it will create diffrent CPU class and do different things. If we do not
> >>>>>> check PVR (and cpu_dt_id and chip_id - the latter is coming soon) and
> >>>>>> migrate power8->power7, we can easily get a broken guest.
> >>>>>
> >>>>> The response is very simple: -cpu host is not supported for migration.
> >>>>> Same as for x86 hosts.
> >>>>
> >>>> Is there any good reason to limit ourselves on POWERPC?
> >>>>
> >>>>> As you say, the domain config is transferred by libvirt:
> >>>>> If you use -cpu POWER7, you can migrate from POWER7 to POWER8 and back;
> >>>>> if you use -cpu POWER8, you can only migrate on POWER8.
> >>>>
> >>>> -cpu other that "host" is not supported by HV KVM, only "compat" which
> >>>> upstream QEMU does not have yet. So you are saying that the migration is
> >>>> not supported by upstream QEMU for at least SPAPR. Well, ok, it is dead
> >>>> anyway so I am fine :)
> >>>>
> >>>
> >>> With s390x we have a similar situation. Thus we came up with a mechanism to limit
> >>> the CPU functionality of a possible target system. Our patch implements CPU models
> >>> based on TYPE and GA like 2817-ga1, etc. (GA represents a CPU facility set and an IBC
> >>> value (Instruction Blocking Control, reduces the instruction set to the requested
> >>> level)) When a guest is started, it receives its CPU model by means of option -cpu.
> >>> "host" equates the configuration of the current system. We implemented "query-cpu-model"
> >>> returning the actual model, here maybe { name: "2817-ga1" }. To find a suitable
> >>> migration target in a remote CEC, libvirt has to "query-cpu-definitions" returning a
> >>> list of models supported by the target system "{{name: "2827-ga2"}, {name: "2827-ga1"},
> >>> {name: "2817-ga2"},...]. A match means the system is suitable and can be used
> >>> as migration target.
> >>
> >> Sorry, I do not follow you. You hacked libvirt to run the destination QEMU
> >> with a specific CPU model? Or it is in QEMU? Where? What I see now is this:
> >>
> >> static const VMStateDescription vmstate_s390_cpu = {
> >>     .name = "cpu",
> >>     .unmigratable = 1,
> >> };
> >>
> >> Does not look like it supports migration :) Thanks!
> >>
> > 
> > The code you're missing is not upstream yet. 
> 
> 
> Is it in some maillist or git (IBM internal?)? I just want to look at some
> details. Thanks!

I had sent out an very early version of the user space patch already:

http://lists.nongnu.org/archive/html/qemu-devel/2013-10/msg00262.html

It does not contain the IBC part because it wasn't legally cleared up to then.

But as you are with IBM I think we can give you access to our internal GIT. I have to
check with Christian Borntraeger tomorrow on how we can most easily handle it.  

> 
> 
> > The s390x guest can be migrated in the meantime.
> > Yes, libvirt currently gets an extension to be able to identify and startup suitable migration
> > targets for s390x on behalf of the mentioned qemu cpu model. BTW can you point me to the above
> > mentioned SPAPR stuff...
> > 
> > Michael
> > 
> 
>