Re: [Qemu-devel] [RFC PATCH] target-ppc: enable migration within the same CPU family

[Alexander Graf <agraf@suse.de>]

On 04/08/2014 02:19 PM, Michael Mueller wrote:
> On Tue, 08 Apr 2014 21:47:39 +1000
> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
>
>> On 04/08/2014 08:32 PM, Michael Mueller wrote:
>>> On Tue, 08 Apr 2014 20:04:42 +1000
>>> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
>>>
>>>> On 04/08/2014 07:47 PM, Michael Mueller wrote:
>>>>> On Tue, 08 Apr 2014 11:23:14 +1000
>>>>> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
>>>>>
>>>>>> On 04/08/2014 04:53 AM, Andreas Färber wrote:
>>>>>>> Am 07.04.2014 05:27, schrieb Alexey Kardashevskiy:
>>>>>>>> On 04/04/2014 11:28 PM, Alexander Graf wrote:
>>>>>>>>> On 04/04/2014 07:17 AM, Alexey Kardashevskiy wrote:
>>>>>>>>>> On 03/24/2014 04:28 PM, Alexey Kardashevskiy wrote:
>>>>>>>>>>> Currently only migration fails if CPU version is different even a bit.
>>>>>>>>>>> For example, migration from POWER7 v2.0 to POWER7 v2.1 fails because of
>>>>>>>>>>> that. Since there is no difference between CPU versions which could
>>>>>>>>>>> affect migration stream, we can safely enable it.
>>>>>>>>>>>
>>>>>>>>>>> This adds a helper to find the closest POWERPC family class (i.e. first
>>>>>>>>>>> abstract class in hierarchy).
>>>>>>>>>>>
>>>>>>>>>>> This replaces VMSTATE_UINTTL_EQUAL statement with a custom handler which
>>>>>>>>>>> checks if the source and destination CPUs belong to the same family and
>>>>>>>>>>> fails if they are not.
>>>>>>>>>>>
>>>>>>>>>>> This adds a PVR reset to the default value as it will be overwritten
>>>>>>>>>>> by VMSTATE_UINTTL_ARRAY(env.spr, PowerPCCPU, 1024).
>>>>>>>>>>>
>>>>>>>>>>> Since the actual migration format is not changed by this patch,
>>>>>>>>>>> @version_id of vmstate_ppc_cpu does not have to be changed either.
>>>>>>>>>>>
>>>>>>>>>>> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
>>>>>>>>>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
>>>>>>>>>> Ping?
>>>>>>>>> Can't we just always allow migration to succeed? It's a problem of the tool
>>>>>>>>> stack above if it allows migration to an incompatible host, no?
>>>>>>>> This is not how libvirt works. It simply sends the source XML, reconstructs
>>>>>>>> a guest on the destination side and then migrates. hoping that the
>>>>>>>> migration will fail is something (which only QEMU has knowledge of) is
>>>>>>>> incompatible. The new guest will start with "-cpu host" (as the source) but
>>>>>>>> it will create diffrent CPU class and do different things. If we do not
>>>>>>>> check PVR (and cpu_dt_id and chip_id - the latter is coming soon) and
>>>>>>>> migrate power8->power7, we can easily get a broken guest.
>>>>>>> The response is very simple: -cpu host is not supported for migration.
>>>>>>> Same as for x86 hosts.
>>>>>> Is there any good reason to limit ourselves on POWERPC?
>>>>>>
>>>>>>> As you say, the domain config is transferred by libvirt:
>>>>>>> If you use -cpu POWER7, you can migrate from POWER7 to POWER8 and back;
>>>>>>> if you use -cpu POWER8, you can only migrate on POWER8.
>>>>>> -cpu other that "host" is not supported by HV KVM, only "compat" which
>>>>>> upstream QEMU does not have yet. So you are saying that the migration is
>>>>>> not supported by upstream QEMU for at least SPAPR. Well, ok, it is dead
>>>>>> anyway so I am fine :)
>>>>>>
>>>>> With s390x we have a similar situation. Thus we came up with a mechanism to limit
>>>>> the CPU functionality of a possible target system. Our patch implements CPU models
>>>>> based on TYPE and GA like 2817-ga1, etc. (GA represents a CPU facility set and an IBC
>>>>> value (Instruction Blocking Control, reduces the instruction set to the requested
>>>>> level)) When a guest is started, it receives its CPU model by means of option -cpu.
>>>>> "host" equates the configuration of the current system. We implemented "query-cpu-model"
>>>>> returning the actual model, here maybe { name: "2817-ga1" }. To find a suitable
>>>>> migration target in a remote CEC, libvirt has to "query-cpu-definitions" returning a
>>>>> list of models supported by the target system "{{name: "2827-ga2"}, {name: "2827-ga1"},
>>>>> {name: "2817-ga2"},...]. A match means the system is suitable and can be used
>>>>> as migration target.
>>>> Sorry, I do not follow you. You hacked libvirt to run the destination QEMU
>>>> with a specific CPU model? Or it is in QEMU? Where? What I see now is this:
>>>>
>>>> static const VMStateDescription vmstate_s390_cpu = {
>>>>      .name = "cpu",
>>>>      .unmigratable = 1,
>>>> };
>>>>
>>>> Does not look like it supports migration :) Thanks!
>>>>
>>> The code you're missing is not upstream yet. The s390x guest can be migrated in the meantime.
>>> Yes, libvirt currently gets an extension to be able to identify and startup suitable migration
>>> targets for s390x on behalf of the mentioned qemu cpu model. BTW can you point me to the above
>>> mentioned SPAPR stuff...
>>
>> Mmm. What stuff? :) At the moment POWERPC guests migrate if PVR (processor
>> version register) value is exactly the same. I am trying to relax this
>> limitation to any version within same CPU family, like power7 v1.0 and v2.1.
> With stuff I referred to to term sPAPR not realizing it relates to
> the Power Architecture Platform Requirements, got it now. :-)
>
> I see, ppc currently has this limitation to enforce compatibility
> VMSTATE_UINTTL_EQUAL(env.spr[SPR_PVR], PowerPCCPU),

Yes, but the s390 approach is a lot cleaner and I'd rather like to move 
into that direction.


Alex