[Qemu-devel] [RFC PATCH] target-ppc: enable migration within the same CPU family

[Qemu-devel] [RFC PATCH] target-ppc: enable migration within the same CPU family

[Alexey Kardashevskiy]

Currently only migration fails if CPU version is different even a bit.
For example, migration from POWER7 v2.0 to POWER7 v2.1 fails because of
that. Since there is no difference between CPU versions which could
affect migration stream, we can safely enable it.

This adds a helper to find the closest POWERPC family class (i.e. first
abstract class in hierarchy).

This replaces VMSTATE_UINTTL_EQUAL statement with a custom handler which
checks if the source and destination CPUs belong to the same family and
fails if they are not.

This adds a PVR reset to the default value as it will be overwritten
by VMSTATE_UINTTL_ARRAY(env.spr, PowerPCCPU, 1024).

Since the actual migration format is not changed by this patch,
@version_id of vmstate_ppc_cpu does not have to be changed either.

Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
 target-ppc/cpu-qom.h        |  1 +
 target-ppc/machine.c        | 40 ++++++++++++++++++++++++++++++++++++++--
 target-ppc/translate_init.c | 12 ++++++++++++
 3 files changed, 51 insertions(+), 2 deletions(-)

diff --git a/target-ppc/cpu-qom.h b/target-ppc/cpu-qom.h
index 47dc8e6..5eb56ea 100644
--- a/target-ppc/cpu-qom.h
+++ b/target-ppc/cpu-qom.h
@@ -105,6 +105,7 @@ static inline PowerPCCPU *ppc_env_get_cpu(CPUPPCState *env)
 
 PowerPCCPUClass *ppc_cpu_class_by_pvr(uint32_t pvr);
 PowerPCCPUClass *ppc_cpu_class_by_pvr_mask(uint32_t pvr);
+PowerPCCPUClass *ppc_cpu_family_class_by_pvr_mask(uint32_t pvr);
 
 void ppc_cpu_do_interrupt(CPUState *cpu);
 void ppc_cpu_dump_state(CPUState *cpu, FILE *f, fprintf_function cpu_fprintf,
diff --git a/target-ppc/machine.c b/target-ppc/machine.c
index 063b379..834297e 100644
--- a/target-ppc/machine.c
+++ b/target-ppc/machine.c
@@ -160,6 +160,11 @@ static int cpu_post_load(void *opaque, int version_id)
     CPUPPCState *env = &cpu->env;
     int i;
 
+    /*
+     * Allow migration between hosts of same processor family
+     * by restoring the default PVR for this VM on this host.
+     */
+    env->spr[SPR_PVR] = env->spr_cb[SPR_PVR].default_value;
     env->lr = env->spr[SPR_LR];
     env->ctr = env->spr[SPR_CTR];
     env->xer = env->spr[SPR_XER];
@@ -462,6 +467,37 @@ static const VMStateDescription vmstate_tlbmas = {
     }
 };
 
+static int get_pvr(QEMUFile *f, void *pv, size_t size)
+{
+    target_ulong pvrdest = *(target_ulong *)pv;
+#if TARGET_LONG_BITS == 64
+    target_ulong pvrsrc = qemu_get_be64(f);
+#else
+    target_ulong pvrsrc = qemu_get_be32(f);
+#endif
+    PowerPCCPUClass *pccdest = ppc_cpu_family_class_by_pvr_mask(pvrdest);
+    PowerPCCPUClass *pccsrc = ppc_cpu_family_class_by_pvr_mask(pvrsrc);
+
+    return (pccdest == pccsrc) ? 0 : -1;
+}
+
+static void put_pvr(QEMUFile *f, void *pv, size_t size)
+{
+    target_ulong pvr = *(target_ulong *)pv;
+
+#if TARGET_LONG_BITS == 64
+    qemu_put_be64(f, pvr);
+#else
+    qemu_put_be32(f, pvr);
+#endif
+}
+
+static const VMStateInfo vmstate_pvr = {
+    .name = "PVR",
+    .get  = get_pvr,
+    .put  = put_pvr,
+};
+
 const VMStateDescription vmstate_ppc_cpu = {
     .name = "cpu",
     .version_id = 5,
@@ -471,8 +507,8 @@ const VMStateDescription vmstate_ppc_cpu = {
     .pre_save = cpu_pre_save,
     .post_load = cpu_post_load,
     .fields      = (VMStateField []) {
-        /* Verify we haven't changed the pvr */
-        VMSTATE_UINTTL_EQUAL(env.spr[SPR_PVR], PowerPCCPU),
+        VMSTATE_SINGLE(env.spr[SPR_PVR], PowerPCCPU, 0, vmstate_pvr,
+                       target_ulong),
 
         /* User mode architected state */
         VMSTATE_UINTTL_ARRAY(env.gpr, PowerPCCPU, 32),
diff --git a/target-ppc/translate_init.c b/target-ppc/translate_init.c
index cdb2d2a..0c5c6a8 100644
--- a/target-ppc/translate_init.c
+++ b/target-ppc/translate_init.c
@@ -8146,6 +8146,18 @@ PowerPCCPUClass *ppc_cpu_class_by_pvr_mask(uint32_t pvr)
     return pcc;
 }
 
+PowerPCCPUClass *ppc_cpu_family_class_by_pvr_mask(uint32_t pvr)
+{
+    PowerPCCPUClass *pcc = ppc_cpu_class_by_pvr_mask(pvr);
+    ObjectClass *oc = OBJECT_CLASS(pcc);
+
+    while (oc && !object_class_is_abstract(oc)) {
+        oc = object_class_get_parent(oc);
+    }
+
+    return POWERPC_CPU_CLASS(oc);
+}
+
 static gint ppc_cpu_compare_class_name(gconstpointer a, gconstpointer b)
 {
     ObjectClass *oc = (ObjectClass *)a;
-- 
1.8.4.rc4

Re: [Qemu-devel] [RFC PATCH] target-ppc: enable migration within the same CPU family

[Alexey Kardashevskiy]

On 03/24/2014 04:28 PM, Alexey Kardashevskiy wrote:
> Currently only migration fails if CPU version is different even a bit.
> For example, migration from POWER7 v2.0 to POWER7 v2.1 fails because of
> that. Since there is no difference between CPU versions which could
> affect migration stream, we can safely enable it.
> 
> This adds a helper to find the closest POWERPC family class (i.e. first
> abstract class in hierarchy).
> 
> This replaces VMSTATE_UINTTL_EQUAL statement with a custom handler which
> checks if the source and destination CPUs belong to the same family and
> fails if they are not.
> 
> This adds a PVR reset to the default value as it will be overwritten
> by VMSTATE_UINTTL_ARRAY(env.spr, PowerPCCPU, 1024).
> 
> Since the actual migration format is not changed by this patch,
> @version_id of vmstate_ppc_cpu does not have to be changed either.
> 
> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>


Ping?


> ---
>  target-ppc/cpu-qom.h        |  1 +
>  target-ppc/machine.c        | 40 ++++++++++++++++++++++++++++++++++++++--
>  target-ppc/translate_init.c | 12 ++++++++++++
>  3 files changed, 51 insertions(+), 2 deletions(-)
> 
> diff --git a/target-ppc/cpu-qom.h b/target-ppc/cpu-qom.h
> index 47dc8e6..5eb56ea 100644
> --- a/target-ppc/cpu-qom.h
> +++ b/target-ppc/cpu-qom.h
> @@ -105,6 +105,7 @@ static inline PowerPCCPU *ppc_env_get_cpu(CPUPPCState *env)
>  
>  PowerPCCPUClass *ppc_cpu_class_by_pvr(uint32_t pvr);
>  PowerPCCPUClass *ppc_cpu_class_by_pvr_mask(uint32_t pvr);
> +PowerPCCPUClass *ppc_cpu_family_class_by_pvr_mask(uint32_t pvr);
>  
>  void ppc_cpu_do_interrupt(CPUState *cpu);
>  void ppc_cpu_dump_state(CPUState *cpu, FILE *f, fprintf_function cpu_fprintf,
> diff --git a/target-ppc/machine.c b/target-ppc/machine.c
> index 063b379..834297e 100644
> --- a/target-ppc/machine.c
> +++ b/target-ppc/machine.c
> @@ -160,6 +160,11 @@ static int cpu_post_load(void *opaque, int version_id)
>      CPUPPCState *env = &cpu->env;
>      int i;
>  
> +    /*
> +     * Allow migration between hosts of same processor family
> +     * by restoring the default PVR for this VM on this host.
> +     */
> +    env->spr[SPR_PVR] = env->spr_cb[SPR_PVR].default_value;
>      env->lr = env->spr[SPR_LR];
>      env->ctr = env->spr[SPR_CTR];
>      env->xer = env->spr[SPR_XER];
> @@ -462,6 +467,37 @@ static const VMStateDescription vmstate_tlbmas = {
>      }
>  };
>  
> +static int get_pvr(QEMUFile *f, void *pv, size_t size)
> +{
> +    target_ulong pvrdest = *(target_ulong *)pv;
> +#if TARGET_LONG_BITS == 64
> +    target_ulong pvrsrc = qemu_get_be64(f);
> +#else
> +    target_ulong pvrsrc = qemu_get_be32(f);
> +#endif
> +    PowerPCCPUClass *pccdest = ppc_cpu_family_class_by_pvr_mask(pvrdest);
> +    PowerPCCPUClass *pccsrc = ppc_cpu_family_class_by_pvr_mask(pvrsrc);
> +
> +    return (pccdest == pccsrc) ? 0 : -1;
> +}
> +
> +static void put_pvr(QEMUFile *f, void *pv, size_t size)
> +{
> +    target_ulong pvr = *(target_ulong *)pv;
> +
> +#if TARGET_LONG_BITS == 64
> +    qemu_put_be64(f, pvr);
> +#else
> +    qemu_put_be32(f, pvr);
> +#endif
> +}
> +
> +static const VMStateInfo vmstate_pvr = {
> +    .name = "PVR",
> +    .get  = get_pvr,
> +    .put  = put_pvr,
> +};
> +
>  const VMStateDescription vmstate_ppc_cpu = {
>      .name = "cpu",
>      .version_id = 5,
> @@ -471,8 +507,8 @@ const VMStateDescription vmstate_ppc_cpu = {
>      .pre_save = cpu_pre_save,
>      .post_load = cpu_post_load,
>      .fields      = (VMStateField []) {
> -        /* Verify we haven't changed the pvr */
> -        VMSTATE_UINTTL_EQUAL(env.spr[SPR_PVR], PowerPCCPU),
> +        VMSTATE_SINGLE(env.spr[SPR_PVR], PowerPCCPU, 0, vmstate_pvr,
> +                       target_ulong),
>  
>          /* User mode architected state */
>          VMSTATE_UINTTL_ARRAY(env.gpr, PowerPCCPU, 32),
> diff --git a/target-ppc/translate_init.c b/target-ppc/translate_init.c
> index cdb2d2a..0c5c6a8 100644
> --- a/target-ppc/translate_init.c
> +++ b/target-ppc/translate_init.c
> @@ -8146,6 +8146,18 @@ PowerPCCPUClass *ppc_cpu_class_by_pvr_mask(uint32_t pvr)
>      return pcc;
>  }
>  
> +PowerPCCPUClass *ppc_cpu_family_class_by_pvr_mask(uint32_t pvr)
> +{
> +    PowerPCCPUClass *pcc = ppc_cpu_class_by_pvr_mask(pvr);
> +    ObjectClass *oc = OBJECT_CLASS(pcc);
> +
> +    while (oc && !object_class_is_abstract(oc)) {
> +        oc = object_class_get_parent(oc);
> +    }
> +
> +    return POWERPC_CPU_CLASS(oc);
> +}
> +
>  static gint ppc_cpu_compare_class_name(gconstpointer a, gconstpointer b)
>  {
>      ObjectClass *oc = (ObjectClass *)a;
> 


-- 
Alexey

Re: [Qemu-devel] [RFC PATCH] target-ppc: enable migration within the same CPU family

[Alexander Graf]

On 04/04/2014 07:17 AM, Alexey Kardashevskiy wrote:
> On 03/24/2014 04:28 PM, Alexey Kardashevskiy wrote:
>> Currently only migration fails if CPU version is different even a bit.
>> For example, migration from POWER7 v2.0 to POWER7 v2.1 fails because of
>> that. Since there is no difference between CPU versions which could
>> affect migration stream, we can safely enable it.
>>
>> This adds a helper to find the closest POWERPC family class (i.e. first
>> abstract class in hierarchy).
>>
>> This replaces VMSTATE_UINTTL_EQUAL statement with a custom handler which
>> checks if the source and destination CPUs belong to the same family and
>> fails if they are not.
>>
>> This adds a PVR reset to the default value as it will be overwritten
>> by VMSTATE_UINTTL_ARRAY(env.spr, PowerPCCPU, 1024).
>>
>> Since the actual migration format is not changed by this patch,
>> @version_id of vmstate_ppc_cpu does not have to be changed either.
>>
>> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
>
> Ping?

Can't we just always allow migration to succeed? It's a problem of the 
tool stack above if it allows migration to an incompatible host, no?


Alex

Re: [Qemu-devel] [RFC PATCH] target-ppc: enable migration within the same CPU family

[Alexey Kardashevskiy]

On 04/04/2014 11:28 PM, Alexander Graf wrote:
> On 04/04/2014 07:17 AM, Alexey Kardashevskiy wrote:
>> On 03/24/2014 04:28 PM, Alexey Kardashevskiy wrote:
>>> Currently only migration fails if CPU version is different even a bit.
>>> For example, migration from POWER7 v2.0 to POWER7 v2.1 fails because of
>>> that. Since there is no difference between CPU versions which could
>>> affect migration stream, we can safely enable it.
>>>
>>> This adds a helper to find the closest POWERPC family class (i.e. first
>>> abstract class in hierarchy).
>>>
>>> This replaces VMSTATE_UINTTL_EQUAL statement with a custom handler which
>>> checks if the source and destination CPUs belong to the same family and
>>> fails if they are not.
>>>
>>> This adds a PVR reset to the default value as it will be overwritten
>>> by VMSTATE_UINTTL_ARRAY(env.spr, PowerPCCPU, 1024).
>>>
>>> Since the actual migration format is not changed by this patch,
>>> @version_id of vmstate_ppc_cpu does not have to be changed either.
>>>
>>> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
>>
>> Ping?
> 
> Can't we just always allow migration to succeed? It's a problem of the tool
> stack above if it allows migration to an incompatible host, no?

This is not how libvirt works. It simply sends the source XML, reconstructs
a guest on the destination side and then migrates. hoping that the
migration will fail is something (which only QEMU has knowledge of) is
incompatible. The new guest will start with "-cpu host" (as the source) but
it will create diffrent CPU class and do different things. If we do not
check PVR (and cpu_dt_id and chip_id - the latter is coming soon) and
migrate power8->power7, we can easily get a broken guest.

How would I fix this CPU parameters (i.e. PVR/cpu_dt_id) check in the
existing libvirt?


-- 
Alexey

Re: [Qemu-devel] [RFC PATCH] target-ppc: enable migration within the same CPU family

[Andreas Färber]

Am 07.04.2014 05:27, schrieb Alexey Kardashevskiy:
> On 04/04/2014 11:28 PM, Alexander Graf wrote:
>> On 04/04/2014 07:17 AM, Alexey Kardashevskiy wrote:
>>> On 03/24/2014 04:28 PM, Alexey Kardashevskiy wrote:
>>>> Currently only migration fails if CPU version is different even a bit.
>>>> For example, migration from POWER7 v2.0 to POWER7 v2.1 fails because of
>>>> that. Since there is no difference between CPU versions which could
>>>> affect migration stream, we can safely enable it.
>>>>
>>>> This adds a helper to find the closest POWERPC family class (i.e. first
>>>> abstract class in hierarchy).
>>>>
>>>> This replaces VMSTATE_UINTTL_EQUAL statement with a custom handler which
>>>> checks if the source and destination CPUs belong to the same family and
>>>> fails if they are not.
>>>>
>>>> This adds a PVR reset to the default value as it will be overwritten
>>>> by VMSTATE_UINTTL_ARRAY(env.spr, PowerPCCPU, 1024).
>>>>
>>>> Since the actual migration format is not changed by this patch,
>>>> @version_id of vmstate_ppc_cpu does not have to be changed either.
>>>>
>>>> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
>>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
>>>
>>> Ping?
>>
>> Can't we just always allow migration to succeed? It's a problem of the tool
>> stack above if it allows migration to an incompatible host, no?
> 
> This is not how libvirt works. It simply sends the source XML, reconstructs
> a guest on the destination side and then migrates. hoping that the
> migration will fail is something (which only QEMU has knowledge of) is
> incompatible. The new guest will start with "-cpu host" (as the source) but
> it will create diffrent CPU class and do different things. If we do not
> check PVR (and cpu_dt_id and chip_id - the latter is coming soon) and
> migrate power8->power7, we can easily get a broken guest.

The response is very simple: -cpu host is not supported for migration.
Same as for x86 hosts.

As you say, the domain config is transferred by libvirt:
If you use -cpu POWER7, you can migrate from POWER7 to POWER8 and back;
if you use -cpu POWER8, you can only migrate on POWER8.

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [RFC PATCH] target-ppc: enable migration within the same CPU family

[Alexey Kardashevskiy]

On 04/08/2014 04:53 AM, Andreas Färber wrote:
> Am 07.04.2014 05:27, schrieb Alexey Kardashevskiy:
>> On 04/04/2014 11:28 PM, Alexander Graf wrote:
>>> On 04/04/2014 07:17 AM, Alexey Kardashevskiy wrote:
>>>> On 03/24/2014 04:28 PM, Alexey Kardashevskiy wrote:
>>>>> Currently only migration fails if CPU version is different even a bit.
>>>>> For example, migration from POWER7 v2.0 to POWER7 v2.1 fails because of
>>>>> that. Since there is no difference between CPU versions which could
>>>>> affect migration stream, we can safely enable it.
>>>>>
>>>>> This adds a helper to find the closest POWERPC family class (i.e. first
>>>>> abstract class in hierarchy).
>>>>>
>>>>> This replaces VMSTATE_UINTTL_EQUAL statement with a custom handler which
>>>>> checks if the source and destination CPUs belong to the same family and
>>>>> fails if they are not.
>>>>>
>>>>> This adds a PVR reset to the default value as it will be overwritten
>>>>> by VMSTATE_UINTTL_ARRAY(env.spr, PowerPCCPU, 1024).
>>>>>
>>>>> Since the actual migration format is not changed by this patch,
>>>>> @version_id of vmstate_ppc_cpu does not have to be changed either.
>>>>>
>>>>> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
>>>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
>>>>
>>>> Ping?
>>>
>>> Can't we just always allow migration to succeed? It's a problem of the tool
>>> stack above if it allows migration to an incompatible host, no?
>>
>> This is not how libvirt works. It simply sends the source XML, reconstructs
>> a guest on the destination side and then migrates. hoping that the
>> migration will fail is something (which only QEMU has knowledge of) is
>> incompatible. The new guest will start with "-cpu host" (as the source) but
>> it will create diffrent CPU class and do different things. If we do not
>> check PVR (and cpu_dt_id and chip_id - the latter is coming soon) and
>> migrate power8->power7, we can easily get a broken guest.
> 
> The response is very simple: -cpu host is not supported for migration.
> Same as for x86 hosts.

Is there any good reason to limit ourselves on POWERPC?

> As you say, the domain config is transferred by libvirt:
> If you use -cpu POWER7, you can migrate from POWER7 to POWER8 and back;
> if you use -cpu POWER8, you can only migrate on POWER8.

-cpu other that "host" is not supported by HV KVM, only "compat" which
upstream QEMU does not have yet. So you are saying that the migration is
not supported by upstream QEMU for at least SPAPR. Well, ok, it is dead
anyway so I am fine :)



-- 
Alexey

Re: [Qemu-devel] [RFC PATCH] target-ppc: enable migration within the same CPU family

[Michael Mueller]

On Tue, 08 Apr 2014 11:23:14 +1000
Alexey Kardashevskiy <aik@ozlabs.ru> wrote:

> On 04/08/2014 04:53 AM, Andreas Färber wrote:
> > Am 07.04.2014 05:27, schrieb Alexey Kardashevskiy:
> >> On 04/04/2014 11:28 PM, Alexander Graf wrote:
> >>> On 04/04/2014 07:17 AM, Alexey Kardashevskiy wrote:
> >>>> On 03/24/2014 04:28 PM, Alexey Kardashevskiy wrote:
> >>>>> Currently only migration fails if CPU version is different even a bit.
> >>>>> For example, migration from POWER7 v2.0 to POWER7 v2.1 fails because of
> >>>>> that. Since there is no difference between CPU versions which could
> >>>>> affect migration stream, we can safely enable it.
> >>>>>
> >>>>> This adds a helper to find the closest POWERPC family class (i.e. first
> >>>>> abstract class in hierarchy).
> >>>>>
> >>>>> This replaces VMSTATE_UINTTL_EQUAL statement with a custom handler which
> >>>>> checks if the source and destination CPUs belong to the same family and
> >>>>> fails if they are not.
> >>>>>
> >>>>> This adds a PVR reset to the default value as it will be overwritten
> >>>>> by VMSTATE_UINTTL_ARRAY(env.spr, PowerPCCPU, 1024).
> >>>>>
> >>>>> Since the actual migration format is not changed by this patch,
> >>>>> @version_id of vmstate_ppc_cpu does not have to be changed either.
> >>>>>
> >>>>> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> >>>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> >>>>
> >>>> Ping?
> >>>
> >>> Can't we just always allow migration to succeed? It's a problem of the tool
> >>> stack above if it allows migration to an incompatible host, no?
> >>
> >> This is not how libvirt works. It simply sends the source XML, reconstructs
> >> a guest on the destination side and then migrates. hoping that the
> >> migration will fail is something (which only QEMU has knowledge of) is
> >> incompatible. The new guest will start with "-cpu host" (as the source) but
> >> it will create diffrent CPU class and do different things. If we do not
> >> check PVR (and cpu_dt_id and chip_id - the latter is coming soon) and
> >> migrate power8->power7, we can easily get a broken guest.
> > 
> > The response is very simple: -cpu host is not supported for migration.
> > Same as for x86 hosts.
> 
> Is there any good reason to limit ourselves on POWERPC?
> 
> > As you say, the domain config is transferred by libvirt:
> > If you use -cpu POWER7, you can migrate from POWER7 to POWER8 and back;
> > if you use -cpu POWER8, you can only migrate on POWER8.
> 
> -cpu other that "host" is not supported by HV KVM, only "compat" which
> upstream QEMU does not have yet. So you are saying that the migration is
> not supported by upstream QEMU for at least SPAPR. Well, ok, it is dead
> anyway so I am fine :)
> 

With s390x we have a similar situation. Thus we came up with a mechanism to limit
the CPU functionality of a possible target system. Our patch implements CPU models
based on TYPE and GA like 2817-ga1, etc. (GA represents a CPU facility set and an IBC
value (Instruction Blocking Control, reduces the instruction set to the requested
level)) When a guest is started, it receives its CPU model by means of option -cpu.
"host" equates the configuration of the current system. We implemented "query-cpu-model"
returning the actual model, here maybe { name: "2817-ga1" }. To find a suitable
migration target in a remote CEC, libvirt has to "query-cpu-definitions" returning a
list of models supported by the target system "{{name: "2827-ga2"}, {name: "2827-ga1"},
{name: "2817-ga2"},...]. A match means the system is suitable and can be used
as migration target.

Thanks
Michael

Re: [Qemu-devel] [RFC PATCH] target-ppc: enable migration within the same CPU family

[Alexey Kardashevskiy]

On 04/08/2014 07:47 PM, Michael Mueller wrote:
> On Tue, 08 Apr 2014 11:23:14 +1000
> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> 
>> On 04/08/2014 04:53 AM, Andreas Färber wrote:
>>> Am 07.04.2014 05:27, schrieb Alexey Kardashevskiy:
>>>> On 04/04/2014 11:28 PM, Alexander Graf wrote:
>>>>> On 04/04/2014 07:17 AM, Alexey Kardashevskiy wrote:
>>>>>> On 03/24/2014 04:28 PM, Alexey Kardashevskiy wrote:
>>>>>>> Currently only migration fails if CPU version is different even a bit.
>>>>>>> For example, migration from POWER7 v2.0 to POWER7 v2.1 fails because of
>>>>>>> that. Since there is no difference between CPU versions which could
>>>>>>> affect migration stream, we can safely enable it.
>>>>>>>
>>>>>>> This adds a helper to find the closest POWERPC family class (i.e. first
>>>>>>> abstract class in hierarchy).
>>>>>>>
>>>>>>> This replaces VMSTATE_UINTTL_EQUAL statement with a custom handler which
>>>>>>> checks if the source and destination CPUs belong to the same family and
>>>>>>> fails if they are not.
>>>>>>>
>>>>>>> This adds a PVR reset to the default value as it will be overwritten
>>>>>>> by VMSTATE_UINTTL_ARRAY(env.spr, PowerPCCPU, 1024).
>>>>>>>
>>>>>>> Since the actual migration format is not changed by this patch,
>>>>>>> @version_id of vmstate_ppc_cpu does not have to be changed either.
>>>>>>>
>>>>>>> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
>>>>>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
>>>>>>
>>>>>> Ping?
>>>>>
>>>>> Can't we just always allow migration to succeed? It's a problem of the tool
>>>>> stack above if it allows migration to an incompatible host, no?
>>>>
>>>> This is not how libvirt works. It simply sends the source XML, reconstructs
>>>> a guest on the destination side and then migrates. hoping that the
>>>> migration will fail is something (which only QEMU has knowledge of) is
>>>> incompatible. The new guest will start with "-cpu host" (as the source) but
>>>> it will create diffrent CPU class and do different things. If we do not
>>>> check PVR (and cpu_dt_id and chip_id - the latter is coming soon) and
>>>> migrate power8->power7, we can easily get a broken guest.
>>>
>>> The response is very simple: -cpu host is not supported for migration.
>>> Same as for x86 hosts.
>>
>> Is there any good reason to limit ourselves on POWERPC?
>>
>>> As you say, the domain config is transferred by libvirt:
>>> If you use -cpu POWER7, you can migrate from POWER7 to POWER8 and back;
>>> if you use -cpu POWER8, you can only migrate on POWER8.
>>
>> -cpu other that "host" is not supported by HV KVM, only "compat" which
>> upstream QEMU does not have yet. So you are saying that the migration is
>> not supported by upstream QEMU for at least SPAPR. Well, ok, it is dead
>> anyway so I am fine :)
>>
> 
> With s390x we have a similar situation. Thus we came up with a mechanism to limit
> the CPU functionality of a possible target system. Our patch implements CPU models
> based on TYPE and GA like 2817-ga1, etc. (GA represents a CPU facility set and an IBC
> value (Instruction Blocking Control, reduces the instruction set to the requested
> level)) When a guest is started, it receives its CPU model by means of option -cpu.
> "host" equates the configuration of the current system. We implemented "query-cpu-model"
> returning the actual model, here maybe { name: "2817-ga1" }. To find a suitable
> migration target in a remote CEC, libvirt has to "query-cpu-definitions" returning a
> list of models supported by the target system "{{name: "2827-ga2"}, {name: "2827-ga1"},
> {name: "2817-ga2"},...]. A match means the system is suitable and can be used
> as migration target.

Sorry, I do not follow you. You hacked libvirt to run the destination QEMU
with a specific CPU model? Or it is in QEMU? Where? What I see now is this:

static const VMStateDescription vmstate_s390_cpu = {
    .name = "cpu",
    .unmigratable = 1,
};

Does not look like it supports migration :) Thanks!


-- 
Alexey

Re: [Qemu-devel] [RFC PATCH] target-ppc: enable migration within the same CPU family

[Michael Mueller]

On Tue, 08 Apr 2014 20:04:42 +1000
Alexey Kardashevskiy <aik@ozlabs.ru> wrote:

> On 04/08/2014 07:47 PM, Michael Mueller wrote:
> > On Tue, 08 Apr 2014 11:23:14 +1000
> > Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> > 
> >> On 04/08/2014 04:53 AM, Andreas Färber wrote:
> >>> Am 07.04.2014 05:27, schrieb Alexey Kardashevskiy:
> >>>> On 04/04/2014 11:28 PM, Alexander Graf wrote:
> >>>>> On 04/04/2014 07:17 AM, Alexey Kardashevskiy wrote:
> >>>>>> On 03/24/2014 04:28 PM, Alexey Kardashevskiy wrote:
> >>>>>>> Currently only migration fails if CPU version is different even a bit.
> >>>>>>> For example, migration from POWER7 v2.0 to POWER7 v2.1 fails because of
> >>>>>>> that. Since there is no difference between CPU versions which could
> >>>>>>> affect migration stream, we can safely enable it.
> >>>>>>>
> >>>>>>> This adds a helper to find the closest POWERPC family class (i.e. first
> >>>>>>> abstract class in hierarchy).
> >>>>>>>
> >>>>>>> This replaces VMSTATE_UINTTL_EQUAL statement with a custom handler which
> >>>>>>> checks if the source and destination CPUs belong to the same family and
> >>>>>>> fails if they are not.
> >>>>>>>
> >>>>>>> This adds a PVR reset to the default value as it will be overwritten
> >>>>>>> by VMSTATE_UINTTL_ARRAY(env.spr, PowerPCCPU, 1024).
> >>>>>>>
> >>>>>>> Since the actual migration format is not changed by this patch,
> >>>>>>> @version_id of vmstate_ppc_cpu does not have to be changed either.
> >>>>>>>
> >>>>>>> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> >>>>>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> >>>>>>
> >>>>>> Ping?
> >>>>>
> >>>>> Can't we just always allow migration to succeed? It's a problem of the tool
> >>>>> stack above if it allows migration to an incompatible host, no?
> >>>>
> >>>> This is not how libvirt works. It simply sends the source XML, reconstructs
> >>>> a guest on the destination side and then migrates. hoping that the
> >>>> migration will fail is something (which only QEMU has knowledge of) is
> >>>> incompatible. The new guest will start with "-cpu host" (as the source) but
> >>>> it will create diffrent CPU class and do different things. If we do not
> >>>> check PVR (and cpu_dt_id and chip_id - the latter is coming soon) and
> >>>> migrate power8->power7, we can easily get a broken guest.
> >>>
> >>> The response is very simple: -cpu host is not supported for migration.
> >>> Same as for x86 hosts.
> >>
> >> Is there any good reason to limit ourselves on POWERPC?
> >>
> >>> As you say, the domain config is transferred by libvirt:
> >>> If you use -cpu POWER7, you can migrate from POWER7 to POWER8 and back;
> >>> if you use -cpu POWER8, you can only migrate on POWER8.
> >>
> >> -cpu other that "host" is not supported by HV KVM, only "compat" which
> >> upstream QEMU does not have yet. So you are saying that the migration is
> >> not supported by upstream QEMU for at least SPAPR. Well, ok, it is dead
> >> anyway so I am fine :)
> >>
> > 
> > With s390x we have a similar situation. Thus we came up with a mechanism to limit
> > the CPU functionality of a possible target system. Our patch implements CPU models
> > based on TYPE and GA like 2817-ga1, etc. (GA represents a CPU facility set and an IBC
> > value (Instruction Blocking Control, reduces the instruction set to the requested
> > level)) When a guest is started, it receives its CPU model by means of option -cpu.
> > "host" equates the configuration of the current system. We implemented "query-cpu-model"
> > returning the actual model, here maybe { name: "2817-ga1" }. To find a suitable
> > migration target in a remote CEC, libvirt has to "query-cpu-definitions" returning a
> > list of models supported by the target system "{{name: "2827-ga2"}, {name: "2827-ga1"},
> > {name: "2817-ga2"},...]. A match means the system is suitable and can be used
> > as migration target.
> 
> Sorry, I do not follow you. You hacked libvirt to run the destination QEMU
> with a specific CPU model? Or it is in QEMU? Where? What I see now is this:
> 
> static const VMStateDescription vmstate_s390_cpu = {
>     .name = "cpu",
>     .unmigratable = 1,
> };
> 
> Does not look like it supports migration :) Thanks!
> 

The code you're missing is not upstream yet. The s390x guest can be migrated in the meantime.
Yes, libvirt currently gets an extension to be able to identify and startup suitable migration
targets for s390x on behalf of the mentioned qemu cpu model. BTW can you point me to the above
mentioned SPAPR stuff...

Michael

Re: [Qemu-devel] [RFC PATCH] target-ppc: enable migration within the same CPU family

[Alexey Kardashevskiy]

On 04/08/2014 08:32 PM, Michael Mueller wrote:
> On Tue, 08 Apr 2014 20:04:42 +1000
> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> 
>> On 04/08/2014 07:47 PM, Michael Mueller wrote:
>>> On Tue, 08 Apr 2014 11:23:14 +1000
>>> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
>>>
>>>> On 04/08/2014 04:53 AM, Andreas Färber wrote:
>>>>> Am 07.04.2014 05:27, schrieb Alexey Kardashevskiy:
>>>>>> On 04/04/2014 11:28 PM, Alexander Graf wrote:
>>>>>>> On 04/04/2014 07:17 AM, Alexey Kardashevskiy wrote:
>>>>>>>> On 03/24/2014 04:28 PM, Alexey Kardashevskiy wrote:
>>>>>>>>> Currently only migration fails if CPU version is different even a bit.
>>>>>>>>> For example, migration from POWER7 v2.0 to POWER7 v2.1 fails because of
>>>>>>>>> that. Since there is no difference between CPU versions which could
>>>>>>>>> affect migration stream, we can safely enable it.
>>>>>>>>>
>>>>>>>>> This adds a helper to find the closest POWERPC family class (i.e. first
>>>>>>>>> abstract class in hierarchy).
>>>>>>>>>
>>>>>>>>> This replaces VMSTATE_UINTTL_EQUAL statement with a custom handler which
>>>>>>>>> checks if the source and destination CPUs belong to the same family and
>>>>>>>>> fails if they are not.
>>>>>>>>>
>>>>>>>>> This adds a PVR reset to the default value as it will be overwritten
>>>>>>>>> by VMSTATE_UINTTL_ARRAY(env.spr, PowerPCCPU, 1024).
>>>>>>>>>
>>>>>>>>> Since the actual migration format is not changed by this patch,
>>>>>>>>> @version_id of vmstate_ppc_cpu does not have to be changed either.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
>>>>>>>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
>>>>>>>>
>>>>>>>> Ping?
>>>>>>>
>>>>>>> Can't we just always allow migration to succeed? It's a problem of the tool
>>>>>>> stack above if it allows migration to an incompatible host, no?
>>>>>>
>>>>>> This is not how libvirt works. It simply sends the source XML, reconstructs
>>>>>> a guest on the destination side and then migrates. hoping that the
>>>>>> migration will fail is something (which only QEMU has knowledge of) is
>>>>>> incompatible. The new guest will start with "-cpu host" (as the source) but
>>>>>> it will create diffrent CPU class and do different things. If we do not
>>>>>> check PVR (and cpu_dt_id and chip_id - the latter is coming soon) and
>>>>>> migrate power8->power7, we can easily get a broken guest.
>>>>>
>>>>> The response is very simple: -cpu host is not supported for migration.
>>>>> Same as for x86 hosts.
>>>>
>>>> Is there any good reason to limit ourselves on POWERPC?
>>>>
>>>>> As you say, the domain config is transferred by libvirt:
>>>>> If you use -cpu POWER7, you can migrate from POWER7 to POWER8 and back;
>>>>> if you use -cpu POWER8, you can only migrate on POWER8.
>>>>
>>>> -cpu other that "host" is not supported by HV KVM, only "compat" which
>>>> upstream QEMU does not have yet. So you are saying that the migration is
>>>> not supported by upstream QEMU for at least SPAPR. Well, ok, it is dead
>>>> anyway so I am fine :)
>>>>
>>>
>>> With s390x we have a similar situation. Thus we came up with a mechanism to limit
>>> the CPU functionality of a possible target system. Our patch implements CPU models
>>> based on TYPE and GA like 2817-ga1, etc. (GA represents a CPU facility set and an IBC
>>> value (Instruction Blocking Control, reduces the instruction set to the requested
>>> level)) When a guest is started, it receives its CPU model by means of option -cpu.
>>> "host" equates the configuration of the current system. We implemented "query-cpu-model"
>>> returning the actual model, here maybe { name: "2817-ga1" }. To find a suitable
>>> migration target in a remote CEC, libvirt has to "query-cpu-definitions" returning a
>>> list of models supported by the target system "{{name: "2827-ga2"}, {name: "2827-ga1"},
>>> {name: "2817-ga2"},...]. A match means the system is suitable and can be used
>>> as migration target.
>>
>> Sorry, I do not follow you. You hacked libvirt to run the destination QEMU
>> with a specific CPU model? Or it is in QEMU? Where? What I see now is this:
>>
>> static const VMStateDescription vmstate_s390_cpu = {
>>     .name = "cpu",
>>     .unmigratable = 1,
>> };
>>
>> Does not look like it supports migration :) Thanks!
>>
> 
> The code you're missing is not upstream yet. The s390x guest can be migrated in the meantime.
> Yes, libvirt currently gets an extension to be able to identify and startup suitable migration
> targets for s390x on behalf of the mentioned qemu cpu model. BTW can you point me to the above
> mentioned SPAPR stuff...


Mmm. What stuff? :) At the moment POWERPC guests migrate if PVR (processor
version register) value is exactly the same. I am trying to relax this
limitation to any version within same CPU family, like power7 v1.0 and v2.1.



-- 
Alexey

Re: [Qemu-devel] [RFC PATCH] target-ppc: enable migration within the same CPU family

[Michael Mueller]

On Tue, 08 Apr 2014 21:47:39 +1000
Alexey Kardashevskiy <aik@ozlabs.ru> wrote:

> On 04/08/2014 08:32 PM, Michael Mueller wrote:
> > On Tue, 08 Apr 2014 20:04:42 +1000
> > Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> > 
> >> On 04/08/2014 07:47 PM, Michael Mueller wrote:
> >>> On Tue, 08 Apr 2014 11:23:14 +1000
> >>> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> >>>
> >>>> On 04/08/2014 04:53 AM, Andreas Färber wrote:
> >>>>> Am 07.04.2014 05:27, schrieb Alexey Kardashevskiy:
> >>>>>> On 04/04/2014 11:28 PM, Alexander Graf wrote:
> >>>>>>> On 04/04/2014 07:17 AM, Alexey Kardashevskiy wrote:
> >>>>>>>> On 03/24/2014 04:28 PM, Alexey Kardashevskiy wrote:
> >>>>>>>>> Currently only migration fails if CPU version is different even a bit.
> >>>>>>>>> For example, migration from POWER7 v2.0 to POWER7 v2.1 fails because of
> >>>>>>>>> that. Since there is no difference between CPU versions which could
> >>>>>>>>> affect migration stream, we can safely enable it.
> >>>>>>>>>
> >>>>>>>>> This adds a helper to find the closest POWERPC family class (i.e. first
> >>>>>>>>> abstract class in hierarchy).
> >>>>>>>>>
> >>>>>>>>> This replaces VMSTATE_UINTTL_EQUAL statement with a custom handler which
> >>>>>>>>> checks if the source and destination CPUs belong to the same family and
> >>>>>>>>> fails if they are not.
> >>>>>>>>>
> >>>>>>>>> This adds a PVR reset to the default value as it will be overwritten
> >>>>>>>>> by VMSTATE_UINTTL_ARRAY(env.spr, PowerPCCPU, 1024).
> >>>>>>>>>
> >>>>>>>>> Since the actual migration format is not changed by this patch,
> >>>>>>>>> @version_id of vmstate_ppc_cpu does not have to be changed either.
> >>>>>>>>>
> >>>>>>>>> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> >>>>>>>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> >>>>>>>>
> >>>>>>>> Ping?
> >>>>>>>
> >>>>>>> Can't we just always allow migration to succeed? It's a problem of the tool
> >>>>>>> stack above if it allows migration to an incompatible host, no?
> >>>>>>
> >>>>>> This is not how libvirt works. It simply sends the source XML, reconstructs
> >>>>>> a guest on the destination side and then migrates. hoping that the
> >>>>>> migration will fail is something (which only QEMU has knowledge of) is
> >>>>>> incompatible. The new guest will start with "-cpu host" (as the source) but
> >>>>>> it will create diffrent CPU class and do different things. If we do not
> >>>>>> check PVR (and cpu_dt_id and chip_id - the latter is coming soon) and
> >>>>>> migrate power8->power7, we can easily get a broken guest.
> >>>>>
> >>>>> The response is very simple: -cpu host is not supported for migration.
> >>>>> Same as for x86 hosts.
> >>>>
> >>>> Is there any good reason to limit ourselves on POWERPC?
> >>>>
> >>>>> As you say, the domain config is transferred by libvirt:
> >>>>> If you use -cpu POWER7, you can migrate from POWER7 to POWER8 and back;
> >>>>> if you use -cpu POWER8, you can only migrate on POWER8.
> >>>>
> >>>> -cpu other that "host" is not supported by HV KVM, only "compat" which
> >>>> upstream QEMU does not have yet. So you are saying that the migration is
> >>>> not supported by upstream QEMU for at least SPAPR. Well, ok, it is dead
> >>>> anyway so I am fine :)
> >>>>
> >>>
> >>> With s390x we have a similar situation. Thus we came up with a mechanism to limit
> >>> the CPU functionality of a possible target system. Our patch implements CPU models
> >>> based on TYPE and GA like 2817-ga1, etc. (GA represents a CPU facility set and an IBC
> >>> value (Instruction Blocking Control, reduces the instruction set to the requested
> >>> level)) When a guest is started, it receives its CPU model by means of option -cpu.
> >>> "host" equates the configuration of the current system. We implemented "query-cpu-model"
> >>> returning the actual model, here maybe { name: "2817-ga1" }. To find a suitable
> >>> migration target in a remote CEC, libvirt has to "query-cpu-definitions" returning a
> >>> list of models supported by the target system "{{name: "2827-ga2"}, {name: "2827-ga1"},
> >>> {name: "2817-ga2"},...]. A match means the system is suitable and can be used
> >>> as migration target.
> >>
> >> Sorry, I do not follow you. You hacked libvirt to run the destination QEMU
> >> with a specific CPU model? Or it is in QEMU? Where? What I see now is this:
> >>
> >> static const VMStateDescription vmstate_s390_cpu = {
> >>     .name = "cpu",
> >>     .unmigratable = 1,
> >> };
> >>
> >> Does not look like it supports migration :) Thanks!
> >>
> > 
> > The code you're missing is not upstream yet. The s390x guest can be migrated in the meantime.
> > Yes, libvirt currently gets an extension to be able to identify and startup suitable migration
> > targets for s390x on behalf of the mentioned qemu cpu model. BTW can you point me to the above
> > mentioned SPAPR stuff...
> 
> 
> Mmm. What stuff? :) At the moment POWERPC guests migrate if PVR (processor
> version register) value is exactly the same. I am trying to relax this
> limitation to any version within same CPU family, like power7 v1.0 and v2.1.

With stuff I referred to to term sPAPR not realizing it relates to
the Power Architecture Platform Requirements, got it now. :-)

I see, ppc currently has this limitation to enforce compatibility
VMSTATE_UINTTL_EQUAL(env.spr[SPR_PVR], PowerPCCPU),

Thanks
Michael

> 
> 
> 

Re: [Qemu-devel] [RFC PATCH] target-ppc: enable migration within the same CPU family

[Alexander Graf]

On 04/08/2014 02:19 PM, Michael Mueller wrote:
> On Tue, 08 Apr 2014 21:47:39 +1000
> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
>
>> On 04/08/2014 08:32 PM, Michael Mueller wrote:
>>> On Tue, 08 Apr 2014 20:04:42 +1000
>>> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
>>>
>>>> On 04/08/2014 07:47 PM, Michael Mueller wrote:
>>>>> On Tue, 08 Apr 2014 11:23:14 +1000
>>>>> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
>>>>>
>>>>>> On 04/08/2014 04:53 AM, Andreas Färber wrote:
>>>>>>> Am 07.04.2014 05:27, schrieb Alexey Kardashevskiy:
>>>>>>>> On 04/04/2014 11:28 PM, Alexander Graf wrote:
>>>>>>>>> On 04/04/2014 07:17 AM, Alexey Kardashevskiy wrote:
>>>>>>>>>> On 03/24/2014 04:28 PM, Alexey Kardashevskiy wrote:
>>>>>>>>>>> Currently only migration fails if CPU version is different even a bit.
>>>>>>>>>>> For example, migration from POWER7 v2.0 to POWER7 v2.1 fails because of
>>>>>>>>>>> that. Since there is no difference between CPU versions which could
>>>>>>>>>>> affect migration stream, we can safely enable it.
>>>>>>>>>>>
>>>>>>>>>>> This adds a helper to find the closest POWERPC family class (i.e. first
>>>>>>>>>>> abstract class in hierarchy).
>>>>>>>>>>>
>>>>>>>>>>> This replaces VMSTATE_UINTTL_EQUAL statement with a custom handler which
>>>>>>>>>>> checks if the source and destination CPUs belong to the same family and
>>>>>>>>>>> fails if they are not.
>>>>>>>>>>>
>>>>>>>>>>> This adds a PVR reset to the default value as it will be overwritten
>>>>>>>>>>> by VMSTATE_UINTTL_ARRAY(env.spr, PowerPCCPU, 1024).
>>>>>>>>>>>
>>>>>>>>>>> Since the actual migration format is not changed by this patch,
>>>>>>>>>>> @version_id of vmstate_ppc_cpu does not have to be changed either.
>>>>>>>>>>>
>>>>>>>>>>> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
>>>>>>>>>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
>>>>>>>>>> Ping?
>>>>>>>>> Can't we just always allow migration to succeed? It's a problem of the tool
>>>>>>>>> stack above if it allows migration to an incompatible host, no?
>>>>>>>> This is not how libvirt works. It simply sends the source XML, reconstructs
>>>>>>>> a guest on the destination side and then migrates. hoping that the
>>>>>>>> migration will fail is something (which only QEMU has knowledge of) is
>>>>>>>> incompatible. The new guest will start with "-cpu host" (as the source) but
>>>>>>>> it will create diffrent CPU class and do different things. If we do not
>>>>>>>> check PVR (and cpu_dt_id and chip_id - the latter is coming soon) and
>>>>>>>> migrate power8->power7, we can easily get a broken guest.
>>>>>>> The response is very simple: -cpu host is not supported for migration.
>>>>>>> Same as for x86 hosts.
>>>>>> Is there any good reason to limit ourselves on POWERPC?
>>>>>>
>>>>>>> As you say, the domain config is transferred by libvirt:
>>>>>>> If you use -cpu POWER7, you can migrate from POWER7 to POWER8 and back;
>>>>>>> if you use -cpu POWER8, you can only migrate on POWER8.
>>>>>> -cpu other that "host" is not supported by HV KVM, only "compat" which
>>>>>> upstream QEMU does not have yet. So you are saying that the migration is
>>>>>> not supported by upstream QEMU for at least SPAPR. Well, ok, it is dead
>>>>>> anyway so I am fine :)
>>>>>>
>>>>> With s390x we have a similar situation. Thus we came up with a mechanism to limit
>>>>> the CPU functionality of a possible target system. Our patch implements CPU models
>>>>> based on TYPE and GA like 2817-ga1, etc. (GA represents a CPU facility set and an IBC
>>>>> value (Instruction Blocking Control, reduces the instruction set to the requested
>>>>> level)) When a guest is started, it receives its CPU model by means of option -cpu.
>>>>> "host" equates the configuration of the current system. We implemented "query-cpu-model"
>>>>> returning the actual model, here maybe { name: "2817-ga1" }. To find a suitable
>>>>> migration target in a remote CEC, libvirt has to "query-cpu-definitions" returning a
>>>>> list of models supported by the target system "{{name: "2827-ga2"}, {name: "2827-ga1"},
>>>>> {name: "2817-ga2"},...]. A match means the system is suitable and can be used
>>>>> as migration target.
>>>> Sorry, I do not follow you. You hacked libvirt to run the destination QEMU
>>>> with a specific CPU model? Or it is in QEMU? Where? What I see now is this:
>>>>
>>>> static const VMStateDescription vmstate_s390_cpu = {
>>>>      .name = "cpu",
>>>>      .unmigratable = 1,
>>>> };
>>>>
>>>> Does not look like it supports migration :) Thanks!
>>>>
>>> The code you're missing is not upstream yet. The s390x guest can be migrated in the meantime.
>>> Yes, libvirt currently gets an extension to be able to identify and startup suitable migration
>>> targets for s390x on behalf of the mentioned qemu cpu model. BTW can you point me to the above
>>> mentioned SPAPR stuff...
>>
>> Mmm. What stuff? :) At the moment POWERPC guests migrate if PVR (processor
>> version register) value is exactly the same. I am trying to relax this
>> limitation to any version within same CPU family, like power7 v1.0 and v2.1.
> With stuff I referred to to term sPAPR not realizing it relates to
> the Power Architecture Platform Requirements, got it now. :-)
>
> I see, ppc currently has this limitation to enforce compatibility
> VMSTATE_UINTTL_EQUAL(env.spr[SPR_PVR], PowerPCCPU),

Yes, but the s390 approach is a lot cleaner and I'd rather like to move 
into that direction.


Alex

Re: [Qemu-devel] [RFC PATCH] target-ppc: enable migration within the same CPU family

[Alexey Kardashevskiy]

On 04/09/2014 12:59 AM, Alexander Graf wrote:
> On 04/08/2014 02:19 PM, Michael Mueller wrote:
>> On Tue, 08 Apr 2014 21:47:39 +1000
>> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
>>
>>> On 04/08/2014 08:32 PM, Michael Mueller wrote:
>>>> On Tue, 08 Apr 2014 20:04:42 +1000
>>>> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
>>>>
>>>>> On 04/08/2014 07:47 PM, Michael Mueller wrote:
>>>>>> On Tue, 08 Apr 2014 11:23:14 +1000
>>>>>> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
>>>>>>
>>>>>>> On 04/08/2014 04:53 AM, Andreas Färber wrote:
>>>>>>>> Am 07.04.2014 05:27, schrieb Alexey Kardashevskiy:
>>>>>>>>> On 04/04/2014 11:28 PM, Alexander Graf wrote:
>>>>>>>>>> On 04/04/2014 07:17 AM, Alexey Kardashevskiy wrote:
>>>>>>>>>>> On 03/24/2014 04:28 PM, Alexey Kardashevskiy wrote:
>>>>>>>>>>>> Currently only migration fails if CPU version is different even
>>>>>>>>>>>> a bit.
>>>>>>>>>>>> For example, migration from POWER7 v2.0 to POWER7 v2.1 fails
>>>>>>>>>>>> because of
>>>>>>>>>>>> that. Since there is no difference between CPU versions which
>>>>>>>>>>>> could
>>>>>>>>>>>> affect migration stream, we can safely enable it.
>>>>>>>>>>>>
>>>>>>>>>>>> This adds a helper to find the closest POWERPC family class
>>>>>>>>>>>> (i.e. first
>>>>>>>>>>>> abstract class in hierarchy).
>>>>>>>>>>>>
>>>>>>>>>>>> This replaces VMSTATE_UINTTL_EQUAL statement with a custom
>>>>>>>>>>>> handler which
>>>>>>>>>>>> checks if the source and destination CPUs belong to the same
>>>>>>>>>>>> family and
>>>>>>>>>>>> fails if they are not.
>>>>>>>>>>>>
>>>>>>>>>>>> This adds a PVR reset to the default value as it will be
>>>>>>>>>>>> overwritten
>>>>>>>>>>>> by VMSTATE_UINTTL_ARRAY(env.spr, PowerPCCPU, 1024).
>>>>>>>>>>>>
>>>>>>>>>>>> Since the actual migration format is not changed by this patch,
>>>>>>>>>>>> @version_id of vmstate_ppc_cpu does not have to be changed either.
>>>>>>>>>>>>
>>>>>>>>>>>> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
>>>>>>>>>>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
>>>>>>>>>>> Ping?
>>>>>>>>>> Can't we just always allow migration to succeed? It's a problem
>>>>>>>>>> of the tool
>>>>>>>>>> stack above if it allows migration to an incompatible host, no?
>>>>>>>>> This is not how libvirt works. It simply sends the source XML,
>>>>>>>>> reconstructs
>>>>>>>>> a guest on the destination side and then migrates. hoping that the
>>>>>>>>> migration will fail is something (which only QEMU has knowledge
>>>>>>>>> of) is
>>>>>>>>> incompatible. The new guest will start with "-cpu host" (as the
>>>>>>>>> source) but
>>>>>>>>> it will create diffrent CPU class and do different things. If we
>>>>>>>>> do not
>>>>>>>>> check PVR (and cpu_dt_id and chip_id - the latter is coming soon) and
>>>>>>>>> migrate power8->power7, we can easily get a broken guest.
>>>>>>>> The response is very simple: -cpu host is not supported for migration.
>>>>>>>> Same as for x86 hosts.
>>>>>>> Is there any good reason to limit ourselves on POWERPC?
>>>>>>>
>>>>>>>> As you say, the domain config is transferred by libvirt:
>>>>>>>> If you use -cpu POWER7, you can migrate from POWER7 to POWER8 and
>>>>>>>> back;
>>>>>>>> if you use -cpu POWER8, you can only migrate on POWER8.
>>>>>>> -cpu other that "host" is not supported by HV KVM, only "compat" which
>>>>>>> upstream QEMU does not have yet. So you are saying that the
>>>>>>> migration is
>>>>>>> not supported by upstream QEMU for at least SPAPR. Well, ok, it is dead
>>>>>>> anyway so I am fine :)
>>>>>>>
>>>>>> With s390x we have a similar situation. Thus we came up with a
>>>>>> mechanism to limit
>>>>>> the CPU functionality of a possible target system. Our patch
>>>>>> implements CPU models
>>>>>> based on TYPE and GA like 2817-ga1, etc. (GA represents a CPU
>>>>>> facility set and an IBC
>>>>>> value (Instruction Blocking Control, reduces the instruction set to
>>>>>> the requested
>>>>>> level)) When a guest is started, it receives its CPU model by means
>>>>>> of option -cpu.
>>>>>> "host" equates the configuration of the current system. We
>>>>>> implemented "query-cpu-model"
>>>>>> returning the actual model, here maybe { name: "2817-ga1" }. To find
>>>>>> a suitable
>>>>>> migration target in a remote CEC, libvirt has to
>>>>>> "query-cpu-definitions" returning a
>>>>>> list of models supported by the target system "{{name: "2827-ga2"},
>>>>>> {name: "2827-ga1"},
>>>>>> {name: "2817-ga2"},...]. A match means the system is suitable and can
>>>>>> be used
>>>>>> as migration target.
>>>>> Sorry, I do not follow you. You hacked libvirt to run the destination
>>>>> QEMU
>>>>> with a specific CPU model? Or it is in QEMU? Where? What I see now is
>>>>> this:
>>>>>
>>>>> static const VMStateDescription vmstate_s390_cpu = {
>>>>>      .name = "cpu",
>>>>>      .unmigratable = 1,
>>>>> };
>>>>>
>>>>> Does not look like it supports migration :) Thanks!
>>>>>
>>>> The code you're missing is not upstream yet. The s390x guest can be
>>>> migrated in the meantime.
>>>> Yes, libvirt currently gets an extension to be able to identify and
>>>> startup suitable migration
>>>> targets for s390x on behalf of the mentioned qemu cpu model. BTW can
>>>> you point me to the above
>>>> mentioned SPAPR stuff...
>>>
>>> Mmm. What stuff? :) At the moment POWERPC guests migrate if PVR (processor
>>> version register) value is exactly the same. I am trying to relax this
>>> limitation to any version within same CPU family, like power7 v1.0 and
>>> v2.1.
>> With stuff I referred to to term sPAPR not realizing it relates to
>> the Power Architecture Platform Requirements, got it now. :-)
>>
>> I see, ppc currently has this limitation to enforce compatibility
>> VMSTATE_UINTTL_EQUAL(env.spr[SPR_PVR], PowerPCCPU),
> 
> Yes, but the s390 approach is a lot cleaner and I'd rather like to move
> into that direction.

Then we will need to support (again) -cpu power7/8 for HV KVM. At the
moment POWER8 is an alias to POWER8_v1.0 and if we try -cpu POWER8 on
versions other than 1.0, HV KVM will fail on attempt to set PVR as it must
be the same as the host PVR (the reason is HV KVM does not emulate PVR, you
know).

To do this, we either need to add every single CPU version to QEMU (very,
very annoying) or allow -cpu to accept CPU family names (which is not
possible now as they are "abstract") or add a version-less CPU class to
every CPU family and redefine aliases to these new CPUs.

Or I do not understand s390 approach, then please correct me :)

What do I miss and what do I have to do now? Thanks!


-- 
Alexey

Re: [Qemu-devel] [RFC PATCH] target-ppc: enable migration within the same CPU family

[Alexander Graf]

> Am 09.04.2014 um 02:41 schrieb Alexey Kardashevskiy <aik@ozlabs.ru>:
> 
>> On 04/09/2014 12:59 AM, Alexander Graf wrote:
>>> On 04/08/2014 02:19 PM, Michael Mueller wrote:
>>> On Tue, 08 Apr 2014 21:47:39 +1000
>>> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
>>> 
>>>>> On 04/08/2014 08:32 PM, Michael Mueller wrote:
>>>>> On Tue, 08 Apr 2014 20:04:42 +1000
>>>>> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
>>>>> 
>>>>>>> On 04/08/2014 07:47 PM, Michael Mueller wrote:
>>>>>>> On Tue, 08 Apr 2014 11:23:14 +1000
>>>>>>> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
>>>>>>> 
>>>>>>>>> On 04/08/2014 04:53 AM, Andreas Färber wrote:
>>>>>>>>> Am 07.04.2014 05:27, schrieb Alexey Kardashevskiy:
>>>>>>>>>> On 04/04/2014 11:28 PM, Alexander Graf wrote:
>>>>>>>>>>> On 04/04/2014 07:17 AM, Alexey Kardashevskiy wrote:
>>>>>>>>>>>> On 03/24/2014 04:28 PM, Alexey Kardashevskiy wrote:
>>>>>>>>>>>>> Currently only migration fails if CPU version is different even
>>>>>>>>>>>>> a bit.
>>>>>>>>>>>>> For example, migration from POWER7 v2.0 to POWER7 v2.1 fails
>>>>>>>>>>>>> because of
>>>>>>>>>>>>> that. Since there is no difference between CPU versions which
>>>>>>>>>>>>> could
>>>>>>>>>>>>> affect migration stream, we can safely enable it.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> This adds a helper to find the closest POWERPC family class
>>>>>>>>>>>>> (i.e. first
>>>>>>>>>>>>> abstract class in hierarchy).
>>>>>>>>>>>>> 
>>>>>>>>>>>>> This replaces VMSTATE_UINTTL_EQUAL statement with a custom
>>>>>>>>>>>>> handler which
>>>>>>>>>>>>> checks if the source and destination CPUs belong to the same
>>>>>>>>>>>>> family and
>>>>>>>>>>>>> fails if they are not.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> This adds a PVR reset to the default value as it will be
>>>>>>>>>>>>> overwritten
>>>>>>>>>>>>> by VMSTATE_UINTTL_ARRAY(env.spr, PowerPCCPU, 1024).
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Since the actual migration format is not changed by this patch,
>>>>>>>>>>>>> @version_id of vmstate_ppc_cpu does not have to be changed either.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
>>>>>>>>>>>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
>>>>>>>>>>>> Ping?
>>>>>>>>>>> Can't we just always allow migration to succeed? It's a problem
>>>>>>>>>>> of the tool
>>>>>>>>>>> stack above if it allows migration to an incompatible host, no?
>>>>>>>>>> This is not how libvirt works. It simply sends the source XML,
>>>>>>>>>> reconstructs
>>>>>>>>>> a guest on the destination side and then migrates. hoping that the
>>>>>>>>>> migration will fail is something (which only QEMU has knowledge
>>>>>>>>>> of) is
>>>>>>>>>> incompatible. The new guest will start with "-cpu host" (as the
>>>>>>>>>> source) but
>>>>>>>>>> it will create diffrent CPU class and do different things. If we
>>>>>>>>>> do not
>>>>>>>>>> check PVR (and cpu_dt_id and chip_id - the latter is coming soon) and
>>>>>>>>>> migrate power8->power7, we can easily get a broken guest.
>>>>>>>>> The response is very simple: -cpu host is not supported for migration.
>>>>>>>>> Same as for x86 hosts.
>>>>>>>> Is there any good reason to limit ourselves on POWERPC?
>>>>>>>> 
>>>>>>>>> As you say, the domain config is transferred by libvirt:
>>>>>>>>> If you use -cpu POWER7, you can migrate from POWER7 to POWER8 and
>>>>>>>>> back;
>>>>>>>>> if you use -cpu POWER8, you can only migrate on POWER8.
>>>>>>>> -cpu other that "host" is not supported by HV KVM, only "compat" which
>>>>>>>> upstream QEMU does not have yet. So you are saying that the
>>>>>>>> migration is
>>>>>>>> not supported by upstream QEMU for at least SPAPR. Well, ok, it is dead
>>>>>>>> anyway so I am fine :)
>>>>>>> With s390x we have a similar situation. Thus we came up with a
>>>>>>> mechanism to limit
>>>>>>> the CPU functionality of a possible target system. Our patch
>>>>>>> implements CPU models
>>>>>>> based on TYPE and GA like 2817-ga1, etc. (GA represents a CPU
>>>>>>> facility set and an IBC
>>>>>>> value (Instruction Blocking Control, reduces the instruction set to
>>>>>>> the requested
>>>>>>> level)) When a guest is started, it receives its CPU model by means
>>>>>>> of option -cpu.
>>>>>>> "host" equates the configuration of the current system. We
>>>>>>> implemented "query-cpu-model"
>>>>>>> returning the actual model, here maybe { name: "2817-ga1" }. To find
>>>>>>> a suitable
>>>>>>> migration target in a remote CEC, libvirt has to
>>>>>>> "query-cpu-definitions" returning a
>>>>>>> list of models supported by the target system "{{name: "2827-ga2"},
>>>>>>> {name: "2827-ga1"},
>>>>>>> {name: "2817-ga2"},...]. A match means the system is suitable and can
>>>>>>> be used
>>>>>>> as migration target.
>>>>>> Sorry, I do not follow you. You hacked libvirt to run the destination
>>>>>> QEMU
>>>>>> with a specific CPU model? Or it is in QEMU? Where? What I see now is
>>>>>> this:
>>>>>> 
>>>>>> static const VMStateDescription vmstate_s390_cpu = {
>>>>>>     .name = "cpu",
>>>>>>     .unmigratable = 1,
>>>>>> };
>>>>>> 
>>>>>> Does not look like it supports migration :) Thanks!
>>>>> The code you're missing is not upstream yet. The s390x guest can be
>>>>> migrated in the meantime.
>>>>> Yes, libvirt currently gets an extension to be able to identify and
>>>>> startup suitable migration
>>>>> targets for s390x on behalf of the mentioned qemu cpu model. BTW can
>>>>> you point me to the above
>>>>> mentioned SPAPR stuff...
>>>> 
>>>> Mmm. What stuff? :) At the moment POWERPC guests migrate if PVR (processor
>>>> version register) value is exactly the same. I am trying to relax this
>>>> limitation to any version within same CPU family, like power7 v1.0 and
>>>> v2.1.
>>> With stuff I referred to to term sPAPR not realizing it relates to
>>> the Power Architecture Platform Requirements, got it now. :-)
>>> 
>>> I see, ppc currently has this limitation to enforce compatibility
>>> VMSTATE_UINTTL_EQUAL(env.spr[SPR_PVR], PowerPCCPU),
>> 
>> Yes, but the s390 approach is a lot cleaner and I'd rather like to move
>> into that direction.
> 
> Then we will need to support (again) -cpu power7/8 for HV KVM. At the
> moment POWER8 is an alias to POWER8_v1.0 and if we try -cpu POWER8 on
> versions other than 1.0, HV KVM will fail on attempt to set PVR as it must
> be the same as the host PVR (the reason is HV KVM does not emulate PVR, you
> know).
> 
> To do this, we either need to add every single CPU version to QEMU (very,
> very annoying) or allow -cpu to accept CPU family names (which is not
> possible now as they are "abstract") or add a version-less CPU class to
> every CPU family and redefine aliases to these new CPUs.

Yes, I think -cpu POWER8 should be an alias to 1.0 for TCG, but work within the constraints of the p8 masks the same way as -cpu host for KVM.


Alex

> 
> Or I do not understand s390 approach, then please correct me :)
> 
> What do I miss and what do I have to do now? Thanks!
> 
> 
> -- 
> Alexey

Re: [Qemu-devel] [RFC PATCH] target-ppc: enable migration within the same CPU family

[Alexey Kardashevskiy]

On 04/04/2014 11:28 PM, Alexander Graf wrote:
> On 04/04/2014 07:17 AM, Alexey Kardashevskiy wrote:
>> On 03/24/2014 04:28 PM, Alexey Kardashevskiy wrote:
>>> Currently only migration fails if CPU version is different even a bit.
>>> For example, migration from POWER7 v2.0 to POWER7 v2.1 fails because of
>>> that. Since there is no difference between CPU versions which could
>>> affect migration stream, we can safely enable it.
>>>
>>> This adds a helper to find the closest POWERPC family class (i.e. first
>>> abstract class in hierarchy).
>>>
>>> This replaces VMSTATE_UINTTL_EQUAL statement with a custom handler which
>>> checks if the source and destination CPUs belong to the same family and
>>> fails if they are not.
>>>
>>> This adds a PVR reset to the default value as it will be overwritten
>>> by VMSTATE_UINTTL_ARRAY(env.spr, PowerPCCPU, 1024).
>>>
>>> Since the actual migration format is not changed by this patch,
>>> @version_id of vmstate_ppc_cpu does not have to be changed either.
>>>
>>> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
>>
>> Ping?
> 
> Can't we just always allow migration to succeed? It's a problem of the tool
> stack above if it allows migration to an incompatible host, no?

After very convincing mail from Peter Maydell, this is the only solution,
correct? If so, I'll make a patch then.


-- 
Alexey

Re: [Qemu-devel] [RFC PATCH] target-ppc: enable migration within the same CPU family

[Michael Mueller]

On Fri, 11 Apr 2014 01:11:10 +1000
Alexey Kardashevskiy <aik@ozlabs.ru> wrote:

> On 04/04/2014 11:28 PM, Alexander Graf wrote:
> > On 04/04/2014 07:17 AM, Alexey Kardashevskiy wrote:
> >> On 03/24/2014 04:28 PM, Alexey Kardashevskiy wrote:
> >>> Currently only migration fails if CPU version is different even a bit.
> >>> For example, migration from POWER7 v2.0 to POWER7 v2.1 fails because of
> >>> that. Since there is no difference between CPU versions which could
> >>> affect migration stream, we can safely enable it.
> >>>
> >>> This adds a helper to find the closest POWERPC family class (i.e. first
> >>> abstract class in hierarchy).
> >>>
> >>> This replaces VMSTATE_UINTTL_EQUAL statement with a custom handler which
> >>> checks if the source and destination CPUs belong to the same family and
> >>> fails if they are not.
> >>>
> >>> This adds a PVR reset to the default value as it will be overwritten
> >>> by VMSTATE_UINTTL_ARRAY(env.spr, PowerPCCPU, 1024).
> >>>
> >>> Since the actual migration format is not changed by this patch,
> >>> @version_id of vmstate_ppc_cpu does not have to be changed either.
> >>>
> >>> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> >>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> >>
> >> Ping?
> > 
> > Can't we just always allow migration to succeed? It's a problem of the tool
> > stack above if it allows migration to an incompatible host, no?

That is the way we go with s390x. Qemu will just give hints by means of QMP queries what the
source model is and what models the target is capable to support. The only specific is, that
s390x is capable to limit the target cpu model. That means although the host is a 2827-GA2, the
KVM might look like an older model (2817-GA2). That simplifies the implementation in the above
management stack to a simple string compare without the need of model specific knowledge in terms
of compatibility.

> 
> After very convincing mail from Peter Maydell, this is the only solution,
> correct? If so, I'll make a patch then.
> 
> 

Re: [Qemu-devel] [RFC PATCH] target-ppc: enable migration within the same CPU family

[Alexey Kardashevskiy]

On 04/08/2014 08:32 PM, Michael Mueller wrote:
> On Tue, 08 Apr 2014 20:04:42 +1000
> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> 
>> On 04/08/2014 07:47 PM, Michael Mueller wrote:
>>> On Tue, 08 Apr 2014 11:23:14 +1000
>>> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
>>>
>>>> On 04/08/2014 04:53 AM, Andreas Färber wrote:
>>>>> Am 07.04.2014 05:27, schrieb Alexey Kardashevskiy:
>>>>>> On 04/04/2014 11:28 PM, Alexander Graf wrote:
>>>>>>> On 04/04/2014 07:17 AM, Alexey Kardashevskiy wrote:
>>>>>>>> On 03/24/2014 04:28 PM, Alexey Kardashevskiy wrote:
>>>>>>>>> Currently only migration fails if CPU version is different even a bit.
>>>>>>>>> For example, migration from POWER7 v2.0 to POWER7 v2.1 fails because of
>>>>>>>>> that. Since there is no difference between CPU versions which could
>>>>>>>>> affect migration stream, we can safely enable it.
>>>>>>>>>
>>>>>>>>> This adds a helper to find the closest POWERPC family class (i.e. first
>>>>>>>>> abstract class in hierarchy).
>>>>>>>>>
>>>>>>>>> This replaces VMSTATE_UINTTL_EQUAL statement with a custom handler which
>>>>>>>>> checks if the source and destination CPUs belong to the same family and
>>>>>>>>> fails if they are not.
>>>>>>>>>
>>>>>>>>> This adds a PVR reset to the default value as it will be overwritten
>>>>>>>>> by VMSTATE_UINTTL_ARRAY(env.spr, PowerPCCPU, 1024).
>>>>>>>>>
>>>>>>>>> Since the actual migration format is not changed by this patch,
>>>>>>>>> @version_id of vmstate_ppc_cpu does not have to be changed either.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
>>>>>>>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
>>>>>>>>
>>>>>>>> Ping?
>>>>>>>
>>>>>>> Can't we just always allow migration to succeed? It's a problem of the tool
>>>>>>> stack above if it allows migration to an incompatible host, no?
>>>>>>
>>>>>> This is not how libvirt works. It simply sends the source XML, reconstructs
>>>>>> a guest on the destination side and then migrates. hoping that the
>>>>>> migration will fail is something (which only QEMU has knowledge of) is
>>>>>> incompatible. The new guest will start with "-cpu host" (as the source) but
>>>>>> it will create diffrent CPU class and do different things. If we do not
>>>>>> check PVR (and cpu_dt_id and chip_id - the latter is coming soon) and
>>>>>> migrate power8->power7, we can easily get a broken guest.
>>>>>
>>>>> The response is very simple: -cpu host is not supported for migration.
>>>>> Same as for x86 hosts.
>>>>
>>>> Is there any good reason to limit ourselves on POWERPC?
>>>>
>>>>> As you say, the domain config is transferred by libvirt:
>>>>> If you use -cpu POWER7, you can migrate from POWER7 to POWER8 and back;
>>>>> if you use -cpu POWER8, you can only migrate on POWER8.
>>>>
>>>> -cpu other that "host" is not supported by HV KVM, only "compat" which
>>>> upstream QEMU does not have yet. So you are saying that the migration is
>>>> not supported by upstream QEMU for at least SPAPR. Well, ok, it is dead
>>>> anyway so I am fine :)
>>>>
>>>
>>> With s390x we have a similar situation. Thus we came up with a mechanism to limit
>>> the CPU functionality of a possible target system. Our patch implements CPU models
>>> based on TYPE and GA like 2817-ga1, etc. (GA represents a CPU facility set and an IBC
>>> value (Instruction Blocking Control, reduces the instruction set to the requested
>>> level)) When a guest is started, it receives its CPU model by means of option -cpu.
>>> "host" equates the configuration of the current system. We implemented "query-cpu-model"
>>> returning the actual model, here maybe { name: "2817-ga1" }. To find a suitable
>>> migration target in a remote CEC, libvirt has to "query-cpu-definitions" returning a
>>> list of models supported by the target system "{{name: "2827-ga2"}, {name: "2827-ga1"},
>>> {name: "2817-ga2"},...]. A match means the system is suitable and can be used
>>> as migration target.
>>
>> Sorry, I do not follow you. You hacked libvirt to run the destination QEMU
>> with a specific CPU model? Or it is in QEMU? Where? What I see now is this:
>>
>> static const VMStateDescription vmstate_s390_cpu = {
>>     .name = "cpu",
>>     .unmigratable = 1,
>> };
>>
>> Does not look like it supports migration :) Thanks!
>>
> 
> The code you're missing is not upstream yet. 


Is it in some maillist or git (IBM internal?)? I just want to look at some
details. Thanks!


> The s390x guest can be migrated in the meantime.
> Yes, libvirt currently gets an extension to be able to identify and startup suitable migration
> targets for s390x on behalf of the mentioned qemu cpu model. BTW can you point me to the above
> mentioned SPAPR stuff...
> 
> Michael
> 


-- 
Alexey

Re: [Qemu-devel] [RFC PATCH] target-ppc: enable migration within the same CPU family

[Michael Mueller]

On Fri, 11 Apr 2014 02:03:29 +1000
Alexey Kardashevskiy <aik@ozlabs.ru> wrote:

> On 04/08/2014 08:32 PM, Michael Mueller wrote:
> > On Tue, 08 Apr 2014 20:04:42 +1000
> > Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> > 
> >> On 04/08/2014 07:47 PM, Michael Mueller wrote:
> >>> On Tue, 08 Apr 2014 11:23:14 +1000
> >>> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> >>>
> >>>> On 04/08/2014 04:53 AM, Andreas Färber wrote:
> >>>>> Am 07.04.2014 05:27, schrieb Alexey Kardashevskiy:
> >>>>>> On 04/04/2014 11:28 PM, Alexander Graf wrote:
> >>>>>>> On 04/04/2014 07:17 AM, Alexey Kardashevskiy wrote:
> >>>>>>>> On 03/24/2014 04:28 PM, Alexey Kardashevskiy wrote:
> >>>>>>>>> Currently only migration fails if CPU version is different even a bit.
> >>>>>>>>> For example, migration from POWER7 v2.0 to POWER7 v2.1 fails because of
> >>>>>>>>> that. Since there is no difference between CPU versions which could
> >>>>>>>>> affect migration stream, we can safely enable it.
> >>>>>>>>>
> >>>>>>>>> This adds a helper to find the closest POWERPC family class (i.e. first
> >>>>>>>>> abstract class in hierarchy).
> >>>>>>>>>
> >>>>>>>>> This replaces VMSTATE_UINTTL_EQUAL statement with a custom handler which
> >>>>>>>>> checks if the source and destination CPUs belong to the same family and
> >>>>>>>>> fails if they are not.
> >>>>>>>>>
> >>>>>>>>> This adds a PVR reset to the default value as it will be overwritten
> >>>>>>>>> by VMSTATE_UINTTL_ARRAY(env.spr, PowerPCCPU, 1024).
> >>>>>>>>>
> >>>>>>>>> Since the actual migration format is not changed by this patch,
> >>>>>>>>> @version_id of vmstate_ppc_cpu does not have to be changed either.
> >>>>>>>>>
> >>>>>>>>> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> >>>>>>>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> >>>>>>>>
> >>>>>>>> Ping?
> >>>>>>>
> >>>>>>> Can't we just always allow migration to succeed? It's a problem of the tool
> >>>>>>> stack above if it allows migration to an incompatible host, no?
> >>>>>>
> >>>>>> This is not how libvirt works. It simply sends the source XML, reconstructs
> >>>>>> a guest on the destination side and then migrates. hoping that the
> >>>>>> migration will fail is something (which only QEMU has knowledge of) is
> >>>>>> incompatible. The new guest will start with "-cpu host" (as the source) but
> >>>>>> it will create diffrent CPU class and do different things. If we do not
> >>>>>> check PVR (and cpu_dt_id and chip_id - the latter is coming soon) and
> >>>>>> migrate power8->power7, we can easily get a broken guest.
> >>>>>
> >>>>> The response is very simple: -cpu host is not supported for migration.
> >>>>> Same as for x86 hosts.
> >>>>
> >>>> Is there any good reason to limit ourselves on POWERPC?
> >>>>
> >>>>> As you say, the domain config is transferred by libvirt:
> >>>>> If you use -cpu POWER7, you can migrate from POWER7 to POWER8 and back;
> >>>>> if you use -cpu POWER8, you can only migrate on POWER8.
> >>>>
> >>>> -cpu other that "host" is not supported by HV KVM, only "compat" which
> >>>> upstream QEMU does not have yet. So you are saying that the migration is
> >>>> not supported by upstream QEMU for at least SPAPR. Well, ok, it is dead
> >>>> anyway so I am fine :)
> >>>>
> >>>
> >>> With s390x we have a similar situation. Thus we came up with a mechanism to limit
> >>> the CPU functionality of a possible target system. Our patch implements CPU models
> >>> based on TYPE and GA like 2817-ga1, etc. (GA represents a CPU facility set and an IBC
> >>> value (Instruction Blocking Control, reduces the instruction set to the requested
> >>> level)) When a guest is started, it receives its CPU model by means of option -cpu.
> >>> "host" equates the configuration of the current system. We implemented "query-cpu-model"
> >>> returning the actual model, here maybe { name: "2817-ga1" }. To find a suitable
> >>> migration target in a remote CEC, libvirt has to "query-cpu-definitions" returning a
> >>> list of models supported by the target system "{{name: "2827-ga2"}, {name: "2827-ga1"},
> >>> {name: "2817-ga2"},...]. A match means the system is suitable and can be used
> >>> as migration target.
> >>
> >> Sorry, I do not follow you. You hacked libvirt to run the destination QEMU
> >> with a specific CPU model? Or it is in QEMU? Where? What I see now is this:
> >>
> >> static const VMStateDescription vmstate_s390_cpu = {
> >>     .name = "cpu",
> >>     .unmigratable = 1,
> >> };
> >>
> >> Does not look like it supports migration :) Thanks!
> >>
> > 
> > The code you're missing is not upstream yet. 
> 
> 
> Is it in some maillist or git (IBM internal?)? I just want to look at some
> details. Thanks!

I had sent out an very early version of the user space patch already:

http://lists.nongnu.org/archive/html/qemu-devel/2013-10/msg00262.html

It does not contain the IBC part because it wasn't legally cleared up to then.

But as you are with IBM I think we can give you access to our internal GIT. I have to
check with Christian Borntraeger tomorrow on how we can most easily handle it.  

> 
> 
> > The s390x guest can be migrated in the meantime.
> > Yes, libvirt currently gets an extension to be able to identify and startup suitable migration
> > targets for s390x on behalf of the mentioned qemu cpu model. BTW can you point me to the above
> > mentioned SPAPR stuff...
> > 
> > Michael
> > 
> 
>