Re: [Qemu-devel] [PATCH v2 1/1] Introduce "xen-load-devices-state"

[Changlong Xie <xiecl.fnst@cn.fujitsu.com>]

On 03/23/2016 05:41 PM, Dr. David Alan Gilbert wrote:
> * Wen Congyang (wency@cn.fujitsu.com) wrote:
>> On 03/23/2016 04:56 PM, Dr. David Alan Gilbert wrote:
>>> * Changlong Xie (xiecl.fnst@cn.fujitsu.com) wrote:
>>>> On 03/22/2016 08:22 PM, Dr. David Alan Gilbert wrote:
>>>>> * Changlong Xie (xiecl.fnst@cn.fujitsu.com) wrote:
>>>>>> From: Wen Congyang <wency@cn.fujitsu.com>
>>>>>>
>>>>>> Introduce a "xen-load-devices-state" QAPI command that can be used to
>>>>>> load the state of all devices, but not the RAM or the block devices of
>>>>>> the VM.
>>>>>>
>>>>>> We only have hmp commands savevm/loadvm, and qmp commands
>>>>>> xen-save-devices-state.
>>>>>
>>>>> Can you explain on Xen how the RAM actually gets loaded?
>>>>
>>>> Xen use xc(xen toolstack) to do RAM restore/save
>>>>
>>>>>
>>>>>>
>>>>>> We use this new command for COLO:
>>>>>> 1. suspend both primary vm and secondary vm
>>>>>> 2. sync the state
>>>>>> 3. resume both primary vm and secondary vm
>>>>>>
>>>>>> In such case, we need to update all devices' state in any time.
>>>>>>
>>>>>> Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
>>>>>> Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
>>>>>> ---
>>>>>>   migration/savevm.c | 36 ++++++++++++++++++++++++++++++++++++
>>>>>>   qapi-schema.json   | 14 ++++++++++++++
>>>>>>   qmp-commands.hx    | 27 +++++++++++++++++++++++++++
>>>>>>   3 files changed, 77 insertions(+)
>>>>>>
>>>>>> diff --git a/migration/savevm.c b/migration/savevm.c
>>>>>> index 96e7db5..aaead12 100644
>>>>>> --- a/migration/savevm.c
>>>>>> +++ b/migration/savevm.c
>>>>>> @@ -50,6 +50,7 @@
>>>>>>   #include "qemu/iov.h"
>>>>>>   #include "block/snapshot.h"
>>>>>>   #include "block/qapi.h"
>>>>>> +#include "hw/xen/xen.h"
>>>>>>
>>>>>>
>>>>>>   #ifndef ETH_P_RARP
>>>>>> @@ -1768,6 +1769,12 @@ qemu_loadvm_section_start_full(QEMUFile *f, MigrationIncomingState *mis)
>>>>>>           return -EINVAL;
>>>>>>       }
>>>>>>
>>>>>> +    /* Validate if it is a device's state */
>>>>>> +    if (xen_enabled() && se->is_ram) {
>>>>>> +        error_report("loadvm: %s RAM loading not allowed on Xen", idstr);
>>>>>> +        return -EINVAL;
>>>>>> +    }
>>>>>> +
>>>>>>       /* Add entry */
>>>>>>       le = g_malloc0(sizeof(*le));
>>>>>>
>>>>>> @@ -2077,6 +2084,35 @@ void qmp_xen_save_devices_state(const char *filename, Error **errp)
>>>>>>       }
>>>>>>   }
>>>>>>
>>>>>> +void qmp_xen_load_devices_state(const char *filename, Error **errp)
>>>>>> +{
>>>>>> +    QEMUFile *f;
>>>>>> +    int saved_vm_running;
>>>>>> +    int ret;
>>>>>> +
>>>>>> +    saved_vm_running = runstate_is_running();
>>>>>> +    vm_stop(RUN_STATE_RESTORE_VM);
>>>>>> +
>>>>>> +    f = qemu_fopen(filename, "rb");
>>>>>> +    if (!f) {
>>>>>> +        error_setg_file_open(errp, errno, filename);
>>>>>> +        goto out;
>>>>>> +    }
>>>>>> +
>>>>>> +    migration_incoming_state_new(f);
>>>>>> +    ret = qemu_loadvm_state(f);
>>>>>> +    qemu_fclose(f);
>>>>>> +    migration_incoming_state_destroy();
>>>>>> +    if (ret < 0) {
>>>>>> +        error_setg(errp, QERR_IO_ERROR);
>>>>>> +    }
>>>>>> +
>>>>>> +out:
>>>>>> +    if (saved_vm_running) {
>>>>>> +        vm_start();
>>>>>> +    }
>>>>>
>>>>> Does it ever happen that you had it running immediately
>>>>> before you did this command? Somehow you'd have to have loaded the RAM
>>>>
>>>> No, we suspend vm before running this command in xen to make sure the
>>>> condition that you discribed never happen.
>>>
>>> OK, so then if the VM is suspended, what does the:
>>>
>>>      if (saved_vm_running) {
>>>          vm_start();
>>>      }
>>>
>>> at the end of your routine do?
>>
>> +    saved_vm_running = runstate_is_running();
>> +    vm_stop(RUN_STATE_RESTORE_VM);
>>
>> It is copied from the other codes in qemu.
>> I think we should return failure if the vm is running:
>> if (runstate_is_running()) {
>>      error_setg(xxx);
>>      return;
>> }
>
> OK, yes; and add a comment to explain.

+    if (runstate_is_running()) {
+        error_setg(errp, "Cannot update device state while vm is running");
+        return;
+    }

Will fix in next version.

Thanks
	-Xie
>
> Dave
>
>>
>> Thanks
>> Wen Congyang
>>
>>>
>>> Dave
>>>
>>>
>>>>
>>>> Thanks
>>>> 	-Xie
>>>>
>>>>> at just the right point, and I don't see how that would happen if the guest
>>>>> was running.
>>>>>
>>>>> Dave
>>>>>
>>>>>> +}
>>>>>> +
>>>>>>   int load_vmstate(const char *name)
>>>>>>   {
>>>>>>       BlockDriverState *bs, *bs_vm_state;
>>>>>> diff --git a/qapi-schema.json b/qapi-schema.json
>>>>>> index 362c9d8..8cca59d 100644
>>>>>> --- a/qapi-schema.json
>>>>>> +++ b/qapi-schema.json
>>>>>> @@ -4122,3 +4122,17 @@
>>>>>>   ##
>>>>>>   { 'enum': 'ReplayMode',
>>>>>>     'data': [ 'none', 'record', 'play' ] }
>>>>>> +
>>>>>> +##
>>>>>> +# @xen-load-devices-state:
>>>>>> +#
>>>>>> +# Load the state of all devices from file. The RAM and the block devices
>>>>>> +# of the VM are not loaded by this command.
>>>>>> +#
>>>>>> +# @filename: the file to load the state of the devices from as binary
>>>>>> +# data. See xen-save-devices-state.txt for a description of the binary
>>>>>> +# format.
>>>>>> +#
>>>>>> +# Since: 2.7
>>>>>> +##
>>>>>> +{ 'command': 'xen-load-devices-state', 'data': {'filename': 'str'} }
>>>>>> diff --git a/qmp-commands.hx b/qmp-commands.hx
>>>>>> index b629673..4925702 100644
>>>>>> --- a/qmp-commands.hx
>>>>>> +++ b/qmp-commands.hx
>>>>>> @@ -587,6 +587,33 @@ Example:
>>>>>>   EQMP
>>>>>>
>>>>>>       {
>>>>>> +        .name       = "xen-load-devices-state",
>>>>>> +        .args_type  = "filename:F",
>>>>>> +        .mhandler.cmd_new = qmp_marshal_xen_load_devices_state,
>>>>>> +    },
>>>>>> +
>>>>>> +SQMP
>>>>>> +xen-load-devices-state
>>>>>> +----------------------
>>>>>> +
>>>>>> +Load the state of all devices from file. The RAM and the block devices
>>>>>> +of the VM are not loaded by this command.
>>>>>> +
>>>>>> +Arguments:
>>>>>> +
>>>>>> +- "filename": the file to load the state of the devices from as binary
>>>>>> +data. See xen-save-devices-state.txt for a description of the binary
>>>>>> +format.
>>>>>> +
>>>>>> +Example:
>>>>>> +
>>>>>> +-> { "execute": "xen-load-devices-state",
>>>>>> +     "arguments": { "filename": "/tmp/resume" } }
>>>>>> +<- { "return": {} }
>>>>>> +
>>>>>> +EQMP
>>>>>> +
>>>>>> +    {
>>>>>>           .name       = "xen-set-global-dirty-log",
>>>>>>           .args_type  = "enable:b",
>>>>>>           .mhandler.cmd_new = qmp_marshal_xen_set_global_dirty_log,
>>>>>> --
>>>>>> 1.9.3
>>>>>>
>>>>>>
>>>>>>
>>>>> --
>>>>> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
>>>>>
>>>>>
>>>>> .
>>>>>
>>>>
>>>>
>>> --
>>> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
>>>
>>>
>>> .
>>>
>>
>>
>>
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
>
>
> .
>