Re: [Qemu-devel] [PATCH v2] virtio: add some migration doc

["Dr. David Alan Gilbert" <dgilbert@redhat.com>]

* Cornelia Huck (cornelia.huck@de.ibm.com) wrote:
> Try to cover the basics of virtio migration.

Thank you for doing this; I don't know the innards of virtio, but having
to debug migration with it in the mix, it is nice to have something to
look at.

Dave

> Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
> Reviewed-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
> ---
> v1->v2: make copyright explicit
> ---
>  docs/virtio-migration.txt | 106 ++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 106 insertions(+)
>  create mode 100644 docs/virtio-migration.txt
> 
> diff --git a/docs/virtio-migration.txt b/docs/virtio-migration.txt
> new file mode 100644
> index 0000000..cf66458
> --- /dev/null
> +++ b/docs/virtio-migration.txt
> @@ -0,0 +1,106 @@
> +Virtio devices and migration
> +============================
> +
> +Copyright 2015 IBM Corp.
> +
> +This work is licensed under the terms of the GNU GPL, version 2 or later.  See
> +the COPYING file in the top-level directory.
> +
> +Saving and restoring the state of virtio devices is a bit of a twisty maze,
> +for several reasons:
> +- state is distributed between several parts:
> +  - virtio core, for common fields like features, number of queues, ...
> +  - virtio transport (pci, ccw, ...), for the different proxy devices and
> +    transport specific state (msix vectors, indicators, ...)
> +  - virtio device (net, blk, ...), for the different device types and their
> +    state (mac address, request queue, ...)
> +- most fields are saved via the stream interface; subsequently, subsections
> +  have been added to make cross-version migration possible
> +
> +This file attempts to document the current procedure and point out some
> +caveats.
> +
> +
> +Save state procedure
> +====================
> +
> +virtio core               virtio transport          virtio device
> +-----------               ----------------          -------------
> +
> +                                                    save() function registered
> +                                                    via register_savevm()
> +virtio_save()                                       <----------
> +             ------>      save_config()
> +                          - save proxy device
> +                          - save transport-specific
> +                            device fields
> +- save common device
> +  fields
> +- save common virtqueue
> +  fields
> +             ------>      save_queue()
> +                          - save transport-specific
> +                            virtqueue fields
> +             ------>                               save_device()
> +                                                   - save device-specific
> +                                                     fields
> +- save subsections
> +  - device endianness,
> +    if changed from
> +    default endianness
> +  - 64 bit features, if
> +    any high feature bit
> +    is set
> +  - virtio-1 virtqueue
> +    fields, if VERSION_1
> +    is set
> +
> +
> +Load state procedure
> +====================
> +
> +virtio core               virtio transport          virtio device
> +-----------               ----------------          -------------
> +
> +                                                    load() function registered
> +                                                    via register_savevm()
> +virtio_load()                                       <----------
> +             ------>      load_config()
> +                          - load proxy device
> +                          - load transport-specific
> +                            device fields
> +- load common device
> +  fields
> +- load common virtqueue
> +  fields
> +             ------>      load_queue()
> +                          - load transport-specific
> +                            virtqueue fields
> +- notify guest
> +             ------>                               load_device()
> +                                                   - load device-specific
> +                                                     fields
> +- load subsections
> +  - device endianness
> +  - 64 bit features
> +  - virtio-1 virtqueue
> +    fields
> +- sanitize endianness
> +- sanitize features
> +- virtqueue index sanity
> +  check
> +                                                   - feature-dependent setup
> +
> +
> +Implications of this setup
> +==========================
> +
> +Devices need to be careful in their state processing during load: The
> +load_device() procedure is invoked by the core before subsections have
> +been loaded. Any code that depends on information transmitted in subsections
> +therefore has to be invoked in the device's load() function _after_
> +virtio_load() returned (like e.g. code depending on features).
> +
> +Any extension of the state being migrated should be done in subsections
> +added to the core for compatibility reasons. If transport or device specific
> +state is added, core needs to invoke a callback from the new subsection.
> -- 
> 2.3.8
> 
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK