Re: [Qemu-devel] [PATCH] virtio: add some migration doc

[Jason Wang <jasowang@redhat.com>]

On 09/17/2015 05:06 PM, Cornelia Huck wrote:
> Try to cover the basics of virtio migration.
>
> Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
> ---
> It might help if we add some documentation; at the very least, it will
> prevent myself getting a headache everytime I look at that code :)
>
> Feedback welcome.
> ---
>  docs/virtio-migration.txt | 101 ++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 101 insertions(+)
>  create mode 100644 docs/virtio-migration.txt

Thanks for the patch. Looks good to me. Keeping this doc to be synced
with code is important in the future. E.g I add a new subsection in core
with my pci virtio 1.0 fixes.

>
> diff --git a/docs/virtio-migration.txt b/docs/virtio-migration.txt
> new file mode 100644
> index 0000000..9c575e6
> --- /dev/null
> +++ b/docs/virtio-migration.txt
> @@ -0,0 +1,101 @@
> +Virtio devices and migration
> +============================
> +
> +Saving and restoring the state of virtio devices is a bit of a twisty maze,
> +for several reasons:
> +- state is distributed between several parts:
> +  - virtio core, for common fields like features, number of queues, ...
> +  - virtio transport (pci, ccw, ...), for the different proxy devices and
> +    transport specific state (msix vectors, indicators, ...)
> +  - virtio device (net, blk, ...), for the different device types and their
> +    state (mac address, request queue, ...)
> +- most fields are saved via the stream interface; subsequently, subsections
> +  have been added to make cross-version migration possible
> +
> +This file attempts to document the current procedure and point out some
> +caveats.
> +
> +
> +Save state procedure
> +====================
> +
> +virtio core               virtio transport          virtio device
> +-----------               ----------------          -------------
> +
> +                                                    save() function registered
> +                                                    via register_savevm()
> +virtio_save()                                       <----------
> +             ------>      save_config()
> +                          - save proxy device
> +                          - save transport-specific
> +                            device fields
> +- save common device
> +  fields
> +- save common virtqueue
> +  fields
> +             ------>      save_queue()
> +                          - save transport-specific
> +                            virtqueue fields
> +             ------>                               save_device()
> +                                                   - save device-specific
> +                                                     fields
> +- save subsections
> +  - device endianness,
> +    if changed from
> +    default endianness
> +  - 64 bit features, if
> +    any high feature bit
> +    is set
> +  - virtio-1 virtqueue
> +    fields, if VERSION_1
> +    is set
> +
> +
> +Load state procedure
> +====================
> +
> +virtio core               virtio transport          virtio device
> +-----------               ----------------          -------------
> +
> +                                                    load() function registered
> +                                                    via register_savevm()
> +virtio_load()                                       <----------
> +             ------>      load_config()
> +                          - load proxy device
> +                          - load transport-specific
> +                            device fields
> +- load common device
> +  fields
> +- load common virtqueue
> +  fields
> +             ------>      load_queue()
> +                          - load transport-specific
> +                            virtqueue fields
> +- notify guest
> +             ------>                               load_device()
> +                                                   - load device-specific
> +                                                     fields
> +- load subsections
> +  - device endianness
> +  - 64 bit features
> +  - virtio-1 virtqueue
> +    fields
> +- sanitize endianness
> +- sanitize features
> +- virtqueue index sanity
> +  check
> +                                                   - feature-dependent setup
> +
> +
> +Implications of this setup
> +==========================
> +
> +Devices need to be careful in their state processing during load: The
> +load_device() procedure is invoked by the core before subsections have
> +been loaded. Any code that depends on information transmitted in subsections
> +therefore has to be invoked in the device's load() function _after_
> +virtio_load() returned (like e.g. code depending on features).
> +
> +Any extension of the state being migrated should be done in subsections
> +added to the core for compatibility reasons. If transport or device specific
> +state is added, core needs to invoke a callback from the new subsection.