From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53067) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zn0p2-000649-6o for qemu-devel@nongnu.org; Fri, 16 Oct 2015 04:57:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Zn0oy-0002Hb-L0 for qemu-devel@nongnu.org; Fri, 16 Oct 2015 04:57:08 -0400 Received: from e06smtp16.uk.ibm.com ([195.75.94.112]:47887) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zn0oy-0002GO-85 for qemu-devel@nongnu.org; Fri, 16 Oct 2015 04:57:04 -0400 Received: from localhost by e06smtp16.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 16 Oct 2015 09:57:03 +0100 Received: from b06cxnps4075.portsmouth.uk.ibm.com (d06relay12.portsmouth.uk.ibm.com [9.149.109.197]) by d06dlp01.portsmouth.uk.ibm.com (Postfix) with ESMTP id 7A6E217D8059 for ; Fri, 16 Oct 2015 09:57:06 +0100 (BST) Received: from d06av10.portsmouth.uk.ibm.com (d06av10.portsmouth.uk.ibm.com [9.149.37.251]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id t9G8v01X35979400 for ; Fri, 16 Oct 2015 08:57:00 GMT Received: from d06av10.portsmouth.uk.ibm.com (localhost [127.0.0.1]) by d06av10.portsmouth.uk.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id t9G7v0NC023446 for ; Fri, 16 Oct 2015 01:57:00 -0600 Date: Fri, 16 Oct 2015 10:56:58 +0200 From: Cornelia Huck Message-ID: <20151016105658.40a4d6e5.cornelia.huck@de.ibm.com> In-Reply-To: <20151007123917.20e0f218.cornelia.huck@de.ibm.com> References: <1442508177-74386-1-git-send-email-cornelia.huck@de.ibm.com> <20151007123917.20e0f218.cornelia.huck@de.ibm.com> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH v2] virtio: add some migration doc List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: mst@redhat.com Cc: jasowang@redhat.com, kraxel@redhat.com, qemu-devel@nongnu.org, gkurz@linux.vnet.ibm.com On Wed, 7 Oct 2015 12:39:17 +0200 Cornelia Huck wrote: > On Thu, 17 Sep 2015 18:42:57 +0200 > Cornelia Huck wrote: > > > Try to cover the basics of virtio migration. > > > > Signed-off-by: Cornelia Huck > > Reviewed-by: Greg Kurz > > --- > > v1->v2: make copyright explicit > > --- > > docs/virtio-migration.txt | 106 ++++++++++++++++++++++++++++++++++++++++++++++ > > 1 file changed, 106 insertions(+) > > create mode 100644 docs/virtio-migration.txt > > Michael: Do you want to take this through your tree? Ping :) > > > > > diff --git a/docs/virtio-migration.txt b/docs/virtio-migration.txt > > new file mode 100644 > > index 0000000..cf66458 > > --- /dev/null > > +++ b/docs/virtio-migration.txt > > @@ -0,0 +1,106 @@ > > +Virtio devices and migration > > +============================ > > + > > +Copyright 2015 IBM Corp. > > + > > +This work is licensed under the terms of the GNU GPL, version 2 or later. See > > +the COPYING file in the top-level directory. > > + > > +Saving and restoring the state of virtio devices is a bit of a twisty maze, > > +for several reasons: > > +- state is distributed between several parts: > > + - virtio core, for common fields like features, number of queues, ... > > + - virtio transport (pci, ccw, ...), for the different proxy devices and > > + transport specific state (msix vectors, indicators, ...) > > + - virtio device (net, blk, ...), for the different device types and their > > + state (mac address, request queue, ...) > > +- most fields are saved via the stream interface; subsequently, subsections > > + have been added to make cross-version migration possible > > + > > +This file attempts to document the current procedure and point out some > > +caveats. > > + > > + > > +Save state procedure > > +==================== > > + > > +virtio core virtio transport virtio device > > +----------- ---------------- ------------- > > + > > + save() function registered > > + via register_savevm() > > +virtio_save() <---------- > > + ------> save_config() > > + - save proxy device > > + - save transport-specific > > + device fields > > +- save common device > > + fields > > +- save common virtqueue > > + fields > > + ------> save_queue() > > + - save transport-specific > > + virtqueue fields > > + ------> save_device() > > + - save device-specific > > + fields > > +- save subsections > > + - device endianness, > > + if changed from > > + default endianness > > + - 64 bit features, if > > + any high feature bit > > + is set > > + - virtio-1 virtqueue > > + fields, if VERSION_1 > > + is set > > + > > + > > +Load state procedure > > +==================== > > + > > +virtio core virtio transport virtio device > > +----------- ---------------- ------------- > > + > > + load() function registered > > + via register_savevm() > > +virtio_load() <---------- > > + ------> load_config() > > + - load proxy device > > + - load transport-specific > > + device fields > > +- load common device > > + fields > > +- load common virtqueue > > + fields > > + ------> load_queue() > > + - load transport-specific > > + virtqueue fields > > +- notify guest > > + ------> load_device() > > + - load device-specific > > + fields > > +- load subsections > > + - device endianness > > + - 64 bit features > > + - virtio-1 virtqueue > > + fields > > +- sanitize endianness > > +- sanitize features > > +- virtqueue index sanity > > + check > > + - feature-dependent setup > > + > > + > > +Implications of this setup > > +========================== > > + > > +Devices need to be careful in their state processing during load: The > > +load_device() procedure is invoked by the core before subsections have > > +been loaded. Any code that depends on information transmitted in subsections > > +therefore has to be invoked in the device's load() function _after_ > > +virtio_load() returned (like e.g. code depending on features). > > + > > +Any extension of the state being migrated should be done in subsections > > +added to the core for compatibility reasons. If transport or device specific > > +state is added, core needs to invoke a callback from the new subsection. >