From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55682) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Wksag-0006gF-UM for qemu-devel@nongnu.org; Thu, 15 May 2014 06:08:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WksaY-0007nh-PU for qemu-devel@nongnu.org; Thu, 15 May 2014 06:08:42 -0400 Received: from e06smtp17.uk.ibm.com ([195.75.94.113]:55425) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WksaY-0007nS-GK for qemu-devel@nongnu.org; Thu, 15 May 2014 06:08:34 -0400 Received: from /spool/local by e06smtp17.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 15 May 2014 11:08:33 +0100 Received: from b06cxnps4075.portsmouth.uk.ibm.com (d06relay12.portsmouth.uk.ibm.com [9.149.109.197]) by d06dlp02.portsmouth.uk.ibm.com (Postfix) with ESMTP id 3F02B2190041 for ; Thu, 15 May 2014 11:08:22 +0100 (BST) Received: from d06av02.portsmouth.uk.ibm.com (d06av02.portsmouth.uk.ibm.com [9.149.37.228]) by b06cxnps4075.portsmouth.uk.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id s4FA8Vt852494522 for ; Thu, 15 May 2014 10:08:31 GMT Received: from d06av02.portsmouth.uk.ibm.com (localhost [127.0.0.1]) by d06av02.portsmouth.uk.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id s4FA8UvM010090 for ; Thu, 15 May 2014 04:08:30 -0600 Date: Thu, 15 May 2014 12:08:26 +0200 From: Greg Kurz Message-ID: <20140515120826.050cf6f8@bahia.local> In-Reply-To: <537486D2.1060609@suse.de> References: <20140514154130.10746.1412.stgit@bahia.local> <20140514154137.10746.94708.stgit@bahia.local> <20140515060425.GA31192@grmbl.mre> <20140515062351.GB14456@redhat.com> <20140515064635.GB31192@grmbl.mre> <20140515090449.2db0cbe0@bahia.local> <537486D2.1060609@suse.de> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Andreas =?UTF-8?B?RsOkcmJlcg==?= Cc: Kevin Wolf , Fam Zheng , Stefan Hajnoczi , Juan Quintela , "Michael S. Tsirkin" , Alexander Graf , qemu-devel@nongnu.org, Anthony Liguori , Amit Shah , Paolo Bonzini On Thu, 15 May 2014 11:20:18 +0200 Andreas F=C3=A4rber wrote: > Am 15.05.2014 09:04, schrieb Greg Kurz: > > On Thu, 15 May 2014 12:16:35 +0530 > > Amit Shah wrote: > >> On (Thu) 15 May 2014 [09:23:51], Michael S. Tsirkin wrote: > >>> On Thu, May 15, 2014 at 11:34:25AM +0530, Amit Shah wrote: > >>>> On (Wed) 14 May 2014 [17:41:38], Greg Kurz wrote: > >>>>> Since each virtio device is streamed in its own section, the idea i= s to > >>>>> stream subsections between the end of the device section and the st= art > >>>>> of the next sections. This allows an older QEMU to complain and exit > >>>>> when fed with subsections: > >>>>> > >>>>> Unknown savevm section type 5 > >>>>> Error -22 while loading VM state > >>>> > >>>> Please make this configurable -- either via configure or device > >>>> properties. That avoids having to break existing configurations that > >>>> work without this patch. >=20 > Since backwards migration is not supported upstream, wouldn't it be > easiest to just add support for the subsection marker and skipping to > the end of section in that downstream? >=20 Not sure I understand well... Do you suggest to stream the markers first, then the device, then the subsections ? And then there would be a way we can have the subsections restored before the device ? > >>>>> All users of virtio_load()/virtio_save() need to be patched because= the > >>>>> subsections are streamed AFTER the device itself. >=20 > IMO this is calling for inversion of control - i.e. let virtio devices > call generic load/save functions that then dispatch to device-specific > code and let us add common stuff in a central place without forgetting > to add calls in some new device. >=20 That makes a lot of sense. > >>>> Since all have the same fixup, I'm wondering if a new section can be > >>>> added to the virtio-bus itself, which gets propagated to all devices > >>>> upon load in the dest. > >>> > >>> This calls for a way for devices to inherit properties from the bus, > >>> which doesn't exist ATM. > >>> Fine but let's not hold up this patchset because of this. > >> > >> No, only suggestion is to add a migration section in the bus, and then > >> it's easier to do this in the post-migrate functions for each device > >> -- so only one new section gets introduced instead of all devices > >> being modified to send a new subsection. > >> > >=20 > > The main problem I see is that virtio sucks: as you see in patch 8, we = have > > to be careful not to call vring or virtqueue stuff before the device kn= ows > > its endianness or it breaks... I need to study how the virtio-bus gets > > migrated to ensure the endian section is streamed before the devices. >=20 > There is no ordering guarantee. The state needs to be migrated in the > device or bus where it sits, if post-load processing is required; i.e., > if it's in VirtIODevice then something like this series, if it were on > VirtioBus exclusively (device asking bus for its endianness each time > and does not do post-load stuff) then endianness could be migrated as a > new bus section. Not sure if that would help the "broken" state though? >=20 IIRW the "broken" state was proposed as a per-device property... Fam, Do you have plans about the "broken" property ? Is it still needed ? > Would touch on Stefan's alias properties for anything but virtio-mmio. >=20 OMG... maybe I should hold on then. > Regards, > Andreas >=20 Thanks ! --=20 Gregory Kurz kurzgreg@fr.ibm.com gkurz@linux.vnet.ibm.com Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.