From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:44821) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Wkw2W-0007ZD-NQ for qemu-devel@nongnu.org; Thu, 15 May 2014 09:49:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Wkw2O-0000Lc-8Q for qemu-devel@nongnu.org; Thu, 15 May 2014 09:49:40 -0400 Received: from e06smtp13.uk.ibm.com ([195.75.94.109]:59178) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Wkw2N-0000LY-Tw for qemu-devel@nongnu.org; Thu, 15 May 2014 09:49:32 -0400 Received: from /spool/local by e06smtp13.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 15 May 2014 14:49:30 +0100 Received: from b06cxnps4075.portsmouth.uk.ibm.com (d06relay12.portsmouth.uk.ibm.com [9.149.109.197]) by d06dlp02.portsmouth.uk.ibm.com (Postfix) with ESMTP id 13BE62190041 for ; Thu, 15 May 2014 14:49:18 +0100 (BST) Received: from d06av06.portsmouth.uk.ibm.com (d06av06.portsmouth.uk.ibm.com [9.149.37.217]) by b06cxnps4075.portsmouth.uk.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id s4FDnQnV52166760 for ; Thu, 15 May 2014 13:49:26 GMT Received: from d06av06.portsmouth.uk.ibm.com (localhost [127.0.0.1]) by d06av06.portsmouth.uk.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id s4FDnQeY004581 for ; Thu, 15 May 2014 07:49:26 -0600 Date: Thu, 15 May 2014 15:49:24 +0200 From: Greg Kurz Message-ID: <20140515154924.6502cd43@bahia.local> In-Reply-To: <5374AC4F.4000507@suse.de> References: <20140514154137.10746.94708.stgit@bahia.local> <20140515060425.GA31192@grmbl.mre> <20140515062351.GB14456@redhat.com> <20140515064635.GB31192@grmbl.mre> <20140515090449.2db0cbe0@bahia.local> <537486D2.1060609@suse.de> <20140515095208.GB16405@redhat.com> <53748FC1.5010702@suse.de> <20140515100309.GA22407@redhat.com> <537492C0.9030100@suse.de> <20140515101651.GB22512@redhat.com> <5374AC4F.4000507@suse.de> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Andreas =?UTF-8?B?RsOkcmJlcg==?= Cc: Kevin Wolf , Fam Zheng , Stefan Hajnoczi , Juan Quintela , "Michael S. Tsirkin" , Alexander Graf , qemu-devel@nongnu.org, Anthony Liguori , Amit Shah , Paolo Bonzini On Thu, 15 May 2014 14:00:15 +0200 Andreas F=C3=A4rber wrote: > Am 15.05.2014 12:16, schrieb Michael S. Tsirkin: > > On Thu, May 15, 2014 at 12:11:12PM +0200, Andreas F=C3=A4rber wrote: > >> Am 15.05.2014 12:03, schrieb Michael S. Tsirkin: > >>> On Thu, May 15, 2014 at 11:58:25AM +0200, Andreas F=C3=A4rber wrote: > >>>> Am 15.05.2014 11:52, schrieb Michael S. Tsirkin: > >>>>> On Thu, May 15, 2014 at 11:20:18AM +0200, Andreas F=C3=A4rber wrote: > >>>>>> Am 15.05.2014 09:04, schrieb Greg Kurz: > >>>>>>> On Thu, 15 May 2014 12:16:35 +0530 > >>>>>>> Amit Shah wrote: > >>>>>>>> On (Thu) 15 May 2014 [09:23:51], Michael S. Tsirkin wrote: > >>>>>>>>> On Thu, May 15, 2014 at 11:34:25AM +0530, Amit Shah wrote: > >>>>>>>>>> On (Wed) 14 May 2014 [17:41:38], Greg Kurz wrote: > >>>>>>>>>>> Since each virtio device is streamed in its own section, the = idea is to > >>>>>>>>>>> stream subsections between the end of the device section and = the start > >>>>>>>>>>> of the next sections. This allows an older QEMU to complain a= nd exit > >>>>>>>>>>> when fed with subsections: > >>>>>>>>>>> > >>>>>>>>>>> Unknown savevm section type 5 > >>>>>>>>>>> Error -22 while loading VM state > >>>>>>>>>> > >>>>>>>>>> Please make this configurable -- either via configure or device > >>>>>>>>>> properties. That avoids having to break existing configuratio= ns that > >>>>>>>>>> work without this patch. > >>>>>> > >>>>>> Since backwards migration is not supported upstream, wouldn't it be > >>>>>> easiest to just add support for the subsection marker and skipping= to > >>>>>> the end of section in that downstream? > >>>>> > >>>>> Backwards and forwards migration need to be supported, > >>>>> customers told us repeatedly. So some downstreams support this > >>>>> and not supporting it upstream just means downstreams need > >>>>> to do their own thing. > >>>>> > >>>>> As importantly, ping-pong migration is the only > >>>>> reliable way to stress migration. > >>>>> > >>>>> So if we want to test cross-version we need it to work > >>>>> both way. > >>>>> > >>>>> Finally, the real issue and difficulty with cross-version migration= is > >>>>> making VM behave in a backwards compatible way. Serializing in a > >>>>> compatible way is a trivial problem, or would be if the code wasn't= a > >>>>> mess :) Once you do the hard part, breaking migration because of the > >>>>> trivial serialization issue is just silly. And special-casing forw= ard > >>>>> migration does not make code simpler, it really only leads to > >>>>> proliferation of hacks and lack of symmetry. > >>>>> > >>>>> So yes it's a useful feature, and no not supporting it does > >>>>> not help anyway. > >>>> > >>>> It seems you misunderstand. I was not saying it's not useful. > >>>> > >>>> My point is that VMStateSubsections added in newer versions (and thus > >>>> not existing in older versions) need to be handled for any > >>>> VMState-converted devices. So why can't we make that work for virtio= too? > >>> > >>> Sure. > >>> AFAIK the way to this works is by adding an "field_exists" callback, > >>> and not sending the section when we are in a compat mode. > >> > >> OK, but upstream always sends the latest version today. > >> So isn't that > >> just two ifs that you would need to add in save and load functions add= ed > >> here for the downstream? x86_64 is unaffected from ppc's endianness > >> changes and you still do ppc64 BE, so there's no real functional probl= em > >> for RHEL, is there? > >=20 > > I'm sorry I don't understand what you are saying here. > > Simply put, if you specify a compatible -M then your > > device should behave, and migrate, exactly like and old > > qemu did. >=20 > Whatever the version_id fields are for, upstream QEMU *always* saves the > newest version_id format that it knows. There is no mechanism that I'm > aware of in upstream QEMU for migrating device fields dependent on -M. > So a device in QEMU only migrates exactly like an old QEMU did if > neither fields nor subsections were added. >=20 > Subsections are usually migrated based on whether the state differs from > the default state (didn't check whether the final patch does this > correctly here, but doesn't matter for 1/8 concept). So for x86 the For this first round, subsections are sent unconditionally. I will change that. > subsection should *never* get written and thus not be a problem for you. > For ppc64 it should not get written either, unless you care about > migration from SLES12 or upstream ppc64le to RHEL ppc64. > As I understood the IRC discussion Alex and me had with Greg about this, > this is copying the exact code VMState uses to write its subsections, so > there would be no forward migration problems at all once we convert to > VMState and VMStateSubsections. The only question remaining is how does > your downstream react when it encounters a subsection marker for > subsections it doesn't know - which is not something upstream can solve > for you unless you contribute backwards migration support upstream. >=20 > Don't forget that Greg is introducing subsection support *because of* > your backwards migration wish, so telling him not to add subsections > because of backwards migration is really weird! Either subsections work > with your downstream or they don't; if they don't, then you have much > larger problems than can be solved by blocking this one section for ppc. > Therefore I was saying if your downstream forgot to handle the case of a > non-VMState device having subsections, then it may be easier to fix > exactly that than make Greg jump through more hoops here for a > theoretical problem you are unlikely to encounter on your downstream. >=20 > Andreas >=20 --=20 Gregory Kurz kurzgreg@fr.ibm.com gkurz@linux.vnet.ibm.com Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.