From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51523) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WFwvg-0003bv-VG for qemu-devel@nongnu.org; Tue, 18 Feb 2014 21:30:42 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WFwvX-0007oH-TS for qemu-devel@nongnu.org; Tue, 18 Feb 2014 21:30:32 -0500 Received: from e8.ny.us.ibm.com ([32.97.182.138]:51832) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WFwvX-0007no-P6 for qemu-devel@nongnu.org; Tue, 18 Feb 2014 21:30:23 -0500 Received: from /spool/local by e8.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 18 Feb 2014 21:30:21 -0500 Received: from b01cxnp22033.gho.pok.ibm.com (b01cxnp22033.gho.pok.ibm.com [9.57.198.23]) by d01dlp01.pok.ibm.com (Postfix) with ESMTP id 1B1CE38C8045 for ; Tue, 18 Feb 2014 21:30:19 -0500 (EST) Received: from d01av05.pok.ibm.com (d01av05.pok.ibm.com [9.56.224.195]) by b01cxnp22033.gho.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id s1J2UJbV7471372 for ; Wed, 19 Feb 2014 02:30:19 GMT Received: from d01av05.pok.ibm.com (localhost [127.0.0.1]) by d01av05.pok.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id s1J2UI9p031461 for ; Tue, 18 Feb 2014 21:30:18 -0500 Message-ID: <53041734.1080801@linux.vnet.ibm.com> Date: Wed, 19 Feb 2014 10:30:12 +0800 From: "Michael R. Hines" MIME-Version: 1.0 References: <1392702802-11592-1-git-send-email-mrhines@linux.vnet.ibm.com> <530362B5.8070908@redhat.com> In-Reply-To: <530362B5.8070908@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [RFC PATCH v1 0/3] provenance: save migration stats after completion to destination List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Eric Blake , qemu-devel@nongnu.org Cc: hinesmr@cn.ibm.com, "Michael R. Hines" , quintela@redhat.com On 02/18/2014 09:40 PM, Eric Blake wrote: > On 02/17/2014 10:53 PM, mrhines@linux.vnet.ibm.com wrote: >> From: "Michael R. Hines" >> >> This series allows us to send the contents of the entire MigrationInfo structure to the >> destination once the migration is over. This is very useful for analyzing >> the result of a migration, and particularly useful for management software and >> policy. Normally, the information ins MigrationInfo is completely lost when >> the source VM is destroyed, but with this series, we can preserve the statistics >> for retrieval. >> >> The QMP architecture has sufficient functionality to serialize and >> deserialize arbitrary objects into JSON and back again (very nice). >> >> We only store the stats from the *last* migration, but not all previous migrations, >> as that could potentially get unwieldy. Thus, it's the management software's (or user's) >> responsibility to retrieve the statistics before another migration occurs. > I like the idea. It seems a little awkward that the final stats are > only on the destination, not the source, since it was the source that > triggered the migration. But on further thought, migration itself is > one-way (destination never sends anything to the source), so destination > really is the only point that can see the full picture without making > the migration protocol changed to be two-way. Meanwhile, libvirt should > be able to cope with it (libvirt does a handshake between source and > destination, and does not kill the source until after the destination > acknowledges success - so libvirt could query the destination stats, > then pass them back over the handshake, so that libvirt could present > the final stats from the source even though qemu can't). Yes, exactly. That makes sense. I would be happy to scrape together a libvirt patch that does that. Sending the stats back over the handshake seems very promising. > Now, as to your assertion that serializing the MigrationInfo will always > work - I'm not quite sure about that. Let's suppose we add a new field > to the struct in qemu 2.1. For back-compat reasons, the new field MUST > be marked as optional in the qapi schema. Here are the possibilities: > > qemu 2.0 -> 2.0: pass the smaller struct from source, expect the smaller > struct on dest, no problem > qemu 2.0 -> 2.1: pass the smaller struct from source, dest notices the > optional field is missing and copes with no problem > qemu 2.1 -> 2.1: pass the larger struct from source, dest handles the > larger struct with no problem > qemu 2.1 -> 2.0: pass the larger struct from source, but dest is only > expecting the smaller struct. > > The last case is the one I worry about: does your implementation > gracefully ignore fields that it was not expecting when reconstituting > the MigrationInfo on the dest, or does it error out, losing all > information in the process? > > On the other hand, upstream qemu seldom worries about down-version > migrations (we strive hard to make sure 2.0 -> 2.1 works, but aren't too > worried if 2.1 -> 2.0 fails) - it tends to be more of a situation that > downstream distros provide value added by worrying about down-migration. > So my concern about what happens on down-migration is not a > show-stopper for your patch idea. > Excellent question! I had not even considered that. I think this could be solved with QObject arcitecture: So when the statistics are received by the destination and deserialized, the conversion from JSON to QObject would need to check to see if the struct has all the expected fields, and if those fields are not there then do not "bomb" or anything. A deeper question would be: Let's assume a migrate to a lower version, as in your example: Should the QMP statistics also include the version of of the source QEMU that the guest originated from? I could easily modify the patch to include that value....... - Michael