From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40915) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Y8bgx-00072L-GH for qemu-devel@nongnu.org; Tue, 06 Jan 2015 16:29:33 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Y8bgs-0000oq-HW for qemu-devel@nongnu.org; Tue, 06 Jan 2015 16:29:31 -0500 Received: from cantor2.suse.de ([195.135.220.15]:41518 helo=mx2.suse.de) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Y8bgs-0000og-6M for qemu-devel@nongnu.org; Tue, 06 Jan 2015 16:29:26 -0500 Message-ID: <54AC53B4.8090602@suse.de> Date: Tue, 06 Jan 2015 22:29:24 +0100 From: Alexander Graf MIME-Version: 1.0 References: <1419604968-87437-1-git-send-email-agraf@suse.de> <1419604968-87437-6-git-send-email-agraf@suse.de> <54AC07AF.80108@redhat.com> In-Reply-To: <54AC07AF.80108@redhat.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH v3 5/5] Add migration stream analyzation script List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Eric Blake , qemu-devel@nongnu.org Cc: amit.shah@redhat.com, pbonzini@redhat.com, afaerber@suse.de, quintela@redhat.com -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 06.01.15 17:05, Eric Blake wrote: > On 12/26/2014 07:42 AM, Alexander Graf wrote: >> This patch adds a python tool to the scripts directory that can >> read a dumped migration stream if it contains the JSON >> description of the device states. I constructs a human readable >> JSON stream out of it. >> >> It's very simple to use: >> >> $ qemu-system-x86_64 (qemu) migrate "exec:cat > mig" $ >> ./scripts/analyze_migration.py -f mig >> >> Signed-off-by: Alexander Graf >> >> --- > >> diff --git a/scripts/analyze-migration.py >> b/scripts/analyze-migration.py new file mode 100755 index >> 0000000..3363172 --- /dev/null +++ >> b/scripts/analyze-migration.py @@ -0,0 +1,592 @@ +#!/usr/bin/env >> python +# +# Migration Stream Analyzer +# +# Copyright (c) 2013 >> Alexander Graf > > Started in 2013, but you may want to add 2014 and 2015 by the time > this patch is polished. Oops :). > >> + + # The VMSD description is at the end of the file, after >> EOF. Look for + # the last NULL byte, then for the beginning >> brace of JSON. > > Hmm, you picked up on an optimization that I missed in my > ramblings about 4/5 :) Since a well-formed JSON description > contains no NUL bytes or control characters, a pipeline read of a > migration stream can just continually look for every '\6' marker > byte followed by '{' as the potential start of an object, and reset > that search every time a '\0' or another '\6' byte is seen; > eventually, this will result in just the tail of the file being > seen as the JSON object (assuming that we never add any other tail > metadata - or that all other tail metadata is added as > backwards-compatible extensions to the JSON). It STILL might be > worth the efficiency gain of stashing an offset in the file > somewhere (as it is faster to seek to an offset than it is to scan > for particular patterns), but at least your scan works forwards > rather than in reverse. Have you tried to run the script yet? Full search, assembly and analysis of a normal sized migration stream take less than a second - and that's with in-memory assembly of all the device objects and properties (which takes the bulk of the time). I don't think we have an efficiency problem :). > >> + + # Find the last NULL byte, then the first brace after >> that. This should + # be the beginning of our JSON data. + >> nulpos = data.rfind("\0") + jsonpos = data.find("{", >> nulpos) > > Are you sure that there will NEVER be a '{' somewhere between the > last NUL and the end-of-stream marker? Shouldn't you really be > searching for the last QEMU_VM_VMDESCRIPTION ('\6') magic byte, > rather than NUL? NUL is the EOF marker, so I'm definitely hoping for it :). If we decide to add more data after EOF (which we really shouldn't I think), we can always adjust the script again IMHO. Alex -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) Comment: GPGTools - http://gpgtools.org iQIcBAEBAgAGBQJUrFO0AAoJECszeR4D/txg2wUQAIp8R22LiQzqQPrqs5VMiUbE kOLvieg0fRRHmgQ/SE7aHc/s4sC88EEKsOBintijgcNPkt8Lu65xm71JvI9660BM 9Y/ASYIliNrBKr28ThszK074xAKsvkjSBULIPH6hMUk9e6P8p5SjnKckerWWc7a/ YK7xa+1uxTaK/3+JSzV4ZKFVwbIePrEXqGxYHFcKVqmNs/OSqNwkIkErFJQCyd8D edqU/zQ4A4v+ZlJl4HZkZqXCBwUGJd6kF0ZUDmF4a7lsAELbv+MTxmVy8enBmH8p KngnDkI67tpQmlxwSYu9tBW/25qcTIiZOigGT/VShDN6cRQfbw+h3lC3klt/GKA7 ghu464ept6/xI5FQpMkdkPQ742c6lhkuQAUTVqlorVtTsc9ZF2ONLl36Y/xPNmZB R2BLDq3zDijWe6spWrfoMmc2pmtohHwyBNDtjVbI3nAkhdPddHdR2SzaCUf0B4Nj tMckMDCucGWYMGTXGMElkw9SZE8kwExPme43nSsGRaHdUVLv24kIv3lNhUlIlpBq Ljg9CMsHV6RhDySIvucfDueOpmgnyc7KGadXVEFr1pfEJrvg/SjhGDdX+ilmQeRg iCk0f0/DyJWhPskGDeKBMm9DiJ0QZ3j+Lnw+ZYnOEazyr5tOy0kTuhmPx1eDK9yg q1YJ7JGn7zVDwoBVWiXK =wERA -----END PGP SIGNATURE-----