From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33015) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z60F1-0000LB-AO for qemu-devel@nongnu.org; Fri, 19 Jun 2015 13:38:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Z60Ex-0002ZV-LD for qemu-devel@nongnu.org; Fri, 19 Jun 2015 13:38:10 -0400 Received: from mx1.redhat.com ([209.132.183.28]:59098) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z60Ex-0002Z7-EA for qemu-devel@nongnu.org; Fri, 19 Jun 2015 13:38:07 -0400 Message-ID: <5584537C.1070606@redhat.com> Date: Fri, 19 Jun 2015 13:38:04 -0400 From: John Snow MIME-Version: 1.0 References: <1431531007-10269-1-git-send-email-vsementsov@parallels.com> <557B5A2A.5060301@redhat.com> <55844A17.1020306@openvz.org> In-Reply-To: <55844A17.1020306@openvz.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [PATCH v5 00/12] Dirty bitmaps migration List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Denis V. Lunev" , Vladimir Sementsov-Ogievskiy , qemu-devel@nongnu.org Cc: kwolf@redhat.com, peter.maydell@linaro.org, quintela@redhat.com, dgilbert@redhat.com, stefanha@redhat.com, amit.shah@redhat.com, pbonzini@redhat.com On 06/19/2015 12:57 PM, Denis V. Lunev wrote: > On 13/06/15 01:16, John Snow wrote: >> Ping for maintainer reviews -- I've given it a first pass and it shoul= d >> hopefully be close. >> >> Thanks, >> --js > do we need to respin? I do not see much attention from maintainers here= . >=20 > Den >=20 Not sure. Either everyone else wants to see the persistence series refined before they merge/review this, or just haven't gotten to this one yet. >=20 >> On 05/13/2015 11:29 AM, Vladimir Sementsov-Ogievskiy wrote: >>> These patches provide dirty bitmap migration feature. Only named dirt= y >>> bitmaps are to be migrated. Migration may be enabled using migration >>> capabilities. >>> >>> v5: >>> - rebase on master >>> - drop [PATCH RFC v4 10/13] iotests: add event_wait to VM class >>> - remove rfc, as incremental backup series by John Snow are in >>> upstream >>> =20 >>> v4 significant changes: >>> 0001: tiny bugfix: out[i] -> out[i-start], same for 'in' >>> 0007: set chunk size to 1kb, disable live iteration for >>> migrating data < 1mb size. >>> >>> tests: only one with md5 sum is here. used function event_wait >>> by John Snow. (I hope, you don't mind me just adding this >>> function with your 'Signed-off-by') >>> >>> rfc: This patch set is based on v13 of >>> "block: incremental backup series" by John Snow, which are >>> not pushed yet. >>> >>> v3: >>> based on v13 of "block: incremental backup series" by John Snow. >>> >>> changes from v2: >>> removed patch for adding dirty parameter (migration capablities use= d >>> instead). >>> 0001: printf's dropped, qapi used >>> 0002: part0 -> zeroes >>> 0003: part0 -> zeroes >>> 0005: dirty_dirty -> meta >>> add comments about meta bitmap >>> 0006: the format is changed, nodes used instead of devices. >>> >>> other patches are new. >>> >>> rfc: there are two tests. They are the same but using different >>> interfaces: md5 checksum of the bitmap last layer in query-block or >>> separate query-block-dirty-bitmap with dirty bitmap regions. >>> The second form is more appropriate for debugging, the first is mor= e >>> appropriate for simple regression control. Which should go to >>> upstream? >>> >>> v2: >>> 1. bug-fixes, that are already in upstream, and renaming of functio= n >>> bdrv_reset_dirty_bitmap (which is already in Snow's series) are >>> dropped >>> 2. bitmap store/restore: the concept renamed to serialization, adde= d >>> function hbitmap_deserialize_part0, to not transfer zero blocks >>> 3. migration dirty parameter: added description comment >>> 4. Other patches are new. >>> >>> v2.rfc: >>> Actually, in this version of the series I'm trying not use >>> migration/block.c at all. Instead a separate migration unit is added >>> in the new file migration/dirty-bitmap.c. Now bitmaps are migrated >>> like blocks in block migration, they have their "dirty-dirty" bitmaps= , >>> for tracking set/unset changes during migration. >>> >>> The advantages are: >>> - no complications of migration/block.c >>> - separate dirty-dirty bitmaps provide handling of "unset's" >>> - more effective meta-data/data ratio - no tiny bitmap-blocks. >>> >>> >>> >>> v1: >>> These patches provide dirty bitmap migration feature. Only named dirt= y >>> bitmaps are to be migrated. Migration is made as a part of block >>> migration in block-migration.c. >>> >>> Dirty bitmap migration may be enabled by "dirty" parameter for qmp >>> migrate >>> command. If "blk" and "inc" parameters are false when "dirty" is true >>> block migration is actually skipped: no allocatoions, no bdrv_read's, >>> no bdrv_write's, only bitmaps are migrated. >>> >>> The patch set includes two my previous bug fixes, which are necessary >>> for it. The patch set is based on Incremental backup series by John >>> Snow. >>> Vladimir Sementsov-Ogievskiy (12): >>> hbitmap: serialization >>> block: BdrvDirtyBitmap serialization interface >>> block: tiny refactoring: minimize hbitmap_(set/reset) usage >>> block: add meta bitmaps >>> block: add bdrv_next_dirty_bitmap() >>> qapi: add dirty-bitmaps migration capability >>> migration: add migration/block-dirty-bitmap.c >>> iotests: maintain several vms in test >>> iotests: add add_incoming_migration to VM class >>> qapi: add md5 checksum of last dirty bitmap level to query-block >>> iotests: add dirty bitmap migration test >>> migration/qemu-file: make functions qemu_(get/put)_string public >>> >>> block.c | 98 +++++- >>> include/block/block.h | 22 ++ >>> include/migration/block.h | 1 + >>> include/migration/migration.h | 1 + >>> include/migration/qemu-file.h | 17 + >>> include/qemu/hbitmap.h | 67 ++++ >>> migration/Makefile.objs | 2 +- >>> migration/block-dirty-bitmap.c | 693 >>> +++++++++++++++++++++++++++++++++++++++++ >>> migration/migration.c | 9 + >>> migration/qemu-file.c | 18 ++ >>> qapi-schema.json | 4 +- >>> qapi/block-core.json | 4 +- >>> tests/qemu-iotests/117 | 84 +++++ >>> tests/qemu-iotests/117.out | 5 + >>> tests/qemu-iotests/group | 1 + >>> tests/qemu-iotests/iotests.py | 14 +- >>> util/hbitmap.c | 106 +++++++ >>> vl.c | 1 + >>> 18 files changed, 1137 insertions(+), 10 deletions(-) >>> create mode 100644 migration/block-dirty-bitmap.c >>> create mode 100755 tests/qemu-iotests/117 >>> create mode 100644 tests/qemu-iotests/117.out >>> >=20 --=20 =97js