From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53426) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZNecN-0004P5-QP for qemu-devel@nongnu.org; Fri, 07 Aug 2015 06:11:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZNecJ-0002VZ-Oz for qemu-devel@nongnu.org; Fri, 07 Aug 2015 06:11:15 -0400 Received: from mx2.parallels.com ([199.115.105.18]:58126) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZNecJ-0002OG-GD for qemu-devel@nongnu.org; Fri, 07 Aug 2015 06:11:11 -0400 Message-ID: <55C4842D.7030805@virtuozzo.com> Date: Fri, 7 Aug 2015 13:10:53 +0300 From: Vladimir Sementsov-Ogievskiy MIME-Version: 1.0 References: <1438939964-12584-1-git-send-email-vsementsov@virtuozzo.com> In-Reply-To: <1438939964-12584-1-git-send-email-vsementsov@virtuozzo.com> Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH v6 00/12] Dirty bitmaps migration List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: kwolf@redhat.com, peter.maydell@linaro.org, quintela@redhat.com, dgilbert@redhat.com, vsementsov@parallels.com, stefanha@redhat.com, pbonzini@redhat.com, amit.shah@redhat.com, den@openvz.org, jsnow@redhat.com Sorry for 'vsementsov@parallels.com' in patches. I've just fixed this locally and I'm ready to roll v7 with all my emails fixed. I'm waiting for other comments. On 07.08.2015 12:32, Vladimir Sementsov-Ogievskiy wrote: > From: Vladimir Sementsov-Ogievskiy > > These patches provide dirty bitmap migration feature. Only named dirty > bitmaps are to be migrated. Migration may be enabled using migration > capabilities. > > v6: > - rebase on master > - function qemu_get_string() deleted, used existed function > qemu_get_counted_string() > - function qemu_put_string() renamed appropriately to > qemu_put_counted_string() > (and now it is new separate patch) > ===================================================== > 0004: Reviewed-by: John Snow > 0006: (since 2.3) -> (since 2.4) > 0007: new patch, mentioned above > 0008: printf format fixed, DEBUG definition set back to 0, in > dirty_bitmap_live_iterate_is_active conditions swapped. > qemu_{get,put}_string removed, used ..counted.. verisions > 0011: (since 2.3) -> (since 2.4) > 0012: assertIsNotNone -> assertNotEqual for compitibility with > old Python > Reviewed-by: John Snow > > > v5: > - rebase on master > - drop [PATCH RFC v4 10/13] iotests: add event_wait to VM class > - remove rfc, as incremental backup series by John Snow are in > upstream > > > v4 significant changes: > 0001: tiny bugfix: out[i] -> out[i-start], same for 'in' > 0007: set chunk size to 1kb, disable live iteration for > migrating data < 1mb size. > > tests: only one with md5 sum is here. used function event_wait > by John Snow. (I hope, you don't mind me just adding this > function with your 'Signed-off-by') > > rfc: This patch set is based on v13 of > "block: incremental backup series" by John Snow, which are > not pushed yet. > > v3: > based on v13 of "block: incremental backup series" by John Snow. > > changes from v2: > removed patch for adding dirty parameter (migration capablities used > instead). > > 0001: printf's dropped, qapi used > 0002: part0 -> zeroes > 0003: part0 -> zeroes > 0005: dirty_dirty -> meta > add comments about meta bitmap > > 0006: the format is changed, nodes used instead of devices. > > other patches are new. > > rfc: there are two tests. They are the same but using different > interfaces: md5 checksum of the bitmap last layer in query-block or > separate query-block-dirty-bitmap with dirty bitmap regions. > The second form is more appropriate for debugging, the first is more > appropriate for simple regression control. Which should go to > upstream? > > v2: > 1. bug-fixes, that are already in upstream, and renaming of function > bdrv_reset_dirty_bitmap (which is already in Snow's series) are > dropped > 2. bitmap store/restore: the concept renamed to serialization, added > function hbitmap_deserialize_part0, to not transfer zero blocks > 3. migration dirty parameter: added description comment > 4. Other patches are new. > > v2.rfc: > Actually, in this version of the series I'm trying not use > migration/block.c at all. Instead a separate migration unit is added > in the new file migration/dirty-bitmap.c. Now bitmaps are migrated > like blocks in block migration, they have their "dirty-dirty" bitmaps, > for tracking set/unset changes during migration. > > The advantages are: > - no complications of migration/block.c > - separate dirty-dirty bitmaps provide handling of "unset's" > - more effective meta-data/data ratio - no tiny bitmap-blocks. > > > > v1: > These patches provide dirty bitmap migration feature. Only named dirty > bitmaps are to be migrated. Migration is made as a part of block > migration in block-migration.c. > > Dirty bitmap migration may be enabled by "dirty" parameter for qmp migrate > command. If "blk" and "inc" parameters are false when "dirty" is true > block migration is actually skipped: no allocatoions, no bdrv_read's, > no bdrv_write's, only bitmaps are migrated. > > The patch set includes two my previous bug fixes, which are necessary > for it. The patch set is based on Incremental backup series by John > Snow. > Vladimir Sementsov-Ogievskiy (12): > hbitmap: serialization > block: BdrvDirtyBitmap serialization interface > block: tiny refactoring: minimize hbitmap_(set/reset) usage > block: add meta bitmaps > block: add bdrv_next_dirty_bitmap() > qapi: add dirty-bitmaps migration capability > migration: add migration/block-dirty-bitmap.c > iotests: maintain several vms in test > iotests: add add_incoming_migration to VM class > qapi: add md5 checksum of last dirty bitmap level to query-block > iotests: add dirty bitmap migration test > migration/qemu-file: make functions qemu_(get/put)_string public > > block.c | 98 +++++- > include/block/block.h | 22 ++ > include/migration/block.h | 1 + > include/migration/migration.h | 1 + > include/migration/qemu-file.h | 17 + > include/qemu/hbitmap.h | 67 ++++ > migration/Makefile.objs | 2 +- > migration/block-dirty-bitmap.c | 693 +++++++++++++++++++++++++++++++++++++++++ > migration/migration.c | 9 + > migration/qemu-file.c | 18 ++ > qapi-schema.json | 4 +- > qapi/block-core.json | 4 +- > tests/qemu-iotests/117 | 84 +++++ > tests/qemu-iotests/117.out | 5 + > tests/qemu-iotests/group | 1 + > tests/qemu-iotests/iotests.py | 14 +- > util/hbitmap.c | 106 +++++++ > vl.c | 1 + > 18 files changed, 1137 insertions(+), 10 deletions(-) > create mode 100644 migration/block-dirty-bitmap.c > create mode 100755 tests/qemu-iotests/117 > create mode 100644 tests/qemu-iotests/117.out > -- Best regards, Vladimir * now, @virtuozzo.com instead of @parallels.com. Sorry for this inconvenience.