From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36003) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YO5CE-00089x-H1 for qemu-devel@nongnu.org; Wed, 18 Feb 2015 09:01:51 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YO5C2-0006Pe-39 for qemu-devel@nongnu.org; Wed, 18 Feb 2015 09:01:46 -0500 Received: from mailhub.sw.ru ([195.214.232.25]:34950 helo=relay.sw.ru) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YO5C1-0005gi-O5 for qemu-devel@nongnu.org; Wed, 18 Feb 2015 09:01:34 -0500 From: Vladimir Sementsov-Ogievskiy Date: Wed, 18 Feb 2015 17:00:00 +0300 Message-Id: <1424268014-13293-1-git-send-email-vsementsov@parallels.com> Subject: [Qemu-devel] [PATCH RFC v3 00/14] Dirty bitmaps migration List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: kwolf@redhat.com, peter.maydell@linaro.org, quintela@redhat.com, dgilbert@redhat.com, vsementsov@parallels.com, stefanha@redhat.com, den@openvz.org, amit.shah@redhat.com, pbonzini@redhat.com v3: based on v13 of "block: incremental backup series" by John Snow. changes from v2: removed patch for adding dirty parameter (migration capablities used instead). 0001: printf's dropped, qapi used 0002: part0 -> zeroes 0003: part0 -> zeroes 0005: dirty_dirty -> meta add comments about meta bitmap 0006: the format is changed, nodes used instead of devices. other patches are new. rfc: there are two tests. They are the same but using different interfaces: md5 checksum of the bitmap last layer in query-block or separate query-block-dirty-bitmap with dirty bitmap regions. The second form is more appropriate for debugging, the first is more appropriate for simple regression control. Which should go to upstream? v2: 1. bug-fixes, that are already in upstream, and renaming of function bdrv_reset_dirty_bitmap (which is already in Snow's series) are dropped 2. bitmap store/restore: the concept renamed to serialization, added function hbitmap_deserialize_part0, to not transfer zero blocks 3. migration dirty parameter: added description comment 4. Other patches are new. v2.rfc: Actually, in this version of the series I'm trying not use migration/block.c at all. Instead a separate migration unit is added in the new file migration/dirty-bitmap.c. Now bitmaps are migrated like blocks in block migration, they have their "dirty-dirty" bitmaps, for tracking set/unset changes during migration. The advantages are: - no complications of migration/block.c - separate dirty-dirty bitmaps provide handling of "unset's" - more effective meta-data/data ratio - no tiny bitmap-blocks. v1: These patches provide dirty bitmap migration feature. Only named dirty bitmaps are to be migrated. Migration is made as a part of block migration in block-migration.c. Dirty bitmap migration may be enabled by "dirty" parameter for qmp migrate command. If "blk" and "inc" parameters are false when "dirty" is true block migration is actually skipped: no allocatoions, no bdrv_read's, no bdrv_write's, only bitmaps are migrated. The patch set includes two my previous bug fixes, which are necessary for it. The patch set is based on Incremental backup series by John Snow. Vladimir Sementsov-Ogievskiy (14): qmp: add query-block-dirty-bitmap hbitmap: serialization block: BdrvDirtyBitmap serialization interface block: tiny refactoring: minimize hbitmap_(set/reset) usage block: add meta bitmaps block: add bdrv_next_dirty_bitmap() qapi: add dirty-bitmaps migration capability migration: add migration/block-dirty-bitmap.c iotests: maintain several vms in test iotests: add add_incoming_migration to VM class iotests: add dirty bitmap migration test qapi: add md5 checksum of last dirty bitmap level to query-block iotests: add dirty bitmap migration test migration/qemu-file: make functions qemu_(get/put)_string public block.c | 135 ++++++++- blockdev.c | 24 ++ hmp-commands.hx | 2 + hmp.c | 32 ++ hmp.h | 1 + include/block/block.h | 22 ++ include/migration/block.h | 1 + include/migration/migration.h | 1 + include/migration/qemu-file.h | 17 ++ include/qemu/hbitmap.h | 67 ++++ migration/Makefile.objs | 2 +- migration/block-dirty-bitmap.c | 673 +++++++++++++++++++++++++++++++++++++++++ migration/migration.c | 9 + migration/qemu-file.c | 18 ++ monitor.c | 7 + qapi-schema.json | 5 +- qapi/block-core.json | 79 ++++- qmp-commands.hx | 5 + tests/qemu-iotests/117 | 88 ++++++ tests/qemu-iotests/117.out | 5 + tests/qemu-iotests/118 | 87 ++++++ tests/qemu-iotests/118.out | 5 + tests/qemu-iotests/group | 2 + tests/qemu-iotests/iotests.py | 12 +- util/hbitmap.c | 104 +++++++ vl.c | 1 + 26 files changed, 1395 insertions(+), 9 deletions(-) create mode 100644 migration/block-dirty-bitmap.c create mode 100755 tests/qemu-iotests/117 create mode 100644 tests/qemu-iotests/117.out create mode 100755 tests/qemu-iotests/118 create mode 100644 tests/qemu-iotests/118.out -- 1.9.1