Re: [Qemu-devel] [PATCH v6 00/12] Dirty bitmaps migration

[Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>]

Sorry for 'vsementsov@parallels.com' in patches. I've just fixed this 
locally and I'm ready to roll v7 with all my emails fixed. I'm waiting 
for other comments.

On 07.08.2015 12:32, Vladimir Sementsov-Ogievskiy wrote:
> From: Vladimir Sementsov-Ogievskiy <vsementsov@parallels.com>
>
> These patches provide dirty bitmap migration feature. Only named dirty
> bitmaps are to be migrated. Migration may be enabled using migration
> capabilities.
>
> v6:
>      - rebase on master
>         - function qemu_get_string() deleted, used existed function
>           qemu_get_counted_string()
>         - function qemu_put_string() renamed appropriately to
>           qemu_put_counted_string()
>           (and now it is new separate patch)
>      =====================================================
>      0004: Reviewed-by: John Snow <jsnow@redhat.com>
>      0006: (since 2.3) -> (since 2.4)
>      0007: new patch, mentioned above
>      0008: printf format fixed, DEBUG definition set back to 0, in
>            dirty_bitmap_live_iterate_is_active conditions swapped.
>            qemu_{get,put}_string removed, used ..counted.. verisions
>      0011: (since 2.3) -> (since 2.4)
>      0012: assertIsNotNone -> assertNotEqual for compitibility with
>            old Python
>            Reviewed-by: John Snow <jsnow@redhat.com>
>
>
> v5:
>      - rebase on master
>      - drop [PATCH RFC v4 10/13] iotests: add event_wait to VM class
>      - remove rfc, as incremental backup series by John Snow are in
>        upstream
>      
>
> v4 significant changes:
>   0001: tiny bugfix: out[i] -> out[i-start], same for 'in'
>   0007: set chunk size to 1kb, disable live iteration for
>         migrating data < 1mb size.
>
>   tests: only one with md5 sum is here. used function event_wait
>          by John Snow. (I hope, you don't mind me just adding this
>          function with your 'Signed-off-by')
>
>   rfc: This patch set is based on v13 of
>        "block: incremental backup series" by John Snow, which are
>        not pushed yet.
>
> v3:
>   based on v13 of "block: incremental backup series" by John Snow.
>
>   changes from v2:
>   removed patch for adding dirty parameter (migration capablities used
>   instead).
>   
>   0001: printf's dropped, qapi used
>   0002: part0 -> zeroes
>   0003: part0 -> zeroes
>   0005: dirty_dirty -> meta
>         add comments about meta bitmap
>         
>   0006: the format is changed, nodes used instead of devices.
>
>   other patches are new.
>
>   rfc: there are two tests. They are the same but using different
>   interfaces: md5 checksum of the bitmap last layer in query-block or
>   separate query-block-dirty-bitmap with dirty bitmap regions.
>   The second form is more appropriate for debugging, the first is more
>   appropriate for simple regression control. Which should go to
>   upstream?
>
> v2:
>   1. bug-fixes, that are already in upstream, and renaming of function
>   bdrv_reset_dirty_bitmap (which is already in Snow's series) are
>   dropped
>   2. bitmap store/restore: the concept renamed to serialization, added
>   function hbitmap_deserialize_part0, to not transfer zero blocks
>   3. migration dirty parameter: added description comment
>   4. Other patches are new.
>
> v2.rfc:
> Actually, in this version of the series I'm trying not use
> migration/block.c at all. Instead a separate migration unit is added
> in the new file migration/dirty-bitmap.c. Now bitmaps are migrated
> like blocks in block migration, they have their "dirty-dirty" bitmaps,
> for tracking set/unset changes during migration.
>
> The advantages are:
>    - no complications of migration/block.c
>    - separate dirty-dirty bitmaps provide handling of "unset's"
>    - more effective meta-data/data ratio - no tiny bitmap-blocks.
>
>
>
> v1:
> These patches provide dirty bitmap migration feature. Only named dirty
> bitmaps are to be migrated. Migration is made as a part of block
> migration in block-migration.c.
>
> Dirty bitmap migration may be enabled by "dirty" parameter for qmp migrate
> command. If "blk" and "inc" parameters are false when "dirty" is true
> block migration is actually skipped: no allocatoions, no bdrv_read's,
> no bdrv_write's, only bitmaps are migrated.
>
> The patch set includes two my previous bug fixes, which are necessary
> for it. The patch set is based on Incremental backup series by John
> Snow.
> Vladimir Sementsov-Ogievskiy (12):
>    hbitmap: serialization
>    block: BdrvDirtyBitmap serialization interface
>    block: tiny refactoring: minimize hbitmap_(set/reset) usage
>    block: add meta bitmaps
>    block: add bdrv_next_dirty_bitmap()
>    qapi: add dirty-bitmaps migration capability
>    migration: add migration/block-dirty-bitmap.c
>    iotests: maintain several vms in test
>    iotests: add add_incoming_migration to VM class
>    qapi: add md5 checksum of last dirty bitmap level to query-block
>    iotests: add dirty bitmap migration test
>    migration/qemu-file: make functions qemu_(get/put)_string public
>
>   block.c                        |  98 +++++-
>   include/block/block.h          |  22 ++
>   include/migration/block.h      |   1 +
>   include/migration/migration.h  |   1 +
>   include/migration/qemu-file.h  |  17 +
>   include/qemu/hbitmap.h         |  67 ++++
>   migration/Makefile.objs        |   2 +-
>   migration/block-dirty-bitmap.c | 693 +++++++++++++++++++++++++++++++++++++++++
>   migration/migration.c          |   9 +
>   migration/qemu-file.c          |  18 ++
>   qapi-schema.json               |   4 +-
>   qapi/block-core.json           |   4 +-
>   tests/qemu-iotests/117         |  84 +++++
>   tests/qemu-iotests/117.out     |   5 +
>   tests/qemu-iotests/group       |   1 +
>   tests/qemu-iotests/iotests.py  |  14 +-
>   util/hbitmap.c                 | 106 +++++++
>   vl.c                           |   1 +
>   18 files changed, 1137 insertions(+), 10 deletions(-)
>   create mode 100644 migration/block-dirty-bitmap.c
>   create mode 100755 tests/qemu-iotests/117
>   create mode 100644 tests/qemu-iotests/117.out
>


-- 
Best regards,
Vladimir
* now, @virtuozzo.com instead of @parallels.com. Sorry for this inconvenience.