[Qemu-devel] [PATCH v5 00/22] qcow2: persistent dirty bitmaps

[Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>]

This series add persistent dirty bitmaps feature to qcow2.
Specification is in docs/spec/qcow2.txt 

v5:
https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=refs%2Ftags%2Fqcow2-bitmap-v5
Rebased on master. RFC removed. All necessary preparations are included.

changes:

patches 01-05 was not included in v4, so:

01: by Fam. only context changed, so I didn't add my s.o.b.
02: was "block: fix bdrv_dirty_bitmap_granularity()". Improve commit message
    and add lost Eric's r.b.
03: was on list. Not mentioned previously change, made after John's r.b.:
    split qemu-qtest by vm number too. Hope you don't mind
04: rewrite to not break 055
05: add lost Eric's r.b.
    changes after r.b. (hope, you don't mind):
    # @md5: md5 checksum of the last bitmap level (since 2.4)
    to
    # @md5: md5 checksum (as a hexadecimal string) of the last bitmap level
    #       (since 2.6)
  

06: new static function hb_restore_levels is actually
    hbitmap_deserialize_finish from list. I've included it here as it
    is the only dependence on serialization interface which is now on
    list in Fam's series. If Fam's series will be merged first I'll rebase
    and switch back to hbitmap_deserialize_finish(). Otherwise, Fam can
    rewrite hbitmap_deserialize_finish() to just call hb_restore_levels().
08: rebase: add #include "qemu/osdep.h"
13: rebase: move bdrv_finalize_persistent_dirty_bitmaps so that it called before
    bdrv_release_named_dirty_bitmaps =)
16: change error handling in dirty_bitmap_func: switch from errp to direct error
    reporting. This is more common approach in vl.c and also it allows to bypass
    one bug (see *)
19: test_launch: in qemu error output: s/qemu-system-\w*/qemu-system-*/ for test
    portability
20: accordingly to previous change: s/x86_64/*/ in 160.out

*: intersting bug, not related to these series:

./configure --target-list=x86_64-softmmu
make
./x86_64-softmmu/qemu-system-x86_64 -object asd
>> qemu-system-x86_64: -object asd: Parameter 'id' is missing

but:

./configure --target-list=x86_64-softmmu --enable-debug
make 
./x86_64-softmmu/qemu-system-x86_64 -object asd
>> qemu-system-x86_64: Parameter 'id' is missing (null): Parameter 'id' is missing

v4: 
https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=refs%2Ftags%2Fqcow2-bitmap-v4
Previous version was posted more than five months ago, so I will not
carefully list all changes.

What should be noted:
 - some changes are made to sutisfy last version of specification
   - removed staff, related to possibility of saving bitmaps for one
     disk in the other qcow2.
 - to make bitmap store/load zero-copy, I've moved load/store code to
   HBitmap - this is new patch 01.
   so, bdrv_dirty_bitmap_serialize_part and friends are not needed,
   only hbitmap_deserialize_finish, to repair bitmap consistency after
   loading its last level.
 - two patches added about AUTO and EXTRA_DATA_COMPATIBLE flags
 - some fixes made after John's comments on v3

Fam Zheng (1):
  block: Add two dirty bitmap getters

Vladimir Sementsov-Ogievskiy (21):
  block: fix bdrv_dirty_bitmap_granularity signature
  iotests: maintain several vms in test
  iotests: add default node-name
  qapi: add md5 checksum of last dirty bitmap level to query-block
  hbitmap: load/store
  qcow2: Bitmaps extension: structs and consts
  qcow2-dirty-bitmap: read dirty bitmap directory
  qcow2-dirty-bitmap: add qcow2_bitmap_load()
  qcow2-dirty-bitmap: add qcow2_bitmap_store()
  qcow2: add dirty bitmaps extension
  qcow2-dirty-bitmap: add qcow2_bitmap_load_check()
  block: store persistent dirty bitmaps
  block: add bdrv_load_dirty_bitmap()
  qcow2-dirty-bitmap: add autoclear bit
  qemu: command line option for dirty bitmaps
  qcow2-dirty-bitmap: add IN_USE flag
  qcow2-dirty-bitmaps: disallow stroing bitmap to other bs
  iotests: add VM.test_launcn()
  iotests: test internal persistent dirty bitmap
  qcow2-dirty-bitmap: add AUTO flag
  qcow2-dirty-bitmap: add EXTRA_DATA_COMPATIBLE flag

 block.c                       |   2 +
 block/Makefile.objs           |   2 +-
 block/dirty-bitmap.c          | 114 +++++-
 block/qcow2-dirty-bitmap.c    | 841 ++++++++++++++++++++++++++++++++++++++++++
 block/qcow2.c                 | 105 +++++-
 block/qcow2.h                 |  59 +++
 blockdev.c                    |  36 ++
 include/block/block_int.h     |   9 +
 include/block/dirty-bitmap.h  |  25 +-
 include/qemu/hbitmap.h        |  20 +
 include/sysemu/blockdev.h     |   1 +
 include/sysemu/sysemu.h       |   1 +
 qapi/block-core.json          |   5 +-
 qemu-options.hx               |  35 ++
 tests/qemu-iotests/160        | 112 ++++++
 tests/qemu-iotests/160.out    |  21 ++
 tests/qemu-iotests/group      |   1 +
 tests/qemu-iotests/iotests.py |  35 +-
 util/hbitmap.c                | 217 +++++++++++
 vl.c                          |  79 ++++
 20 files changed, 1711 insertions(+), 9 deletions(-)
 create mode 100644 block/qcow2-dirty-bitmap.c
 create mode 100755 tests/qemu-iotests/160
 create mode 100644 tests/qemu-iotests/160.out

-- 
1.8.3.1