[Qemu-devel] [PATCH v13 00/17] block: incremental backup series

[John Snow <jsnow@redhat.com>]

This series requires: [PATCH v3] blkdebug: fix "once" rule

Welcome to the "incremental backup" newsletter, where we discuss
exciting developments in non-redundant backup technology.
This issue is called the "Max Reitz Fixup" issue.

This patchset enables the in-memory part of the incremental backup
feature. There are two series on the mailing list now by Vladimir
Sementsov-Ogievskiy that enable the migration and persistence of
dirty bitmaps.

This series and most patches were originally by Fam Zheng.

Highlights:
 - Renamed "node-ref" parameter to "node" in all QMP patches.
   (A rose by any other name? ...)
 - Fuller, more robust iotests.
 - Fixed progress percentages for incremental backup jobs.
 - This patch relies upon a blkdebug fix, included as patch #16,
   but also posted separately on-list.

V13:
 - 02: New patch, trying to iron out type issues.
 - 03: Patch no longer includes type fixes.
 - 04: Patch no longer includes tpye fixes.
 - 07: Error message fixes.
 - 08: Removed superfluous empty lines
 -     Progress updates occur before cancellation check.
 - 09: Fixed a critical bug where I added an extra '-' to docs.
 -     Corrected clear semantics (checks for disabled status)
 -     More full stops and newlines removed from error_setg.
 - 10: Error message cleanup.
 -     disabled status check added to clear transaction
 -     Added version documentation for new transactions
 -     Refiddled the add transaction to not delete erroneously
       if we abort after an unsuccessful prepare
 - 13: Joined lines where possible.
 - 15: Removed unnecessary imports
 -     "%s" changed to "%i"
 -     'is 0' replaced by '== 0'
 -     removed .add_args(["-S"]) and hmp("c") and added a docstring.
 -     Check for '%' in dirnames.
 - 16: removed hmp("c"), added a doctsring.
 - xx  Dropped blkdebug patch, was #16.
 - 17: Removed hmp("c"), added a docstring.
 -     Moved all blkdebug config into QMP, because Max asked very nicely.

V12:
 - Changed authorship from Fam Zheng to John Snow on most patches
 - 02: Fix the error_setg leak in bdrv_dirty_bitmap_lookup
 -     Fix error phrasing in bdrv_dirty_bitmap_lookup
 -     Renamed "node-ref" to "node" for QMP commands.
 - 03: Granularity helper no longer requires a BDS argument.
 - 04: Return early if the second bitmap is empty.
 - 05: Renamed the 'enabled' field to 'disabled to emphasize what the
       default operating state is.
 -     We now guard against bit sets or resets with the bitmap is
       disabled, making it a more pure "read only" mode.
 -     Some documentation phrasing changes.
 - 06: Removed explicit "frozen" state in favor of an implicit one.
       A successor present implies a frozen state.
 -     Updated all functions that target a single bitmap to use
       assertions that the bitmap they are trying to modify is not
       frozen/disabled.
 -     Functions that target multiple bitmaps use only a conditional,
       and will silently skip disabled bitmaps.
 -     thaw() function removed. It is implicitly handled in reclaim
       and abdicate.
 -     Added check for return code of hbitmap_merge.
 -     Functions now check against enable OR disable when in frozen
       state, for consistency and simplicity.
 -     Add "frozen" state documentation to remove/enable/disable
       QMP commands.
 - 07: Some documentation for bdrv_set_dirty_iter.
 -     Move function calls outside of assert()
 -     Cleanup the unused successor if we do not start the backup
 -     Version documentation added for dirty-bitmap to block-core.json
 -     Job progress is now reported for incremental backup jobs.
 - 08: bdrv_dirty_bitmap_clear is now in its own patch, here.
 -     bdrv_dirty_bitmap_clear no longer takes a BDS argument.
 - 09: Added a transaction for bdrv_dirty_bitmap_clear.
 - 10: Change 'enabled' field to 'disabled' field, to match
       above decision in patch 05.
 - 12: Removed extraneous BDS arguments from most bitmap functions.
 - 13-15: New set of iotests.
 - 16: blkdebug fix, already posted upstream.
 - 17: Final iotest, testing failure case.

Fam Zheng (1):
  qapi: Add optional field "name" to block dirty bitmap

John Snow (16):
  qmp: Ensure consistent granularity type
  qmp: Add block-dirty-bitmap-add and block-dirty-bitmap-remove
  block: Introduce bdrv_dirty_bitmap_granularity()
  hbitmap: add hbitmap_merge
  qmp: Add block-dirty-bitmap-enable and block-dirty-bitmap-disable
  block: Add bitmap successors
  qmp: Add support of "dirty-bitmap" sync mode for drive-backup
  qmp: add block-dirty-bitmap-clear
  qapi: Add transaction support to block-dirty-bitmap operations
  qmp: Add dirty bitmap status fields in query-block
  block: add BdrvDirtyBitmap documentation
  block: Ensure consistent bitmap function prototypes
  iotests: add invalid input incremental backup tests
  iotests: add simple incremental backup case
  iotests: add transactional incremental backup test
  iotests: add incremental backup failure recovery test

 block.c                       | 226 ++++++++++++++++++++++--
 block/backup.c                | 149 +++++++++++++---
 block/mirror.c                |  46 ++---
 blockdev.c                    | 387 +++++++++++++++++++++++++++++++++++++++++-
 hmp.c                         |   3 +-
 include/block/block.h         |  33 +++-
 include/block/block_int.h     |   4 +-
 include/qemu/hbitmap.h        |  11 ++
 migration/block.c             |   9 +-
 qapi-schema.json              |  10 +-
 qapi/block-core.json          | 122 ++++++++++++-
 qmp-commands.hx               |  92 +++++++++-
 tests/qemu-iotests/112        | 315 ++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/112.out    |   5 +
 tests/qemu-iotests/group      |   1 +
 tests/qemu-iotests/iotests.py |   6 +-
 util/hbitmap.c                |  32 ++++
 17 files changed, 1360 insertions(+), 91 deletions(-)
 create mode 100644 tests/qemu-iotests/112
 create mode 100644 tests/qemu-iotests/112.out

-- 
1.9.3