* [PATCH v2 0/4] mirror: allow specifying working bitmap
@ 2024-03-07 13:47 Fiona Ebner
2024-03-07 13:47 ` [PATCH v2 1/4] qapi/block-core: avoid the re-use of MirrorSyncMode for backup Fiona Ebner
` (3 more replies)
0 siblings, 4 replies; 11+ messages in thread
From: Fiona Ebner @ 2024-03-07 13:47 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, vsementsov, jsnow,
f.gruenbichler, t.lamprecht, mahaocong, xiechanglong.d,
wencongyang2
Changes from RFC/v1 (discussion here [0]):
* Add patch to split BackupSyncMode and MirrorSyncMode.
* Drop bitmap-mode parameter and use passed-in bitmap as the working
bitmap instead. Users can get the desired behaviors by
using the block-dirty-bitmap-clear and block-dirty-bitmap-merge
calls (see commit message in patch 2/4 for how exactly).
* Add patch to check whether target image's cluster size is at most
mirror job's granularity. Optional, it's an extra safety check
that's useful when the target is a "diff" image that does not have
previously synced data.
Use cases:
* Possibility to resume a failed mirror later.
* Possibility to only mirror deltas to a previously mirrored volume.
* Possibility to (efficiently) mirror an drive that was previously
mirrored via some external mechanism (e.g. ZFS replication).
We are using the last one in production without any issues since about
4 years now. In particular, like mentioned in [1]:
> - create bitmap(s)
> - (incrementally) replicate storage volume(s) out of band (using ZFS)
> - incrementally drive mirror as part of a live migration of VM
> - drop bitmap(s)
Now, the IO test added in patch 4/4 actually contains yet another use
case, namely doing incremental mirrors to stand-alone qcow2 "diff"
images, that only contain the delta and can be rebased later. I had to
adapt the IO test, because its output expected the mirror bitmap to
still be dirty, but nowadays the mirror is apparently already done
when the bitmaps are queried. So I thought, I'll just use
'write-blocking' mode to avoid any potential timing issues.
But this exposed an issue with the diff image approach. If a write is
not aligned to the granularity of the mirror target, then rebasing the
diff image onto a backing image will not yield the desired result,
because the full cluster is considered to be allocated and will "hide"
some part of the base/backing image. The failure can be seen by either
using 'write-blocking' mode in the IO test or setting the (bitmap)
granularity to 32 KiB rather than the current 64 KiB.
For the latter case, patch 4/4 adds a check. For the former, the
limitation is documented (I'd expect this to be a niche use case in
practice).
[0]: https://lore.kernel.org/qemu-devel/b91dba34-7969-4d51-ba40-96a91038cc54@yandex-team.ru/T/#m4ae27dc8ca1fb053e0a32cc4ffa2cfab6646805c
[1]: https://lore.kernel.org/qemu-devel/1599127031.9uxdp5h9o2.astroid@nora.none/
Fabian Grünbichler (1):
iotests: add test for bitmap mirror
Fiona Ebner (2):
qapi/block-core: avoid the re-use of MirrorSyncMode for backup
blockdev: mirror: check for target's cluster size when using bitmap
John Snow (1):
mirror: allow specifying working bitmap
block/backup.c | 18 +-
block/mirror.c | 102 +-
block/monitor/block-hmp-cmds.c | 2 +-
block/replication.c | 2 +-
blockdev.c | 84 +-
include/block/block_int-global-state.h | 7 +-
qapi/block-core.json | 64 +-
tests/qemu-iotests/tests/bitmap-sync-mirror | 571 ++++
.../qemu-iotests/tests/bitmap-sync-mirror.out | 2946 +++++++++++++++++
tests/unit/test-block-iothread.c | 2 +-
10 files changed, 3729 insertions(+), 69 deletions(-)
create mode 100755 tests/qemu-iotests/tests/bitmap-sync-mirror
create mode 100644 tests/qemu-iotests/tests/bitmap-sync-mirror.out
--
2.39.2
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v2 1/4] qapi/block-core: avoid the re-use of MirrorSyncMode for backup
2024-03-07 13:47 [PATCH v2 0/4] mirror: allow specifying working bitmap Fiona Ebner
@ 2024-03-07 13:47 ` Fiona Ebner
2024-03-08 7:34 ` Markus Armbruster
2024-04-01 12:51 ` Vladimir Sementsov-Ogievskiy
2024-03-07 13:47 ` [PATCH v2 2/4] mirror: allow specifying working bitmap Fiona Ebner
` (2 subsequent siblings)
3 siblings, 2 replies; 11+ messages in thread
From: Fiona Ebner @ 2024-03-07 13:47 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, vsementsov, jsnow,
f.gruenbichler, t.lamprecht, mahaocong, xiechanglong.d,
wencongyang2
Backup supports all modes listed in MirrorSyncMode, while mirror does
not. Introduce BackupSyncMode by copying the current MirrorSyncMode
and drop the variants mirror does not support from MirrorSyncMode as
well as the corresponding manual check in mirror_start().
Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
I felt like keeping the "Since: X.Y" as before makes the most sense as
to not lose history. Or is it necessary to change this for
BackupSyncMode (and its members) since it got a new name?
block/backup.c | 18 ++++++++---------
block/mirror.c | 7 -------
block/monitor/block-hmp-cmds.c | 2 +-
block/replication.c | 2 +-
blockdev.c | 26 ++++++++++++-------------
include/block/block_int-global-state.h | 2 +-
qapi/block-core.json | 27 +++++++++++++++++++++-----
7 files changed, 47 insertions(+), 37 deletions(-)
diff --git a/block/backup.c b/block/backup.c
index ec29d6b810..1cc4e055c6 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -37,7 +37,7 @@ typedef struct BackupBlockJob {
BdrvDirtyBitmap *sync_bitmap;
- MirrorSyncMode sync_mode;
+ BackupSyncMode sync_mode;
BitmapSyncMode bitmap_mode;
BlockdevOnError on_source_error;
BlockdevOnError on_target_error;
@@ -111,7 +111,7 @@ void backup_do_checkpoint(BlockJob *job, Error **errp)
assert(block_job_driver(job) == &backup_job_driver);
- if (backup_job->sync_mode != MIRROR_SYNC_MODE_NONE) {
+ if (backup_job->sync_mode != BACKUP_SYNC_MODE_NONE) {
error_setg(errp, "The backup job only supports block checkpoint in"
" sync=none mode");
return;
@@ -231,11 +231,11 @@ static void backup_init_bcs_bitmap(BackupBlockJob *job)
uint64_t estimate;
BdrvDirtyBitmap *bcs_bitmap = block_copy_dirty_bitmap(job->bcs);
- if (job->sync_mode == MIRROR_SYNC_MODE_BITMAP) {
+ if (job->sync_mode == BACKUP_SYNC_MODE_BITMAP) {
bdrv_clear_dirty_bitmap(bcs_bitmap, NULL);
bdrv_dirty_bitmap_merge_internal(bcs_bitmap, job->sync_bitmap, NULL,
true);
- } else if (job->sync_mode == MIRROR_SYNC_MODE_TOP) {
+ } else if (job->sync_mode == BACKUP_SYNC_MODE_TOP) {
/*
* We can't hog the coroutine to initialize this thoroughly.
* Set a flag and resume work when we are able to yield safely.
@@ -254,7 +254,7 @@ static int coroutine_fn backup_run(Job *job, Error **errp)
backup_init_bcs_bitmap(s);
- if (s->sync_mode == MIRROR_SYNC_MODE_TOP) {
+ if (s->sync_mode == BACKUP_SYNC_MODE_TOP) {
int64_t offset = 0;
int64_t count;
@@ -282,7 +282,7 @@ static int coroutine_fn backup_run(Job *job, Error **errp)
block_copy_set_skip_unallocated(s->bcs, false);
}
- if (s->sync_mode == MIRROR_SYNC_MODE_NONE) {
+ if (s->sync_mode == BACKUP_SYNC_MODE_NONE) {
/*
* All bits are set in bcs bitmap to allow any cluster to be copied.
* This does not actually require them to be copied.
@@ -354,7 +354,7 @@ static const BlockJobDriver backup_job_driver = {
BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
BlockDriverState *target, int64_t speed,
- MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap,
+ BackupSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap,
BitmapSyncMode bitmap_mode,
bool compress,
const char *filter_node_name,
@@ -376,8 +376,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
GLOBAL_STATE_CODE();
/* QMP interface protects us from these cases */
- assert(sync_mode != MIRROR_SYNC_MODE_INCREMENTAL);
- assert(sync_bitmap || sync_mode != MIRROR_SYNC_MODE_BITMAP);
+ assert(sync_mode != BACKUP_SYNC_MODE_INCREMENTAL);
+ assert(sync_bitmap || sync_mode != BACKUP_SYNC_MODE_BITMAP);
if (bs == target) {
error_setg(errp, "Source and target cannot be the same");
diff --git a/block/mirror.c b/block/mirror.c
index 5145eb53e1..1609354db3 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -2011,13 +2011,6 @@ void mirror_start(const char *job_id, BlockDriverState *bs,
GLOBAL_STATE_CODE();
- if ((mode == MIRROR_SYNC_MODE_INCREMENTAL) ||
- (mode == MIRROR_SYNC_MODE_BITMAP)) {
- error_setg(errp, "Sync mode '%s' not supported",
- MirrorSyncMode_str(mode));
- return;
- }
-
bdrv_graph_rdlock_main_loop();
is_none_mode = mode == MIRROR_SYNC_MODE_NONE;
base = mode == MIRROR_SYNC_MODE_TOP ? bdrv_backing_chain_next(bs) : NULL;
diff --git a/block/monitor/block-hmp-cmds.c b/block/monitor/block-hmp-cmds.c
index d954bec6f1..9633d000c0 100644
--- a/block/monitor/block-hmp-cmds.c
+++ b/block/monitor/block-hmp-cmds.c
@@ -266,7 +266,7 @@ void hmp_drive_backup(Monitor *mon, const QDict *qdict)
.device = (char *)device,
.target = (char *)filename,
.format = (char *)format,
- .sync = full ? MIRROR_SYNC_MODE_FULL : MIRROR_SYNC_MODE_TOP,
+ .sync = full ? BACKUP_SYNC_MODE_FULL : BACKUP_SYNC_MODE_TOP,
.has_mode = true,
.mode = reuse ? NEW_IMAGE_MODE_EXISTING : NEW_IMAGE_MODE_ABSOLUTE_PATHS,
.has_compress = !!compress,
diff --git a/block/replication.c b/block/replication.c
index ca6bd0a720..1355e686f4 100644
--- a/block/replication.c
+++ b/block/replication.c
@@ -582,7 +582,7 @@ static void replication_start(ReplicationState *rs, ReplicationMode mode,
s->backup_job = backup_job_create(
NULL, s->secondary_disk->bs, s->hidden_disk->bs,
- 0, MIRROR_SYNC_MODE_NONE, NULL, 0, false, NULL,
+ 0, BACKUP_SYNC_MODE_NONE, NULL, 0, false, NULL,
&perf,
BLOCKDEV_ON_ERROR_REPORT,
BLOCKDEV_ON_ERROR_REPORT, JOB_INTERNAL,
diff --git a/blockdev.c b/blockdev.c
index f8bb0932f8..8dadbb353b 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1642,7 +1642,7 @@ static void drive_backup_action(DriveBackup *backup,
* See if we have a backing HD we can use to create our new image
* on top of.
*/
- if (backup->sync == MIRROR_SYNC_MODE_TOP) {
+ if (backup->sync == BACKUP_SYNC_MODE_TOP) {
/*
* Backup will not replace the source by the target, so none
* of the filters skipped here will be removed (in contrast to
@@ -1651,10 +1651,10 @@ static void drive_backup_action(DriveBackup *backup,
*/
source = bdrv_cow_bs(bdrv_skip_filters(bs));
if (!source) {
- backup->sync = MIRROR_SYNC_MODE_FULL;
+ backup->sync = BACKUP_SYNC_MODE_FULL;
}
}
- if (backup->sync == MIRROR_SYNC_MODE_NONE) {
+ if (backup->sync == BACKUP_SYNC_MODE_NONE) {
source = bs;
flags |= BDRV_O_NO_BACKING;
set_backing_hd = true;
@@ -2655,27 +2655,27 @@ static BlockJob *do_backup_common(BackupCommon *backup,
}
}
- if ((backup->sync == MIRROR_SYNC_MODE_BITMAP) ||
- (backup->sync == MIRROR_SYNC_MODE_INCREMENTAL)) {
+ if ((backup->sync == BACKUP_SYNC_MODE_BITMAP) ||
+ (backup->sync == BACKUP_SYNC_MODE_INCREMENTAL)) {
/* done before desugaring 'incremental' to print the right message */
if (!backup->bitmap) {
error_setg(errp, "must provide a valid bitmap name for "
- "'%s' sync mode", MirrorSyncMode_str(backup->sync));
+ "'%s' sync mode", BackupSyncMode_str(backup->sync));
return NULL;
}
}
- if (backup->sync == MIRROR_SYNC_MODE_INCREMENTAL) {
+ if (backup->sync == BACKUP_SYNC_MODE_INCREMENTAL) {
if (backup->has_bitmap_mode &&
backup->bitmap_mode != BITMAP_SYNC_MODE_ON_SUCCESS) {
error_setg(errp, "Bitmap sync mode must be '%s' "
"when using sync mode '%s'",
BitmapSyncMode_str(BITMAP_SYNC_MODE_ON_SUCCESS),
- MirrorSyncMode_str(backup->sync));
+ BackupSyncMode_str(backup->sync));
return NULL;
}
backup->has_bitmap_mode = true;
- backup->sync = MIRROR_SYNC_MODE_BITMAP;
+ backup->sync = BACKUP_SYNC_MODE_BITMAP;
backup->bitmap_mode = BITMAP_SYNC_MODE_ON_SUCCESS;
}
@@ -2695,19 +2695,19 @@ static BlockJob *do_backup_common(BackupCommon *backup,
}
/* This does not produce a useful bitmap artifact: */
- if (backup->sync == MIRROR_SYNC_MODE_NONE) {
+ if (backup->sync == BACKUP_SYNC_MODE_NONE) {
error_setg(errp, "sync mode '%s' does not produce meaningful bitmap"
- " outputs", MirrorSyncMode_str(backup->sync));
+ " outputs", BackupSyncMode_str(backup->sync));
return NULL;
}
/* If the bitmap isn't used for input or output, this is useless: */
if (backup->bitmap_mode == BITMAP_SYNC_MODE_NEVER &&
- backup->sync != MIRROR_SYNC_MODE_BITMAP) {
+ backup->sync != BACKUP_SYNC_MODE_BITMAP) {
error_setg(errp, "Bitmap sync mode '%s' has no meaningful effect"
" when combined with sync mode '%s'",
BitmapSyncMode_str(backup->bitmap_mode),
- MirrorSyncMode_str(backup->sync));
+ BackupSyncMode_str(backup->sync));
return NULL;
}
}
diff --git a/include/block/block_int-global-state.h b/include/block/block_int-global-state.h
index d2201e27f4..54f8c8cbcb 100644
--- a/include/block/block_int-global-state.h
+++ b/include/block/block_int-global-state.h
@@ -190,7 +190,7 @@ void mirror_start(const char *job_id, BlockDriverState *bs,
*/
BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
BlockDriverState *target, int64_t speed,
- MirrorSyncMode sync_mode,
+ BackupSyncMode sync_mode,
BdrvDirtyBitmap *sync_bitmap,
BitmapSyncMode bitmap_mode,
bool compress,
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 1874f880a8..59d75b0793 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1304,10 +1304,10 @@
'data': ['report', 'ignore', 'enospc', 'stop', 'auto'] }
##
-# @MirrorSyncMode:
+# @BackupSyncMode:
#
-# An enumeration of possible behaviors for the initial synchronization
-# phase of storage mirroring.
+# An enumeration of possible behaviors for image synchronization used
+# by backup jobs.
#
# @top: copies data in the topmost image to the destination
#
@@ -1323,7 +1323,7 @@
#
# Since: 1.3
##
-{ 'enum': 'MirrorSyncMode',
+{ 'enum': 'BackupSyncMode',
'data': ['top', 'full', 'none', 'incremental', 'bitmap'] }
##
@@ -1347,6 +1347,23 @@
{ 'enum': 'BitmapSyncMode',
'data': ['on-success', 'never', 'always'] }
+##
+# @MirrorSyncMode:
+#
+# An enumeration of possible behaviors for the initial synchronization
+# phase of storage mirroring.
+#
+# @top: copies data in the topmost image to the destination
+#
+# @full: copies data from all images to the destination
+#
+# @none: only copy data written from now on
+#
+# Since: 1.3
+##
+{ 'enum': 'MirrorSyncMode',
+ 'data': ['top', 'full', 'none'] }
+
##
# @MirrorCopyMode:
#
@@ -1624,7 +1641,7 @@
##
{ 'struct': 'BackupCommon',
'data': { '*job-id': 'str', 'device': 'str',
- 'sync': 'MirrorSyncMode', '*speed': 'int',
+ 'sync': 'BackupSyncMode', '*speed': 'int',
'*bitmap': 'str', '*bitmap-mode': 'BitmapSyncMode',
'*compress': 'bool',
'*on-source-error': 'BlockdevOnError',
--
2.39.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v2 2/4] mirror: allow specifying working bitmap
2024-03-07 13:47 [PATCH v2 0/4] mirror: allow specifying working bitmap Fiona Ebner
2024-03-07 13:47 ` [PATCH v2 1/4] qapi/block-core: avoid the re-use of MirrorSyncMode for backup Fiona Ebner
@ 2024-03-07 13:47 ` Fiona Ebner
2024-03-08 7:41 ` Markus Armbruster
2024-04-02 20:14 ` Vladimir Sementsov-Ogievskiy
2024-03-07 13:47 ` [PATCH v2 3/4] iotests: add test for bitmap mirror Fiona Ebner
2024-03-07 13:47 ` [PATCH v2 4/4] blockdev: mirror: check for target's cluster size when using bitmap Fiona Ebner
3 siblings, 2 replies; 11+ messages in thread
From: Fiona Ebner @ 2024-03-07 13:47 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, vsementsov, jsnow,
f.gruenbichler, t.lamprecht, mahaocong, xiechanglong.d,
wencongyang2
From: John Snow <jsnow@redhat.com>
for the mirror job. The bitmap's granularity is used as the job's
granularity.
The new @bitmap parameter is marked unstable in the QAPI and can
currently only be used for @sync=full mode.
Clusters initially dirty in the bitmap as well as new writes are
copied to the target.
Using block-dirty-bitmap-clear and block-dirty-bitmap-merge API,
callers can simulate the three kinds of @BitmapSyncMode (which is used
by backup):
1. always: default, just pass bitmap as working bitmap.
2. never: copy bitmap and pass copy to the mirror job.
3. on-success: copy bitmap and pass copy to the mirror job and if
successful, merge bitmap into original afterwards.
When the target image is a fresh "diff image", i.e. one that was not
used as the target of a previous mirror and the target image's cluster
size is larger than the bitmap's granularity, or when
@copy-mode=write-blocking is used, there is a pitfall, because the
cluster in the target image will be allocated, but not contain all the
data corresponding to the same region in the source image.
An idea to avoid the limitation would be to mark clusters which are
affected by unaligned writes and are not allocated in the target image
dirty, so they would be copied fully later. However, for migration,
the invariant that an actively synced mirror stays actively synced
(unless an error happens) is useful, because without that invariant,
migration might inactivate block devices when mirror still got work
to do and run into an assertion failure [0].
Another approach would be to read the missing data from the source
upon unaligned writes to be able to write the full target cluster
instead.
But certain targets like NBD do not allow querying the cluster size.
To avoid limiting/breaking the use case of syncing to an existing
target, which is arguably more common than the diff image use case,
document the limiation in QAPI.
This patch was originally based on one by Ma Haocong, but it has since
been modified pretty heavily, first by John and then again by Fiona.
[0]: https://lore.kernel.org/qemu-devel/1db7f571-cb7f-c293-04cc-cd856e060c3f@proxmox.com/
Suggested-by: Ma Haocong <mahaocong@didichuxing.com>
Signed-off-by: Ma Haocong <mahaocong@didichuxing.com>
Signed-off-by: John Snow <jsnow@redhat.com>
[FG: switch to bdrv_dirty_bitmap_merge_internal]
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
[FE: rebase for 9.0
get rid of bitmap mode parameter
use caller-provided bitmap as working bitmap
turn bitmap parameter experimental]
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
block/mirror.c | 95 ++++++++++++++++++++------
blockdev.c | 39 +++++++++--
include/block/block_int-global-state.h | 5 +-
qapi/block-core.json | 37 +++++++++-
tests/unit/test-block-iothread.c | 2 +-
5 files changed, 146 insertions(+), 32 deletions(-)
diff --git a/block/mirror.c b/block/mirror.c
index 1609354db3..5c9a00b574 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -51,7 +51,7 @@ typedef struct MirrorBlockJob {
BlockDriverState *to_replace;
/* Used to block operations on the drive-mirror-replace target */
Error *replace_blocker;
- bool is_none_mode;
+ MirrorSyncMode sync_mode;
BlockMirrorBackingMode backing_mode;
/* Whether the target image requires explicit zero-initialization */
bool zero_target;
@@ -73,6 +73,11 @@ typedef struct MirrorBlockJob {
size_t buf_size;
int64_t bdev_length;
unsigned long *cow_bitmap;
+ /*
+ * Whether the bitmap is created locally or provided by the caller (for
+ * incremental sync).
+ */
+ bool dirty_bitmap_is_local;
BdrvDirtyBitmap *dirty_bitmap;
BdrvDirtyBitmapIter *dbi;
uint8_t *buf;
@@ -687,7 +692,11 @@ static int mirror_exit_common(Job *job)
bdrv_unfreeze_backing_chain(mirror_top_bs, target_bs);
}
- bdrv_release_dirty_bitmap(s->dirty_bitmap);
+ if (s->dirty_bitmap_is_local) {
+ bdrv_release_dirty_bitmap(s->dirty_bitmap);
+ } else {
+ bdrv_enable_dirty_bitmap(s->dirty_bitmap);
+ }
/* Make sure that the source BDS doesn't go away during bdrv_replace_node,
* before we can call bdrv_drained_end */
@@ -718,7 +727,8 @@ static int mirror_exit_common(Job *job)
&error_abort);
if (!abort && s->backing_mode == MIRROR_SOURCE_BACKING_CHAIN) {
- BlockDriverState *backing = s->is_none_mode ? src : s->base;
+ BlockDriverState *backing;
+ backing = s->sync_mode == MIRROR_SYNC_MODE_NONE ? src : s->base;
BlockDriverState *unfiltered_target = bdrv_skip_filters(target_bs);
if (bdrv_cow_bs(unfiltered_target) != backing) {
@@ -815,6 +825,16 @@ static void mirror_abort(Job *job)
assert(ret == 0);
}
+/* Always called after commit/abort. */
+static void mirror_clean(Job *job)
+{
+ MirrorBlockJob *s = container_of(job, MirrorBlockJob, common.job);
+
+ if (!s->dirty_bitmap_is_local && s->dirty_bitmap) {
+ bdrv_dirty_bitmap_set_busy(s->dirty_bitmap, false);
+ }
+}
+
static void coroutine_fn mirror_throttle(MirrorBlockJob *s)
{
int64_t now = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
@@ -1011,7 +1031,8 @@ static int coroutine_fn mirror_run(Job *job, Error **errp)
mirror_free_init(s);
s->last_pause_ns = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
- if (!s->is_none_mode) {
+ if ((s->sync_mode == MIRROR_SYNC_MODE_TOP ||
+ s->sync_mode == MIRROR_SYNC_MODE_FULL) && s->dirty_bitmap_is_local) {
ret = mirror_dirty_init(s);
if (ret < 0 || job_is_cancelled(&s->common.job)) {
goto immediate_exit;
@@ -1024,6 +1045,14 @@ static int coroutine_fn mirror_run(Job *job, Error **errp)
*/
mirror_top_opaque->job = s;
+ /*
+ * External/caller-provided bitmap can only be disabled now that
+ * bdrv_mirror_top_do_write() can access it.
+ */
+ if (!s->dirty_bitmap_is_local) {
+ bdrv_disable_dirty_bitmap(s->dirty_bitmap);
+ }
+
assert(!s->dbi);
s->dbi = bdrv_dirty_iter_new(s->dirty_bitmap);
for (;;) {
@@ -1302,6 +1331,7 @@ static const BlockJobDriver mirror_job_driver = {
.run = mirror_run,
.prepare = mirror_prepare,
.abort = mirror_abort,
+ .clean = mirror_clean,
.pause = mirror_pause,
.complete = mirror_complete,
.cancel = mirror_cancel,
@@ -1320,6 +1350,7 @@ static const BlockJobDriver commit_active_job_driver = {
.run = mirror_run,
.prepare = mirror_prepare,
.abort = mirror_abort,
+ .clean = mirror_clean,
.pause = mirror_pause,
.complete = mirror_complete,
.cancel = commit_active_cancel,
@@ -1712,7 +1743,9 @@ static BlockJob *mirror_start_job(
BlockCompletionFunc *cb,
void *opaque,
const BlockJobDriver *driver,
- bool is_none_mode, BlockDriverState *base,
+ MirrorSyncMode sync_mode,
+ BdrvDirtyBitmap *bitmap,
+ BlockDriverState *base,
bool auto_complete, const char *filter_node_name,
bool is_mirror, MirrorCopyMode copy_mode,
Error **errp)
@@ -1726,10 +1759,15 @@ static BlockJob *mirror_start_job(
GLOBAL_STATE_CODE();
- if (granularity == 0) {
+ /* QMP interface ensures these conditions */
+ assert(!bitmap || sync_mode == MIRROR_SYNC_MODE_FULL);
+ assert(!(bitmap && granularity));
+
+ if (bitmap) {
+ granularity = bdrv_dirty_bitmap_granularity(bitmap);
+ } else if (granularity == 0) {
granularity = bdrv_get_default_bitmap_granularity(target);
}
-
assert(is_power_of_2(granularity));
if (buf_size < 0) {
@@ -1869,7 +1907,7 @@ static BlockJob *mirror_start_job(
s->replaces = g_strdup(replaces);
s->on_source_error = on_source_error;
s->on_target_error = on_target_error;
- s->is_none_mode = is_none_mode;
+ s->sync_mode = sync_mode;
s->backing_mode = backing_mode;
s->zero_target = zero_target;
qatomic_set(&s->copy_mode, copy_mode);
@@ -1883,17 +1921,27 @@ static BlockJob *mirror_start_job(
}
bdrv_graph_rdunlock_main_loop();
- s->dirty_bitmap = bdrv_create_dirty_bitmap(s->mirror_top_bs, granularity,
- NULL, errp);
- if (!s->dirty_bitmap) {
- goto fail;
+ if (bitmap) {
+ s->dirty_bitmap_is_local = false;
+ s->dirty_bitmap = bitmap;
+ bdrv_dirty_bitmap_set_busy(s->dirty_bitmap, true);
+ } else {
+ s->dirty_bitmap_is_local = true;
+ s->dirty_bitmap = bdrv_create_dirty_bitmap(s->mirror_top_bs,
+ granularity, NULL, errp);
+ if (!s->dirty_bitmap) {
+ goto fail;
+ }
}
/*
* The dirty bitmap is set by bdrv_mirror_top_do_write() when not in active
- * mode.
+ * mode. For external/caller-provided bitmap, need to wait until
+ * bdrv_mirror_top_do_write() can actually access it before disabling.
*/
- bdrv_disable_dirty_bitmap(s->dirty_bitmap);
+ if (s->dirty_bitmap_is_local) {
+ bdrv_disable_dirty_bitmap(s->dirty_bitmap);
+ }
bdrv_graph_wrlock();
ret = block_job_add_bdrv(&s->common, "source", bs, 0,
@@ -1975,7 +2023,11 @@ fail:
blk_unref(s->target);
bs_opaque->job = NULL;
if (s->dirty_bitmap) {
- bdrv_release_dirty_bitmap(s->dirty_bitmap);
+ if (s->dirty_bitmap_is_local) {
+ bdrv_release_dirty_bitmap(s->dirty_bitmap);
+ } else {
+ bdrv_dirty_bitmap_set_busy(s->dirty_bitmap, false);
+ }
}
job_early_fail(&s->common.job);
}
@@ -1999,27 +2051,26 @@ void mirror_start(const char *job_id, BlockDriverState *bs,
BlockDriverState *target, const char *replaces,
int creation_flags, int64_t speed,
uint32_t granularity, int64_t buf_size,
- MirrorSyncMode mode, BlockMirrorBackingMode backing_mode,
+ MirrorSyncMode mode, BdrvDirtyBitmap *bitmap,
+ BlockMirrorBackingMode backing_mode,
bool zero_target,
BlockdevOnError on_source_error,
BlockdevOnError on_target_error,
bool unmap, const char *filter_node_name,
MirrorCopyMode copy_mode, Error **errp)
{
- bool is_none_mode;
BlockDriverState *base;
GLOBAL_STATE_CODE();
bdrv_graph_rdlock_main_loop();
- is_none_mode = mode == MIRROR_SYNC_MODE_NONE;
base = mode == MIRROR_SYNC_MODE_TOP ? bdrv_backing_chain_next(bs) : NULL;
bdrv_graph_rdunlock_main_loop();
mirror_start_job(job_id, bs, creation_flags, target, replaces,
speed, granularity, buf_size, backing_mode, zero_target,
on_source_error, on_target_error, unmap, NULL, NULL,
- &mirror_job_driver, is_none_mode, base, false,
+ &mirror_job_driver, mode, bitmap, base, false,
filter_node_name, true, copy_mode, errp);
}
@@ -2047,9 +2098,9 @@ BlockJob *commit_active_start(const char *job_id, BlockDriverState *bs,
job_id, bs, creation_flags, base, NULL, speed, 0, 0,
MIRROR_LEAVE_BACKING_CHAIN, false,
on_error, on_error, true, cb, opaque,
- &commit_active_job_driver, false, base, auto_complete,
- filter_node_name, false, MIRROR_COPY_MODE_BACKGROUND,
- errp);
+ &commit_active_job_driver, MIRROR_SYNC_MODE_FULL, NULL,
+ base, auto_complete, filter_node_name, false,
+ MIRROR_COPY_MODE_BACKGROUND, errp);
if (!job) {
goto error_restore_flags;
}
diff --git a/blockdev.c b/blockdev.c
index 8dadbb353b..c76eb97a4c 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2776,6 +2776,7 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
BlockDriverState *target,
const char *replaces,
enum MirrorSyncMode sync,
+ const char *bitmap_name,
BlockMirrorBackingMode backing_mode,
bool zero_target,
bool has_speed, int64_t speed,
@@ -2794,6 +2795,7 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
{
BlockDriverState *unfiltered_bs;
int job_flags = JOB_DEFAULT;
+ BdrvDirtyBitmap *bitmap = NULL;
GLOBAL_STATE_CODE();
GRAPH_RDLOCK_GUARD_MAINLOOP();
@@ -2844,6 +2846,28 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
return;
}
+ if (bitmap_name) {
+ if (sync != MIRROR_SYNC_MODE_FULL) {
+ error_setg(errp, "Sync mode '%s' not supported with bitmap.",
+ MirrorSyncMode_str(sync));
+ return;
+ }
+ if (granularity) {
+ error_setg(errp, "Granularity and bitmap cannot both be set");
+ return;
+ }
+
+ bitmap = bdrv_find_dirty_bitmap(bs, bitmap_name);
+ if (!bitmap) {
+ error_setg(errp, "Dirty bitmap '%s' not found", bitmap_name);
+ return;
+ }
+
+ if (bdrv_dirty_bitmap_check(bitmap, BDRV_BITMAP_ALLOW_RO, errp)) {
+ return;
+ }
+ }
+
if (!bdrv_backing_chain_next(bs) && sync == MIRROR_SYNC_MODE_TOP) {
sync = MIRROR_SYNC_MODE_FULL;
}
@@ -2889,10 +2913,9 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
* and will allow to check whether the node still exist at mirror completion
*/
mirror_start(job_id, bs, target,
- replaces, job_flags,
- speed, granularity, buf_size, sync, backing_mode, zero_target,
- on_source_error, on_target_error, unmap, filter_node_name,
- copy_mode, errp);
+ replaces, job_flags, speed, granularity, buf_size, sync,
+ bitmap, backing_mode, zero_target, on_source_error,
+ on_target_error, unmap, filter_node_name, copy_mode, errp);
}
void qmp_drive_mirror(DriveMirror *arg, Error **errp)
@@ -3033,7 +3056,7 @@ void qmp_drive_mirror(DriveMirror *arg, Error **errp)
}
blockdev_mirror_common(arg->job_id, bs, target_bs,
- arg->replaces, arg->sync,
+ arg->replaces, arg->sync, arg->bitmap,
backing_mode, zero_target,
arg->has_speed, arg->speed,
arg->has_granularity, arg->granularity,
@@ -3053,6 +3076,7 @@ void qmp_blockdev_mirror(const char *job_id,
const char *device, const char *target,
const char *replaces,
MirrorSyncMode sync,
+ const char *bitmap,
bool has_speed, int64_t speed,
bool has_granularity, uint32_t granularity,
bool has_buf_size, int64_t buf_size,
@@ -3093,8 +3117,9 @@ void qmp_blockdev_mirror(const char *job_id,
}
blockdev_mirror_common(job_id, bs, target_bs,
- replaces, sync, backing_mode,
- zero_target, has_speed, speed,
+ replaces, sync, bitmap,
+ backing_mode, zero_target,
+ has_speed, speed,
has_granularity, granularity,
has_buf_size, buf_size,
has_on_source_error, on_source_error,
diff --git a/include/block/block_int-global-state.h b/include/block/block_int-global-state.h
index 54f8c8cbcb..8b93db017e 100644
--- a/include/block/block_int-global-state.h
+++ b/include/block/block_int-global-state.h
@@ -138,6 +138,8 @@ BlockJob *commit_active_start(const char *job_id, BlockDriverState *bs,
* @granularity: The chosen granularity for the dirty bitmap.
* @buf_size: The amount of data that can be in flight at one time.
* @mode: Whether to collapse all images in the chain to the target.
+ * @bitmap: Use this bitmap as a working bitmap, i.e. non-dirty clusters are
+ only mirrored if written to later.
* @backing_mode: How to establish the target's backing chain after completion.
* @zero_target: Whether the target should be explicitly zero-initialized
* @on_source_error: The action to take upon error reading from the source.
@@ -158,7 +160,8 @@ void mirror_start(const char *job_id, BlockDriverState *bs,
BlockDriverState *target, const char *replaces,
int creation_flags, int64_t speed,
uint32_t granularity, int64_t buf_size,
- MirrorSyncMode mode, BlockMirrorBackingMode backing_mode,
+ MirrorSyncMode mode, BdrvDirtyBitmap *bitmap,
+ BlockMirrorBackingMode backing_mode,
bool zero_target,
BlockdevOnError on_source_error,
BlockdevOnError on_target_error,
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 59d75b0793..4859fffd48 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2191,6 +2191,18 @@
# destination (all the disk, only the sectors allocated in the
# topmost image, or only new I/O).
#
+# @bitmap: The name of a bitmap to use as a working bitmap for
+# sync=full mode. This argument must be not be present for other
+# sync modes and not at the same time as @granularity. The
+# bitmap's granularity is used as the job's granularity. When
+# the target is a diff image, i.e. one that should only contain
+# the delta and was not synced to previously, the target's
+# cluster size must not be larger than the bitmap's granularity.
+# For a diff image target, using copy-mode=write-blocking should
+# not be used, because unaligned writes will lead to allocated
+# clusters with partial data in the target image! The bitmap
+# will be enabled after the job finishes. (Since 9.0)
+#
# @granularity: granularity of the dirty bitmap, default is 64K if the
# image format doesn't have clusters, 4K if the clusters are
# smaller than that, else the cluster size. Must be a power of 2
@@ -2228,12 +2240,18 @@
# disappear from the query list without user intervention.
# Defaults to true. (Since 3.1)
#
+# Features:
+#
+# @unstable: Member @bitmap is experimental.
+#
# Since: 1.3
##
{ 'struct': 'DriveMirror',
'data': { '*job-id': 'str', 'device': 'str', 'target': 'str',
'*format': 'str', '*node-name': 'str', '*replaces': 'str',
- 'sync': 'MirrorSyncMode', '*mode': 'NewImageMode',
+ 'sync': 'MirrorSyncMode',
+ '*bitmap': { 'type': 'str', 'features': [ 'unstable' ] },
+ '*mode': 'NewImageMode',
'*speed': 'int', '*granularity': 'uint32',
'*buf-size': 'int', '*on-source-error': 'BlockdevOnError',
'*on-target-error': 'BlockdevOnError',
@@ -2513,6 +2531,18 @@
# destination (all the disk, only the sectors allocated in the
# topmost image, or only new I/O).
#
+# @bitmap: The name of a bitmap to use as a working bitmap for
+# sync=full mode. This argument must be not be present for other
+# sync modes and not at the same time as @granularity. The
+# bitmap's granularity is used as the job's granularity. When
+# the target is a diff image, i.e. one that should only contain
+# the delta and was not synced to previously, the target's
+# cluster size must not be larger than the bitmap's granularity.
+# For a diff image target, using copy-mode=write-blocking should
+# not be used, because unaligned writes will lead to allocated
+# clusters with partial data in the target image! The bitmap
+# will be enabled after the job finishes. (Since 9.0)
+#
# @granularity: granularity of the dirty bitmap, default is 64K if the
# image format doesn't have clusters, 4K if the clusters are
# smaller than that, else the cluster size. Must be a power of 2
@@ -2548,6 +2578,10 @@
# disappear from the query list without user intervention.
# Defaults to true. (Since 3.1)
#
+# Features:
+#
+# @unstable: Member @bitmap is experimental.
+#
# Since: 2.6
#
# Example:
@@ -2562,6 +2596,7 @@
'data': { '*job-id': 'str', 'device': 'str', 'target': 'str',
'*replaces': 'str',
'sync': 'MirrorSyncMode',
+ '*bitmap': { 'type': 'str', 'features': [ 'unstable' ] },
'*speed': 'int', '*granularity': 'uint32',
'*buf-size': 'int', '*on-source-error': 'BlockdevOnError',
'*on-target-error': 'BlockdevOnError',
diff --git a/tests/unit/test-block-iothread.c b/tests/unit/test-block-iothread.c
index 3766d5de6b..b64158aa11 100644
--- a/tests/unit/test-block-iothread.c
+++ b/tests/unit/test-block-iothread.c
@@ -755,7 +755,7 @@ static void test_propagate_mirror(void)
/* Start a mirror job */
mirror_start("job0", src, target, NULL, JOB_DEFAULT, 0, 0, 0,
- MIRROR_SYNC_MODE_NONE, MIRROR_OPEN_BACKING_CHAIN, false,
+ MIRROR_SYNC_MODE_NONE, NULL, MIRROR_OPEN_BACKING_CHAIN, false,
BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT,
false, "filter_node", MIRROR_COPY_MODE_BACKGROUND,
&error_abort);
--
2.39.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v2 3/4] iotests: add test for bitmap mirror
2024-03-07 13:47 [PATCH v2 0/4] mirror: allow specifying working bitmap Fiona Ebner
2024-03-07 13:47 ` [PATCH v2 1/4] qapi/block-core: avoid the re-use of MirrorSyncMode for backup Fiona Ebner
2024-03-07 13:47 ` [PATCH v2 2/4] mirror: allow specifying working bitmap Fiona Ebner
@ 2024-03-07 13:47 ` Fiona Ebner
2024-03-07 13:47 ` [PATCH v2 4/4] blockdev: mirror: check for target's cluster size when using bitmap Fiona Ebner
3 siblings, 0 replies; 11+ messages in thread
From: Fiona Ebner @ 2024-03-07 13:47 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, vsementsov, jsnow,
f.gruenbichler, t.lamprecht, mahaocong, xiechanglong.d,
wencongyang2
From: Fabian Grünbichler <f.gruenbichler@proxmox.com>
heavily based on/practically forked off iotest 257 for bitmap backups,
but:
- no writes to filter node 'mirror-top' between completion and
finalization, as those seem to deadlock?
- extra set of reference/test mirrors to verify that writes in parallel
with active mirror work
Intentionally keeping copyright and ownership of original test case to
honor provenance.
The test was originally adapted by Fabian from 257, but has seen
rather big changes, because the interface for mirror with bitmap was
changed, i.e. no @bitmap-mode parameter anymore and bitmap is used as
the working bitmap.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
[FE: rebase for 9.0
adapt to changes to mirror bitmap interface
rename test from '384' to 'bitmap-sync-mirror']
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
tests/qemu-iotests/tests/bitmap-sync-mirror | 565 ++++
.../qemu-iotests/tests/bitmap-sync-mirror.out | 2939 +++++++++++++++++
2 files changed, 3504 insertions(+)
create mode 100755 tests/qemu-iotests/tests/bitmap-sync-mirror
create mode 100644 tests/qemu-iotests/tests/bitmap-sync-mirror.out
diff --git a/tests/qemu-iotests/tests/bitmap-sync-mirror b/tests/qemu-iotests/tests/bitmap-sync-mirror
new file mode 100755
index 0000000000..898f1f4ba4
--- /dev/null
+++ b/tests/qemu-iotests/tests/bitmap-sync-mirror
@@ -0,0 +1,565 @@
+#!/usr/bin/env python3
+# group: rw
+#
+# Test bitmap-sync mirrors (incremental, differential, and partials)
+#
+# Copyright (c) 2019 John Snow for Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+# owner=jsnow@redhat.com
+
+import math
+import os
+
+import iotests
+from iotests import log, qemu_img
+
+SIZE = 64 * 1024 * 1024
+GRANULARITY = 64 * 1024
+
+
+class Pattern:
+ def __init__(self, byte, offset, size=GRANULARITY):
+ self.byte = byte
+ self.offset = offset
+ self.size = size
+
+ def bits(self, granularity):
+ lower = self.offset // granularity
+ upper = (self.offset + self.size - 1) // granularity
+ return set(range(lower, upper + 1))
+
+
+class PatternGroup:
+ """Grouping of Pattern objects. Initialize with an iterable of Patterns."""
+ def __init__(self, patterns):
+ self.patterns = patterns
+
+ def bits(self, granularity):
+ """Calculate the unique bits dirtied by this pattern grouping"""
+ res = set()
+ for pattern in self.patterns:
+ res |= pattern.bits(granularity)
+ return res
+
+
+GROUPS = [
+ PatternGroup([
+ # Batch 0: 4 clusters
+ Pattern('0x49', 0x0000000),
+ Pattern('0x6c', 0x0100000), # 1M
+ Pattern('0x6f', 0x2000000), # 32M
+ Pattern('0x76', 0x3ff0000)]), # 64M - 64K
+ PatternGroup([
+ # Batch 1: 6 clusters (3 new)
+ Pattern('0x65', 0x0000000), # Full overwrite
+ Pattern('0x77', 0x00f8000), # Partial-left (1M-32K)
+ Pattern('0x72', 0x2008000), # Partial-right (32M+32K)
+ Pattern('0x69', 0x3fe0000)]), # Adjacent-left (64M - 128K)
+ PatternGroup([
+ # Batch 2: 7 clusters (3 new)
+ Pattern('0x74', 0x0010000), # Adjacent-right
+ Pattern('0x69', 0x00e8000), # Partial-left (1M-96K)
+ Pattern('0x6e', 0x2018000), # Partial-right (32M+96K)
+ Pattern('0x67', 0x3fe0000,
+ 2*GRANULARITY)]), # Overwrite [(64M-128K)-64M)
+ PatternGroup([
+ # Batch 3: 8 clusters (5 new)
+ # Carefully chosen such that nothing re-dirties the one cluster
+ # that copies out successfully before failure in Group #1.
+ Pattern('0xaa', 0x0010000,
+ 3*GRANULARITY), # Overwrite and 2x Adjacent-right
+ Pattern('0xbb', 0x00d8000), # Partial-left (1M-160K)
+ Pattern('0xcc', 0x2028000), # Partial-right (32M+160K)
+ Pattern('0xdd', 0x3fc0000)]), # New; leaving a gap to the right
+]
+
+
+class EmulatedBitmap:
+ def __init__(self, granularity=GRANULARITY):
+ self._bits = set()
+ self.granularity = granularity
+
+ def dirty_bits(self, bits):
+ self._bits |= set(bits)
+
+ def dirty_group(self, n):
+ self.dirty_bits(GROUPS[n].bits(self.granularity))
+
+ def clear(self):
+ self._bits = set()
+
+ def clear_bits(self, bits):
+ self._bits -= set(bits)
+
+ def clear_bit(self, bit):
+ self.clear_bits({bit})
+
+ def clear_group(self, n):
+ self.clear_bits(GROUPS[n].bits(self.granularity))
+
+ @property
+ def first_bit(self):
+ return sorted(self.bits)[0]
+
+ @property
+ def bits(self):
+ return self._bits
+
+ @property
+ def count(self):
+ return len(self.bits)
+
+ def compare(self, qmp_bitmap):
+ """
+ Print a nice human-readable message checking that a bitmap as reported
+ by the QMP interface has as many bits set as we expect it to.
+ """
+
+ name = qmp_bitmap.get('name', '(anonymous)')
+ log("= Checking Bitmap {:s} =".format(name))
+
+ want = self.count
+ have = qmp_bitmap['count'] // qmp_bitmap['granularity']
+
+ log("expecting {:d} dirty sectors; have {:d}. {:s}".format(
+ want, have, "OK!" if want == have else "ERROR!"))
+ log('')
+
+
+class Drive:
+ """Represents, vaguely, a drive attached to a VM.
+ Includes format, graph, and device information."""
+
+ def __init__(self, path, vm=None):
+ self.path = path
+ self.vm = vm
+ self.fmt = None
+ self.size = None
+ self.node = None
+
+ def img_create(self, fmt, size):
+ self.fmt = fmt
+ self.size = size
+ iotests.qemu_img_create('-f', self.fmt, self.path, str(self.size))
+
+ def create_target(self, name, fmt, size):
+ basename = os.path.basename(self.path)
+ file_node_name = "file_{}".format(basename)
+ vm = self.vm
+
+ log(vm.cmd('blockdev-create', job_id='bdc-file-job',
+ options={
+ 'driver': 'file',
+ 'filename': self.path,
+ 'size': 0,
+ }))
+ vm.run_job('bdc-file-job')
+ log(vm.cmd('blockdev-add', driver='file',
+ node_name=file_node_name, filename=self.path))
+
+ log(vm.cmd('blockdev-create', job_id='bdc-fmt-job',
+ options={
+ 'driver': fmt,
+ 'file': file_node_name,
+ 'size': size,
+ }))
+ vm.run_job('bdc-fmt-job')
+ log(vm.cmd('blockdev-add', driver=fmt,
+ node_name=name,
+ file=file_node_name))
+ self.fmt = fmt
+ self.size = size
+ self.node = name
+
+def blockdev_mirror(vm, device, target, sync, **kwargs):
+ # Strip any arguments explicitly nulled by the caller:
+ kwargs = {key: val for key, val in kwargs.items() if val is not None}
+ result = vm.qmp_log('blockdev-mirror',
+ device=device,
+ target=target,
+ sync=sync,
+ filter_node_name='mirror-top',
+ **kwargs)
+ return result
+
+def blockdev_mirror_mktarget(drive, target_id, filepath, sync, **kwargs):
+ target_drive = Drive(filepath, vm=drive.vm)
+ target_drive.create_target(target_id, drive.fmt, drive.size)
+ blockdev_mirror(drive.vm, drive.node, target_id, sync, **kwargs)
+
+def reference_mirror(drive, n, filepath):
+ log("--- Reference mirror #{:d} ---\n".format(n))
+ target_id = "ref_target_{:d}".format(n)
+ job_id = "ref_mirror_{:d}".format(n)
+ blockdev_mirror_mktarget(drive, target_id, filepath, "full",
+ job_id=job_id)
+ drive.vm.run_job(job_id, auto_dismiss=True)
+ log('')
+
+def bitmap_sync_pre(drive, bitmap_sync, bitmap):
+ vm = drive.vm
+ log("--- Prepare bitmap for sync mode '{}' ---\n".format(bitmap_sync))
+ if bitmap_sync == 'never' or bitmap_sync == 'on-success':
+ vm.qmp_log("block-dirty-bitmap-add", node=drive.node,
+ name="mirror-bitmap", granularity=GRANULARITY)
+ vm.qmp_log("block-dirty-bitmap-merge", node=drive.node,
+ target="mirror-bitmap", bitmaps=[bitmap])
+ log('')
+ return "mirror-bitmap";
+ elif bitmap_sync == 'always':
+ log("passing current bitmap '{}'\n".format(bitmap))
+ return bitmap;
+ else:
+ raise ValueError("unkown bitmap sync mode '{}'".format(bitmap_sync))
+
+def bitmap_sync_post(drive, bitmap_sync, bitmap, success):
+ vm = drive.vm
+ log("--- Post-process bitmap for sync mode '{}' ---\n".format(bitmap_sync))
+ if bitmap_sync == 'never':
+ vm.qmp_log("block-dirty-bitmap-remove", node=drive.node,
+ name="mirror-bitmap")
+ log('')
+ return
+ elif bitmap_sync == 'on-success':
+ if success:
+ vm.qmp_log("block-dirty-bitmap-clear", node=drive.node, name=bitmap)
+ vm.qmp_log("block-dirty-bitmap-merge", node=drive.node,
+ target=bitmap, bitmaps=["mirror-bitmap"])
+ vm.qmp_log("block-dirty-bitmap-remove", node=drive.node,
+ name="mirror-bitmap")
+ log('')
+ return
+ elif bitmap_sync == 'always':
+ log('nothing to do\n')
+ return
+ else:
+ raise ValueError("unkown bitmap sync mode '{}'".format(bitmap_sync))
+
+def mirror(drive, n, filepath, sync, **kwargs):
+ log("--- Test mirror #{:d} ---\n".format(n))
+ target_id = "mirror_target_{:d}".format(n)
+ job_id = "mirror_{:d}".format(n)
+ kwargs.setdefault('auto-finalize', False)
+ blockdev_mirror_mktarget(drive, target_id, filepath, sync,
+ job_id=job_id, **kwargs)
+ return job_id
+
+def perform_writes(drive, n, filter_node_name=None):
+ log("--- Write #{:d} ---\n".format(n))
+ for pattern in GROUPS[n].patterns:
+ cmd = "write -P{:s} 0x{:07x} 0x{:x}".format(
+ pattern.byte,
+ pattern.offset,
+ pattern.size)
+ log(cmd)
+ log(drive.vm.hmp_qemu_io(filter_node_name or drive.node, cmd))
+ bitmaps = drive.vm.query_bitmaps()
+ log({'bitmaps': bitmaps}, indent=2)
+ log('')
+ return bitmaps
+
+
+def compare_images(image, reference, baseimg=None, expected_match=True):
+ """
+ Print a nice human-readable message comparing these images.
+ """
+ expected_ret = 0 if expected_match else 1
+ if baseimg:
+ qemu_img("rebase", "-u", "-b", baseimg, '-F', iotests.imgfmt, image)
+
+ sub = qemu_img("compare", image, reference, check=False)
+
+ log('qemu_img compare "{:s}" "{:s}" ==> {:s}, {:s}'.format(
+ image, reference,
+ "Identical" if sub.returncode == 0 else "Mismatch",
+ "OK!" if sub.returncode == expected_ret else "ERROR!"),
+ filters=[iotests.filter_testfiles])
+
+def test_bitmap_sync(bsync_mode, failure=None):
+ """
+ Test bitmap mirror routines.
+
+ :param bsync_mode: Is the Bitmap Sync mode, and can be any of:
+ - on-success: This is the "incremental" style mode. Bitmaps are
+ synchronized to what was copied out only on success.
+ (Partial images must be discarded.)
+ - never: This is the "differential" style mode.
+ Bitmaps are never synchronized.
+ - always: This is a "best effort" style mode.
+ Bitmaps are always synchronized, regardless of failure.
+ (Partial images must be kept.)
+
+ :param failure: Is the (optional) failure mode, and can be any of:
+ - None: No failure. Test the normative path. Default.
+ - simulated: Cancel the job right before it completes.
+ This also tests writes "during" the job.
+ - intermediate: This tests a job that fails mid-process and produces
+ an incomplete mirror. Testing limitations prevent
+ testing competing writes.
+ """
+ with iotests.FilePath('img', 'bsync1', 'bsync2', 'bsync3',
+ 'fmirror0', 'fmirror1', 'fmirror2', 'fmirror3') as \
+ (img_path, bsync1, bsync2, bsync3,
+ fmirror0, fmirror1, fmirror2, fmirror3), \
+ iotests.VM() as vm:
+
+ mode = "Bitmap Sync {:s}".format(bsync_mode)
+ preposition = "with" if failure else "without"
+ cond = "{:s} {:s}".format(preposition,
+ "{:s} failure".format(failure) if failure
+ else "failure")
+ log("\n=== {:s} {:s} ===\n".format(mode, cond))
+
+ log('--- Preparing image & VM ---\n')
+ drive0 = Drive(img_path, vm=vm)
+ drive0.img_create(iotests.imgfmt, SIZE)
+ vm.add_device("virtio-scsi,id=scsi0")
+ vm.launch()
+
+ file_config = {
+ 'driver': 'file',
+ 'filename': drive0.path
+ }
+
+ if failure == 'intermediate':
+ file_config = {
+ 'driver': 'blkdebug',
+ 'image': file_config,
+ 'set-state': [{
+ 'event': 'flush_to_disk',
+ 'state': 1,
+ 'new_state': 2
+ }, {
+ 'event': 'read_aio',
+ 'state': 2,
+ 'new_state': 3
+ }, {
+ 'event': 'read_aio',
+ 'state': 3,
+ 'new_state': 4
+ }],
+ 'inject-error': [{
+ 'event': 'read_aio',
+ 'errno': 5,
+ 'state': 3,
+ 'immediately': False,
+ 'once': True
+ }, {
+ 'event': 'read_aio',
+ 'errno': 5,
+ 'state': 4,
+ 'immediately': False,
+ 'once': True
+ }]
+ }
+
+ drive0.node = 'drive0'
+ vm.qmp_log('blockdev-add',
+ filters=[iotests.filter_qmp_testfiles],
+ node_name=drive0.node,
+ driver=drive0.fmt,
+ file=file_config)
+ log('')
+
+ # 0 - Writes and Reference mirror
+ perform_writes(drive0, 0)
+ reference_mirror(drive0, 0, fmirror0)
+ log('--- Add Bitmap ---\n')
+ vm.qmp_log("block-dirty-bitmap-add", node=drive0.node,
+ name="bitmap0", granularity=GRANULARITY)
+ log('')
+ ebitmap = EmulatedBitmap()
+
+ # 1 - Writes and Reference mirror
+ bitmaps = perform_writes(drive0, 1)
+ ebitmap.dirty_group(1)
+ bitmap = vm.get_bitmap(drive0.node, 'bitmap0', bitmaps=bitmaps)
+ ebitmap.compare(bitmap)
+ reference_mirror(drive0, 1, fmirror1)
+
+ # 1 - Test mirror (w/ Optional induced failure)
+ if failure == 'intermediate':
+ # Activate blkdebug induced failure for second-to-next read
+ log(vm.hmp_qemu_io(drive0.node, 'flush'))
+ log('')
+ mirror_bitmap = bitmap_sync_pre(drive0, bsync_mode, "bitmap0")
+ job = mirror(drive0, 1, bsync1, 'full', bitmap=mirror_bitmap)
+
+ vm.run_job(job, auto_dismiss=True, auto_finalize=False,
+ cancel=(failure == 'simulated'))
+ bitmap_sync_post(drive0, bsync_mode, "bitmap0", not failure)
+
+ bitmaps = vm.query_bitmaps()
+ log({'bitmaps': bitmaps}, indent=2)
+ log('')
+
+ if bsync_mode == 'always':
+ if failure == 'intermediate':
+ # We manage to copy one sector (one bit) before the error.
+ ebitmap.clear_bit(ebitmap.first_bit)
+ else:
+ # successful mirror / cancelled complete mirror
+ ebitmap.clear()
+
+ if bsync_mode == 'on-success' and not failure:
+ ebitmap.clear()
+
+ ebitmap.compare(vm.get_bitmap(drive0.node, 'bitmap0', bitmaps=bitmaps))
+
+ # 2 - Reference mirror
+ reference_mirror(drive0, 2, fmirror2)
+
+ # 2 - Bitmap mirror with writes before completion
+ mirror_bitmap = bitmap_sync_pre(drive0, bsync_mode, "bitmap0")
+ job = mirror(drive0, 2, bsync2, "full", bitmap=mirror_bitmap)
+
+ bitmaps = perform_writes(drive0, 2, filter_node_name='mirror-top')
+ ebitmap.dirty_group(2)
+
+ # Can't compare against bitmap0 if used as the mirror bitmap, because
+ # mirror could already clear it.
+ if mirror_bitmap != "bitmap0":
+ ebitmap.compare(vm.get_bitmap(drive0.node, "bitmap0",
+ bitmaps=bitmaps))
+
+ # don't use run_job as that logs too much even with use_log=False
+ events = [('JOB_STATUS_CHANGE', {'data': {'id': job}})]
+ while True:
+ ev = iotests.filter_qmp_event(vm.events_wait(events, timeout=10))
+ status = ev['data']['status']
+ if status == 'ready':
+ vm.qmp('job-complete', id=job)
+ elif status == 'standby':
+ vm.qmp('job-resume', id=job)
+ elif status == 'pending':
+ vm.qmp('job-finalize', id=job)
+ elif status == 'null':
+ break
+
+ bitmap_sync_post(drive0, bsync_mode, "bitmap0", True)
+
+ if bsync_mode != 'never':
+ ebitmap.clear()
+
+ bitmaps = vm.query_bitmaps()
+ ebitmap.compare(vm.get_bitmap(drive0.node, 'bitmap0', bitmaps=bitmaps))
+
+ # 3 - Writes and Reference mirror
+ bitmaps = perform_writes(drive0, 3)
+ ebitmap.dirty_group(3)
+ ebitmap.compare(vm.get_bitmap(drive0.node, 'bitmap0', bitmaps=bitmaps))
+ reference_mirror(drive0, 3, fmirror3)
+
+ # 3 - Bitmap mirror (In failure modes, this is a recovery.)
+ mirror_bitmap = bitmap_sync_pre(drive0, bsync_mode, "bitmap0")
+ job = mirror(drive0, 3, bsync3, "full", bitmap=mirror_bitmap)
+
+ vm.run_job(job, auto_dismiss=True, auto_finalize=False)
+ bitmap_sync_post(drive0, bsync_mode, "bitmap0", True)
+ bitmaps = vm.query_bitmaps()
+
+ log({'bitmaps': bitmaps}, indent=2)
+ log('')
+ if bsync_mode != 'never':
+ ebitmap.clear()
+ ebitmap.compare(vm.get_bitmap(drive0.node, 'bitmap0', bitmaps=bitmaps))
+
+ log('--- Cleanup ---\n')
+ vm.qmp_log("block-dirty-bitmap-remove",
+ node=drive0.node, name="bitmap0")
+ bitmaps = vm.query_bitmaps()
+ log({'bitmaps': bitmaps}, indent=2)
+ vm.shutdown()
+ log('')
+
+ log('--- Verification ---\n')
+ compare_images(bsync1, fmirror1, baseimg=fmirror0,
+ expected_match=failure != 'intermediate')
+ if not failure or bsync_mode == 'always':
+ # Always keep the last mirror on success or when using 'always'
+ base = bsync1
+ else:
+ base = fmirror1
+
+ compare_images(bsync2, fmirror2, baseimg=base, expected_match=0)
+ compare_images(bsync3, fmirror3, baseimg=bsync2)
+ compare_images(img_path, fmirror3)
+ log('')
+
+def test_mirror_api():
+ """
+ Test malformed and prohibited invocations of the mirror API.
+ """
+ with iotests.FilePath('img', 'bsync1') as \
+ (img_path, mirror_path), \
+ iotests.VM() as vm:
+
+ log("\n=== API failure tests ===\n")
+ log('--- Preparing image & VM ---\n')
+ drive0 = Drive(img_path, vm=vm)
+ drive0.img_create(iotests.imgfmt, SIZE)
+ vm.add_device("virtio-scsi,id=scsi0")
+ vm.launch()
+
+ file_config = {
+ 'driver': 'file',
+ 'filename': drive0.path
+ }
+
+ drive0.node = 'drive0'
+ vm.qmp_log('blockdev-add',
+ filters=[iotests.filter_qmp_testfiles],
+ node_name=drive0.node,
+ driver=drive0.fmt,
+ file=file_config)
+ log('')
+
+ target0 = Drive(mirror_path, vm=vm)
+ target0.create_target("mirror_target", drive0.fmt, drive0.size)
+ log('')
+
+ vm.qmp_log("block-dirty-bitmap-add", node=drive0.node,
+ name="bitmap0", granularity=GRANULARITY)
+ log('')
+
+ log('-- Testing invalid QMP commands --\n')
+
+ error_cases = {
+ 'full' : ['bitmap404'],
+ 'top' : ['bitmap404', 'bitmap0'],
+ 'none' : ['bitmap404', 'bitmap0'],
+ }
+
+ # Dicts, as always, are not stably-ordered prior to 3.7, so use tuples:
+ for sync_mode in ('full', 'top', 'none'):
+ log("-- Sync mode {:s} tests --\n".format(sync_mode))
+ for bitmap in error_cases[sync_mode]:
+ blockdev_mirror(drive0.vm, drive0.node, "mirror_target",
+ sync_mode, job_id='api_job',
+ bitmap=bitmap)
+ log('')
+
+
+def main():
+ for bsync_mode in ("never", "on-success", "always"):
+ for failure in ("simulated", "intermediate", None):
+ test_bitmap_sync(bsync_mode, failure)
+
+ test_mirror_api()
+
+if __name__ == '__main__':
+ iotests.script_main(main, supported_fmts=['qcow2'],
+ supported_protocols=['file'])
diff --git a/tests/qemu-iotests/tests/bitmap-sync-mirror.out b/tests/qemu-iotests/tests/bitmap-sync-mirror.out
new file mode 100644
index 0000000000..c05b4788c6
--- /dev/null
+++ b/tests/qemu-iotests/tests/bitmap-sync-mirror.out
@@ -0,0 +1,2939 @@
+
+=== Bitmap Sync never with simulated failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {}
+}
+
+--- Reference mirror #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_0"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_1"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Prepare bitmap for sync mode 'never' ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["bitmap0"], "node": "drive0", "target": "mirror-bitmap"}}
+{"return": {}}
+
+--- Test mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "mirror-bitmap", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_1", "sync": "full", "target": "mirror_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_1"}}
+{"return": {}}
+{"data": {"device": "mirror_1", "len": 393216, "offset": 393216, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-cancel", "arguments": {"id": "mirror_1"}}
+{"return": {}}
+{"data": {"id": "mirror_1", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_1", "len": 393216, "offset": 393216, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+--- Post-process bitmap for sync mode 'never' ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_2"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Prepare bitmap for sync mode 'never' ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["bitmap0"], "node": "drive0", "target": "mirror-bitmap"}}
+{"return": {}}
+
+--- Test mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "mirror-bitmap", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_2", "sync": "full", "target": "mirror_target_2"}}
+{"return": {}}
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": true,
+ "count": 0,
+ "granularity": 65536,
+ "name": "mirror-bitmap",
+ "persistent": false,
+ "recording": false
+ },
+ {
+ "busy": false,
+ "count": 655360,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 10 dirty sectors; have 10. OK!
+
+--- Post-process bitmap for sync mode 'never' ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+
+= Checking Bitmap bitmap0 =
+expecting 10 dirty sectors; have 10. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 983040,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 15 dirty sectors; have 15. OK!
+
+--- Reference mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_3", "sync": "full", "target": "ref_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_3"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Prepare bitmap for sync mode 'never' ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["bitmap0"], "node": "drive0", "target": "mirror-bitmap"}}
+{"return": {}}
+
+--- Test mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "mirror-bitmap", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_3", "sync": "full", "target": "mirror_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"device": "mirror_3", "len": 983040, "offset": 983040, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"id": "mirror_3", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_3", "len": 983040, "offset": 983040, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+--- Post-process bitmap for sync mode 'never' ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 983040,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 15 dirty sectors; have 15. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {}
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fmirror1" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fmirror2" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync3" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+
+
+=== Bitmap Sync never with intermediate failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false, "once": true, "state": 3}, {"errno": 5, "event": "read_aio", "immediately": false, "once": true, "state": 4}], "set-state": [{"event": "flush_to_disk", "new-state": 2, "state": 1}, {"event": "read_aio", "new-state": 3, "state": 2}, {"event": "read_aio", "new-state": 4, "state": 3}]}, "node-name": "drive0"}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {}
+}
+
+--- Reference mirror #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_0"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_1"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+{"return": ""}
+
+--- Prepare bitmap for sync mode 'never' ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["bitmap0"], "node": "drive0", "target": "mirror-bitmap"}}
+{"return": {}}
+
+--- Test mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "mirror-bitmap", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_1", "sync": "full", "target": "mirror_target_1"}}
+{"return": {}}
+{"data": {"action": "report", "device": "mirror_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"action": "report", "device": "mirror_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_1", "error": "Input/output error", "len": 393216, "offset": 65536, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+--- Post-process bitmap for sync mode 'never' ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_2"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Prepare bitmap for sync mode 'never' ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["bitmap0"], "node": "drive0", "target": "mirror-bitmap"}}
+{"return": {}}
+
+--- Test mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "mirror-bitmap", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_2", "sync": "full", "target": "mirror_target_2"}}
+{"return": {}}
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": true,
+ "count": 0,
+ "granularity": 65536,
+ "name": "mirror-bitmap",
+ "persistent": false,
+ "recording": false
+ },
+ {
+ "busy": false,
+ "count": 655360,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 10 dirty sectors; have 10. OK!
+
+--- Post-process bitmap for sync mode 'never' ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+
+= Checking Bitmap bitmap0 =
+expecting 10 dirty sectors; have 10. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 983040,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 15 dirty sectors; have 15. OK!
+
+--- Reference mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_3", "sync": "full", "target": "ref_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_3"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Prepare bitmap for sync mode 'never' ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["bitmap0"], "node": "drive0", "target": "mirror-bitmap"}}
+{"return": {}}
+
+--- Test mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "mirror-bitmap", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_3", "sync": "full", "target": "mirror_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"device": "mirror_3", "len": 983040, "offset": 983040, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"id": "mirror_3", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_3", "len": 983040, "offset": 983040, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+--- Post-process bitmap for sync mode 'never' ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 983040,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 15 dirty sectors; have 15. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {}
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fmirror1" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fmirror2" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync3" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+
+
+=== Bitmap Sync never without failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {}
+}
+
+--- Reference mirror #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_0"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_1"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Prepare bitmap for sync mode 'never' ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["bitmap0"], "node": "drive0", "target": "mirror-bitmap"}}
+{"return": {}}
+
+--- Test mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "mirror-bitmap", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_1", "sync": "full", "target": "mirror_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_1"}}
+{"return": {}}
+{"data": {"device": "mirror_1", "len": 393216, "offset": 393216, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "mirror_1"}}
+{"return": {}}
+{"data": {"id": "mirror_1", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_1", "len": 393216, "offset": 393216, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+--- Post-process bitmap for sync mode 'never' ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_2"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Prepare bitmap for sync mode 'never' ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["bitmap0"], "node": "drive0", "target": "mirror-bitmap"}}
+{"return": {}}
+
+--- Test mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "mirror-bitmap", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_2", "sync": "full", "target": "mirror_target_2"}}
+{"return": {}}
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": true,
+ "count": 0,
+ "granularity": 65536,
+ "name": "mirror-bitmap",
+ "persistent": false,
+ "recording": false
+ },
+ {
+ "busy": false,
+ "count": 655360,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 10 dirty sectors; have 10. OK!
+
+--- Post-process bitmap for sync mode 'never' ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+
+= Checking Bitmap bitmap0 =
+expecting 10 dirty sectors; have 10. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 983040,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 15 dirty sectors; have 15. OK!
+
+--- Reference mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_3", "sync": "full", "target": "ref_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_3"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Prepare bitmap for sync mode 'never' ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["bitmap0"], "node": "drive0", "target": "mirror-bitmap"}}
+{"return": {}}
+
+--- Test mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "mirror-bitmap", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_3", "sync": "full", "target": "mirror_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"device": "mirror_3", "len": 983040, "offset": 983040, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"id": "mirror_3", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_3", "len": 983040, "offset": 983040, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+--- Post-process bitmap for sync mode 'never' ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 983040,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 15 dirty sectors; have 15. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {}
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fmirror1" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fmirror2" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync3" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+
+
+=== Bitmap Sync on-success with simulated failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {}
+}
+
+--- Reference mirror #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_0"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_1"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Prepare bitmap for sync mode 'on-success' ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["bitmap0"], "node": "drive0", "target": "mirror-bitmap"}}
+{"return": {}}
+
+--- Test mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "mirror-bitmap", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_1", "sync": "full", "target": "mirror_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_1"}}
+{"return": {}}
+{"data": {"device": "mirror_1", "len": 393216, "offset": 393216, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-cancel", "arguments": {"id": "mirror_1"}}
+{"return": {}}
+{"data": {"id": "mirror_1", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_1", "len": 393216, "offset": 393216, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+--- Post-process bitmap for sync mode 'on-success' ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_2"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Prepare bitmap for sync mode 'on-success' ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["bitmap0"], "node": "drive0", "target": "mirror-bitmap"}}
+{"return": {}}
+
+--- Test mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "mirror-bitmap", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_2", "sync": "full", "target": "mirror_target_2"}}
+{"return": {}}
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": true,
+ "count": 0,
+ "granularity": 65536,
+ "name": "mirror-bitmap",
+ "persistent": false,
+ "recording": false
+ },
+ {
+ "busy": false,
+ "count": 655360,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 10 dirty sectors; have 10. OK!
+
+--- Post-process bitmap for sync mode 'on-success' ---
+
+{"execute": "block-dirty-bitmap-clear", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["mirror-bitmap"], "node": "drive0", "target": "bitmap0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 524288,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 8 dirty sectors; have 8. OK!
+
+--- Reference mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_3", "sync": "full", "target": "ref_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_3"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Prepare bitmap for sync mode 'on-success' ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["bitmap0"], "node": "drive0", "target": "mirror-bitmap"}}
+{"return": {}}
+
+--- Test mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "mirror-bitmap", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_3", "sync": "full", "target": "mirror_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"device": "mirror_3", "len": 524288, "offset": 524288, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"id": "mirror_3", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_3", "len": 524288, "offset": 524288, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+--- Post-process bitmap for sync mode 'on-success' ---
+
+{"execute": "block-dirty-bitmap-clear", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["mirror-bitmap"], "node": "drive0", "target": "bitmap0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {}
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fmirror1" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fmirror2" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync3" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+
+
+=== Bitmap Sync on-success with intermediate failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false, "once": true, "state": 3}, {"errno": 5, "event": "read_aio", "immediately": false, "once": true, "state": 4}], "set-state": [{"event": "flush_to_disk", "new-state": 2, "state": 1}, {"event": "read_aio", "new-state": 3, "state": 2}, {"event": "read_aio", "new-state": 4, "state": 3}]}, "node-name": "drive0"}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {}
+}
+
+--- Reference mirror #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_0"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_1"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+{"return": ""}
+
+--- Prepare bitmap for sync mode 'on-success' ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["bitmap0"], "node": "drive0", "target": "mirror-bitmap"}}
+{"return": {}}
+
+--- Test mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "mirror-bitmap", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_1", "sync": "full", "target": "mirror_target_1"}}
+{"return": {}}
+{"data": {"action": "report", "device": "mirror_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"action": "report", "device": "mirror_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_1", "error": "Input/output error", "len": 393216, "offset": 65536, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+--- Post-process bitmap for sync mode 'on-success' ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_2"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Prepare bitmap for sync mode 'on-success' ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["bitmap0"], "node": "drive0", "target": "mirror-bitmap"}}
+{"return": {}}
+
+--- Test mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "mirror-bitmap", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_2", "sync": "full", "target": "mirror_target_2"}}
+{"return": {}}
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": true,
+ "count": 0,
+ "granularity": 65536,
+ "name": "mirror-bitmap",
+ "persistent": false,
+ "recording": false
+ },
+ {
+ "busy": false,
+ "count": 655360,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 10 dirty sectors; have 10. OK!
+
+--- Post-process bitmap for sync mode 'on-success' ---
+
+{"execute": "block-dirty-bitmap-clear", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["mirror-bitmap"], "node": "drive0", "target": "bitmap0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 524288,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 8 dirty sectors; have 8. OK!
+
+--- Reference mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_3", "sync": "full", "target": "ref_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_3"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Prepare bitmap for sync mode 'on-success' ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["bitmap0"], "node": "drive0", "target": "mirror-bitmap"}}
+{"return": {}}
+
+--- Test mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "mirror-bitmap", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_3", "sync": "full", "target": "mirror_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"device": "mirror_3", "len": 524288, "offset": 524288, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"id": "mirror_3", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_3", "len": 524288, "offset": 524288, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+--- Post-process bitmap for sync mode 'on-success' ---
+
+{"execute": "block-dirty-bitmap-clear", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["mirror-bitmap"], "node": "drive0", "target": "bitmap0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {}
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fmirror1" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fmirror2" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync3" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+
+
+=== Bitmap Sync on-success without failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {}
+}
+
+--- Reference mirror #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_0"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_1"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Prepare bitmap for sync mode 'on-success' ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["bitmap0"], "node": "drive0", "target": "mirror-bitmap"}}
+{"return": {}}
+
+--- Test mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "mirror-bitmap", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_1", "sync": "full", "target": "mirror_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_1"}}
+{"return": {}}
+{"data": {"device": "mirror_1", "len": 393216, "offset": 393216, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "mirror_1"}}
+{"return": {}}
+{"data": {"id": "mirror_1", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_1", "len": 393216, "offset": 393216, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+--- Post-process bitmap for sync mode 'on-success' ---
+
+{"execute": "block-dirty-bitmap-clear", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["mirror-bitmap"], "node": "drive0", "target": "bitmap0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Reference mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_2"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Prepare bitmap for sync mode 'on-success' ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["bitmap0"], "node": "drive0", "target": "mirror-bitmap"}}
+{"return": {}}
+
+--- Test mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "mirror-bitmap", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_2", "sync": "full", "target": "mirror_target_2"}}
+{"return": {}}
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": true,
+ "count": 0,
+ "granularity": 65536,
+ "name": "mirror-bitmap",
+ "persistent": false,
+ "recording": false
+ },
+ {
+ "busy": false,
+ "count": 458752,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 7 dirty sectors; have 7. OK!
+
+--- Post-process bitmap for sync mode 'on-success' ---
+
+{"execute": "block-dirty-bitmap-clear", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["mirror-bitmap"], "node": "drive0", "target": "bitmap0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 524288,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 8 dirty sectors; have 8. OK!
+
+--- Reference mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_3", "sync": "full", "target": "ref_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_3"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Prepare bitmap for sync mode 'on-success' ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["bitmap0"], "node": "drive0", "target": "mirror-bitmap"}}
+{"return": {}}
+
+--- Test mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "mirror-bitmap", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_3", "sync": "full", "target": "mirror_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"device": "mirror_3", "len": 524288, "offset": 524288, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"id": "mirror_3", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_3", "len": 524288, "offset": 524288, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+--- Post-process bitmap for sync mode 'on-success' ---
+
+{"execute": "block-dirty-bitmap-clear", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-merge", "arguments": {"bitmaps": ["mirror-bitmap"], "node": "drive0", "target": "bitmap0"}}
+{"return": {}}
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "mirror-bitmap", "node": "drive0"}}
+{"return": {}}
+
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {}
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fmirror1" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fmirror2" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync3" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+
+
+=== Bitmap Sync always with simulated failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {}
+}
+
+--- Reference mirror #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_0"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_1"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Prepare bitmap for sync mode 'always' ---
+
+passing current bitmap 'bitmap0'
+
+--- Test mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_1", "sync": "full", "target": "mirror_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_1"}}
+{"return": {}}
+{"data": {"device": "mirror_1", "len": 393216, "offset": 393216, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-cancel", "arguments": {"id": "mirror_1"}}
+{"return": {}}
+{"data": {"id": "mirror_1", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_1", "len": 393216, "offset": 393216, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+--- Post-process bitmap for sync mode 'always' ---
+
+nothing to do
+
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Reference mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_2"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Prepare bitmap for sync mode 'always' ---
+
+passing current bitmap 'bitmap0'
+
+--- Test mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_2", "sync": "full", "target": "mirror_target_2"}}
+{"return": {}}
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": true,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": false
+ }
+ ]
+ }
+}
+
+--- Post-process bitmap for sync mode 'always' ---
+
+nothing to do
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 524288,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 8 dirty sectors; have 8. OK!
+
+--- Reference mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_3", "sync": "full", "target": "ref_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_3"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Prepare bitmap for sync mode 'always' ---
+
+passing current bitmap 'bitmap0'
+
+--- Test mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_3", "sync": "full", "target": "mirror_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"device": "mirror_3", "len": 524288, "offset": 524288, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"id": "mirror_3", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_3", "len": 524288, "offset": 524288, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+--- Post-process bitmap for sync mode 'always' ---
+
+nothing to do
+
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {}
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fmirror1" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fmirror2" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync3" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+
+
+=== Bitmap Sync always with intermediate failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false, "once": true, "state": 3}, {"errno": 5, "event": "read_aio", "immediately": false, "once": true, "state": 4}], "set-state": [{"event": "flush_to_disk", "new-state": 2, "state": 1}, {"event": "read_aio", "new-state": 3, "state": 2}, {"event": "read_aio", "new-state": 4, "state": 3}]}, "node-name": "drive0"}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {}
+}
+
+--- Reference mirror #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_0"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_1"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+{"return": ""}
+
+--- Prepare bitmap for sync mode 'always' ---
+
+passing current bitmap 'bitmap0'
+
+--- Test mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_1", "sync": "full", "target": "mirror_target_1"}}
+{"return": {}}
+{"data": {"action": "report", "device": "mirror_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"action": "report", "device": "mirror_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_1", "error": "Input/output error", "len": 393216, "offset": 65536, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+--- Post-process bitmap for sync mode 'always' ---
+
+nothing to do
+
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 327680,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 5 dirty sectors; have 5. OK!
+
+--- Reference mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_2"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Prepare bitmap for sync mode 'always' ---
+
+passing current bitmap 'bitmap0'
+
+--- Test mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_2", "sync": "full", "target": "mirror_target_2"}}
+{"return": {}}
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": true,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": false
+ }
+ ]
+ }
+}
+
+--- Post-process bitmap for sync mode 'always' ---
+
+nothing to do
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 524288,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 8 dirty sectors; have 8. OK!
+
+--- Reference mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_3", "sync": "full", "target": "ref_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_3"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Prepare bitmap for sync mode 'always' ---
+
+passing current bitmap 'bitmap0'
+
+--- Test mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_3", "sync": "full", "target": "mirror_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"device": "mirror_3", "len": 524288, "offset": 524288, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"id": "mirror_3", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_3", "len": 524288, "offset": 524288, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+--- Post-process bitmap for sync mode 'always' ---
+
+nothing to do
+
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {}
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fmirror1" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fmirror2" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync3" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+
+
+=== Bitmap Sync always without failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {}
+}
+
+--- Reference mirror #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_0"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_1"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Prepare bitmap for sync mode 'always' ---
+
+passing current bitmap 'bitmap0'
+
+--- Test mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_1", "sync": "full", "target": "mirror_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_1"}}
+{"return": {}}
+{"data": {"device": "mirror_1", "len": 393216, "offset": 393216, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "mirror_1"}}
+{"return": {}}
+{"data": {"id": "mirror_1", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_1", "len": 393216, "offset": 393216, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+--- Post-process bitmap for sync mode 'always' ---
+
+nothing to do
+
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Reference mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_2"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Prepare bitmap for sync mode 'always' ---
+
+passing current bitmap 'bitmap0'
+
+--- Test mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_2", "sync": "full", "target": "mirror_target_2"}}
+{"return": {}}
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": true,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": false
+ }
+ ]
+ }
+}
+
+--- Post-process bitmap for sync mode 'always' ---
+
+nothing to do
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 524288,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 8 dirty sectors; have 8. OK!
+
+--- Reference mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_3", "sync": "full", "target": "ref_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_3"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Prepare bitmap for sync mode 'always' ---
+
+passing current bitmap 'bitmap0'
+
+--- Test mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_3", "sync": "full", "target": "mirror_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"device": "mirror_3", "len": 524288, "offset": 524288, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"id": "mirror_3", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_3", "len": 524288, "offset": 524288, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+--- Post-process bitmap for sync mode 'always' ---
+
+nothing to do
+
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {}
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fmirror1" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fmirror2" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync3" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+
+
+=== API failure tests ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+-- Testing invalid QMP commands --
+
+-- Sync mode full tests --
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap404", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "full", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Dirty bitmap 'bitmap404' not found"}}
+
+-- Sync mode top tests --
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap404", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "top", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'top' not supported with bitmap."}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap0", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "top", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'top' not supported with bitmap."}}
+
+-- Sync mode none tests --
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap404", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "none", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'none' not supported with bitmap."}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap0", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "none", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'none' not supported with bitmap."}}
+
--
2.39.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v2 4/4] blockdev: mirror: check for target's cluster size when using bitmap
2024-03-07 13:47 [PATCH v2 0/4] mirror: allow specifying working bitmap Fiona Ebner
` (2 preceding siblings ...)
2024-03-07 13:47 ` [PATCH v2 3/4] iotests: add test for bitmap mirror Fiona Ebner
@ 2024-03-07 13:47 ` Fiona Ebner
3 siblings, 0 replies; 11+ messages in thread
From: Fiona Ebner @ 2024-03-07 13:47 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, vsementsov, jsnow,
f.gruenbichler, t.lamprecht, mahaocong, xiechanglong.d,
wencongyang2
When using mirror with a bitmap and the target is a diff image, i.e.
one that should only contain the delta and was not synced to
previously, a too large cluster size for the target can be
problematic. In particular, when the mirror sends data to the target
aligned to the jobs granularity, but not aligned to the larger target
image's cluster size, the target's cluster would be allocated but only
be filled partially. When rebasing such a diff image later, the
corresponding cluster of the base image would get "masked" and the
part of the cluster not in the diff image is not accessible anymore.
Unfortunately, it is not always possible to check for the target
image's cluster size, e.g. when it's NBD. Because the limitation is
already documented in the QAPI description for the @bitmap parameter
and it's only required for special diff image use-case, simply skip
the check then.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
blockdev.c | 19 +++++++++++++++++++
tests/qemu-iotests/tests/bitmap-sync-mirror | 6 ++++++
.../qemu-iotests/tests/bitmap-sync-mirror.out | 7 +++++++
3 files changed, 32 insertions(+)
diff --git a/blockdev.c b/blockdev.c
index c76eb97a4c..968d44cd3b 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2847,6 +2847,9 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
}
if (bitmap_name) {
+ BlockDriverInfo bdi;
+ uint32_t bitmap_granularity;
+
if (sync != MIRROR_SYNC_MODE_FULL) {
error_setg(errp, "Sync mode '%s' not supported with bitmap.",
MirrorSyncMode_str(sync));
@@ -2863,6 +2866,22 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
return;
}
+ bitmap_granularity = bdrv_dirty_bitmap_granularity(bitmap);
+ /*
+ * Ignore if unable to get the info, e.g. when target is NBD. It's only
+ * relevant for syncing to a diff image and the documentation already
+ * states that the target's cluster size needs to small enough then.
+ */
+ if (bdrv_get_info(target, &bdi) >= 0) {
+ if (bitmap_granularity < bdi.cluster_size ||
+ bitmap_granularity % bdi.cluster_size != 0) {
+ error_setg(errp, "Bitmap granularity %u is not a multiple of "
+ "the target image's cluster size %u",
+ bitmap_granularity, bdi.cluster_size);
+ return;
+ }
+ }
+
if (bdrv_dirty_bitmap_check(bitmap, BDRV_BITMAP_ALLOW_RO, errp)) {
return;
}
diff --git a/tests/qemu-iotests/tests/bitmap-sync-mirror b/tests/qemu-iotests/tests/bitmap-sync-mirror
index 898f1f4ba4..cbd5cc99cc 100755
--- a/tests/qemu-iotests/tests/bitmap-sync-mirror
+++ b/tests/qemu-iotests/tests/bitmap-sync-mirror
@@ -552,6 +552,12 @@ def test_mirror_api():
bitmap=bitmap)
log('')
+ log("-- Test bitmap with too small granularity --\n".format(sync_mode))
+ vm.qmp_log("block-dirty-bitmap-add", node=drive0.node,
+ name="bitmap-small", granularity=(GRANULARITY // 2))
+ blockdev_mirror(drive0.vm, drive0.node, "mirror_target", "full",
+ job_id='api_job', bitmap="bitmap-small")
+ log('')
def main():
for bsync_mode in ("never", "on-success", "always"):
diff --git a/tests/qemu-iotests/tests/bitmap-sync-mirror.out b/tests/qemu-iotests/tests/bitmap-sync-mirror.out
index c05b4788c6..d40ea7d689 100644
--- a/tests/qemu-iotests/tests/bitmap-sync-mirror.out
+++ b/tests/qemu-iotests/tests/bitmap-sync-mirror.out
@@ -2937,3 +2937,10 @@ qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap0", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "none", "target": "mirror_target"}}
{"error": {"class": "GenericError", "desc": "Sync mode 'none' not supported with bitmap."}}
+-- Test bitmap with too small granularity --
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 32768, "name": "bitmap-small", "node": "drive0"}}
+{"return": {}}
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap-small", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "full", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap granularity 32768 is not a multiple of the target image's cluster size 65536"}}
+
--
2.39.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH v2 1/4] qapi/block-core: avoid the re-use of MirrorSyncMode for backup
2024-03-07 13:47 ` [PATCH v2 1/4] qapi/block-core: avoid the re-use of MirrorSyncMode for backup Fiona Ebner
@ 2024-03-08 7:34 ` Markus Armbruster
2024-04-01 12:51 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 11+ messages in thread
From: Markus Armbruster @ 2024-03-08 7:34 UTC (permalink / raw)
To: Fiona Ebner
Cc: qemu-devel, qemu-block, eblake, hreitz, kwolf, vsementsov, jsnow,
f.gruenbichler, t.lamprecht, mahaocong, xiechanglong.d,
wencongyang2
Fiona Ebner <f.ebner@proxmox.com> writes:
> Backup supports all modes listed in MirrorSyncMode, while mirror does
> not. Introduce BackupSyncMode by copying the current MirrorSyncMode
> and drop the variants mirror does not support from MirrorSyncMode as
> well as the corresponding manual check in mirror_start().
Results in tighter introspection: query-qmp-schema no longer reports
drive-mirror and blockdev-mirror accepting @sync values they actually
reject. Suggest to mention this in the commit message.
> Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
> ---
>
> I felt like keeping the "Since: X.Y" as before makes the most sense as
> to not lose history. Or is it necessary to change this for
> BackupSyncMode (and its members) since it got a new name?
Doc comments are for users of the QMP interface. Type names do not
matter there. I agree with your decision not to update the "since"
tags.
[...]
> diff --git a/qapi/block-core.json b/qapi/block-core.json
> index 1874f880a8..59d75b0793 100644
> --- a/qapi/block-core.json
> +++ b/qapi/block-core.json
> @@ -1304,10 +1304,10 @@
> 'data': ['report', 'ignore', 'enospc', 'stop', 'auto'] }
>
> ##
> -# @MirrorSyncMode:
> +# @BackupSyncMode:
> #
> -# An enumeration of possible behaviors for the initial synchronization
> -# phase of storage mirroring.
> +# An enumeration of possible behaviors for image synchronization used
> +# by backup jobs.
> #
> # @top: copies data in the topmost image to the destination
> #
> @@ -1323,7 +1323,7 @@
> #
> # Since: 1.3
> ##
> -{ 'enum': 'MirrorSyncMode',
> +{ 'enum': 'BackupSyncMode',
> 'data': ['top', 'full', 'none', 'incremental', 'bitmap'] }
>
> ##
> @@ -1347,6 +1347,23 @@
> { 'enum': 'BitmapSyncMode',
> 'data': ['on-success', 'never', 'always'] }
>
> +##
> +# @MirrorSyncMode:
> +#
> +# An enumeration of possible behaviors for the initial synchronization
> +# phase of storage mirroring.
> +#
> +# @top: copies data in the topmost image to the destination
> +#
> +# @full: copies data from all images to the destination
> +#
> +# @none: only copy data written from now on
> +#
> +# Since: 1.3
> +##
> +{ 'enum': 'MirrorSyncMode',
> + 'data': ['top', 'full', 'none'] }
> +
> ##
> # @MirrorCopyMode:
> #
> @@ -1624,7 +1641,7 @@
> ##
> { 'struct': 'BackupCommon',
> 'data': { '*job-id': 'str', 'device': 'str',
> - 'sync': 'MirrorSyncMode', '*speed': 'int',
> + 'sync': 'BackupSyncMode', '*speed': 'int',
> '*bitmap': 'str', '*bitmap-mode': 'BitmapSyncMode',
> '*compress': 'bool',
> '*on-source-error': 'BlockdevOnError',
QAPI schema
Acked-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2 2/4] mirror: allow specifying working bitmap
2024-03-07 13:47 ` [PATCH v2 2/4] mirror: allow specifying working bitmap Fiona Ebner
@ 2024-03-08 7:41 ` Markus Armbruster
2024-04-02 20:14 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 11+ messages in thread
From: Markus Armbruster @ 2024-03-08 7:41 UTC (permalink / raw)
To: Fiona Ebner
Cc: qemu-devel, qemu-block, eblake, hreitz, kwolf, vsementsov, jsnow,
f.gruenbichler, t.lamprecht, mahaocong, xiechanglong.d,
wencongyang2
Fiona Ebner <f.ebner@proxmox.com> writes:
> From: John Snow <jsnow@redhat.com>
>
> for the mirror job. The bitmap's granularity is used as the job's
> granularity.
>
> The new @bitmap parameter is marked unstable in the QAPI and can
> currently only be used for @sync=full mode.
>
> Clusters initially dirty in the bitmap as well as new writes are
> copied to the target.
>
> Using block-dirty-bitmap-clear and block-dirty-bitmap-merge API,
> callers can simulate the three kinds of @BitmapSyncMode (which is used
> by backup):
> 1. always: default, just pass bitmap as working bitmap.
> 2. never: copy bitmap and pass copy to the mirror job.
> 3. on-success: copy bitmap and pass copy to the mirror job and if
> successful, merge bitmap into original afterwards.
>
> When the target image is a fresh "diff image", i.e. one that was not
> used as the target of a previous mirror and the target image's cluster
> size is larger than the bitmap's granularity, or when
> @copy-mode=write-blocking is used, there is a pitfall, because the
> cluster in the target image will be allocated, but not contain all the
> data corresponding to the same region in the source image.
>
> An idea to avoid the limitation would be to mark clusters which are
> affected by unaligned writes and are not allocated in the target image
> dirty, so they would be copied fully later. However, for migration,
> the invariant that an actively synced mirror stays actively synced
> (unless an error happens) is useful, because without that invariant,
> migration might inactivate block devices when mirror still got work
> to do and run into an assertion failure [0].
>
> Another approach would be to read the missing data from the source
> upon unaligned writes to be able to write the full target cluster
> instead.
>
> But certain targets like NBD do not allow querying the cluster size.
> To avoid limiting/breaking the use case of syncing to an existing
> target, which is arguably more common than the diff image use case,
> document the limiation in QAPI.
>
> This patch was originally based on one by Ma Haocong, but it has since
> been modified pretty heavily, first by John and then again by Fiona.
>
> [0]: https://lore.kernel.org/qemu-devel/1db7f571-cb7f-c293-04cc-cd856e060c3f@proxmox.com/
>
> Suggested-by: Ma Haocong <mahaocong@didichuxing.com>
> Signed-off-by: Ma Haocong <mahaocong@didichuxing.com>
> Signed-off-by: John Snow <jsnow@redhat.com>
> [FG: switch to bdrv_dirty_bitmap_merge_internal]
> Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
> Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
> [FE: rebase for 9.0
> get rid of bitmap mode parameter
> use caller-provided bitmap as working bitmap
> turn bitmap parameter experimental]
> Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
[...]
> diff --git a/qapi/block-core.json b/qapi/block-core.json
> index 59d75b0793..4859fffd48 100644
> --- a/qapi/block-core.json
> +++ b/qapi/block-core.json
> @@ -2191,6 +2191,18 @@
> # destination (all the disk, only the sectors allocated in the
> # topmost image, or only new I/O).
> #
> +# @bitmap: The name of a bitmap to use as a working bitmap for
> +# sync=full mode. This argument must be not be present for other
> +# sync modes and not at the same time as @granularity. The
> +# bitmap's granularity is used as the job's granularity. When
> +# the target is a diff image, i.e. one that should only contain
> +# the delta and was not synced to previously, the target's
> +# cluster size must not be larger than the bitmap's granularity.
> +# For a diff image target, using copy-mode=write-blocking should
> +# not be used, because unaligned writes will lead to allocated
> +# clusters with partial data in the target image! The bitmap
> +# will be enabled after the job finishes. (Since 9.0)
That's a lot of restrictions and caveats. Okay as long as the thing
remains experimental, I guess.
> +#
> # @granularity: granularity of the dirty bitmap, default is 64K if the
> # image format doesn't have clusters, 4K if the clusters are
> # smaller than that, else the cluster size. Must be a power of 2
> @@ -2228,12 +2240,18 @@
> # disappear from the query list without user intervention.
> # Defaults to true. (Since 3.1)
> #
> +# Features:
> +#
> +# @unstable: Member @bitmap is experimental.
> +#
> # Since: 1.3
> ##
> { 'struct': 'DriveMirror',
> 'data': { '*job-id': 'str', 'device': 'str', 'target': 'str',
> '*format': 'str', '*node-name': 'str', '*replaces': 'str',
> - 'sync': 'MirrorSyncMode', '*mode': 'NewImageMode',
> + 'sync': 'MirrorSyncMode',
> + '*bitmap': { 'type': 'str', 'features': [ 'unstable' ] },
> + '*mode': 'NewImageMode',
> '*speed': 'int', '*granularity': 'uint32',
> '*buf-size': 'int', '*on-source-error': 'BlockdevOnError',
> '*on-target-error': 'BlockdevOnError',
> @@ -2513,6 +2531,18 @@
> # destination (all the disk, only the sectors allocated in the
> # topmost image, or only new I/O).
> #
> +# @bitmap: The name of a bitmap to use as a working bitmap for
> +# sync=full mode. This argument must be not be present for other
> +# sync modes and not at the same time as @granularity. The
> +# bitmap's granularity is used as the job's granularity. When
> +# the target is a diff image, i.e. one that should only contain
> +# the delta and was not synced to previously, the target's
> +# cluster size must not be larger than the bitmap's granularity.
> +# For a diff image target, using copy-mode=write-blocking should
> +# not be used, because unaligned writes will lead to allocated
> +# clusters with partial data in the target image! The bitmap
> +# will be enabled after the job finishes. (Since 9.0)
> +#
> # @granularity: granularity of the dirty bitmap, default is 64K if the
> # image format doesn't have clusters, 4K if the clusters are
> # smaller than that, else the cluster size. Must be a power of 2
> @@ -2548,6 +2578,10 @@
> # disappear from the query list without user intervention.
> # Defaults to true. (Since 3.1)
> #
> +# Features:
> +#
> +# @unstable: Member @bitmap is experimental.
> +#
> # Since: 2.6
> #
> # Example:
> @@ -2562,6 +2596,7 @@
> 'data': { '*job-id': 'str', 'device': 'str', 'target': 'str',
> '*replaces': 'str',
> 'sync': 'MirrorSyncMode',
> + '*bitmap': { 'type': 'str', 'features': [ 'unstable' ] },
> '*speed': 'int', '*granularity': 'uint32',
> '*buf-size': 'int', '*on-source-error': 'BlockdevOnError',
> '*on-target-error': 'BlockdevOnError',
Acked-by: Markus Armbruster <armbru@redhat.com>
[...]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2 1/4] qapi/block-core: avoid the re-use of MirrorSyncMode for backup
2024-03-07 13:47 ` [PATCH v2 1/4] qapi/block-core: avoid the re-use of MirrorSyncMode for backup Fiona Ebner
2024-03-08 7:34 ` Markus Armbruster
@ 2024-04-01 12:51 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 11+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2024-04-01 12:51 UTC (permalink / raw)
To: Fiona Ebner, qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, jsnow, f.gruenbichler,
t.lamprecht, mahaocong, xiechanglong.d, wencongyang2
On 07.03.24 16:47, Fiona Ebner wrote:
> Backup supports all modes listed in MirrorSyncMode, while mirror does
> not. Introduce BackupSyncMode by copying the current MirrorSyncMode
> and drop the variants mirror does not support from MirrorSyncMode as
> well as the corresponding manual check in mirror_start().
>
> Suggested-by: Vladimir Sementsov-Ogievskiy<vsementsov@yandex-team.ru>
> Signed-off-by: Fiona Ebner<f.ebner@proxmox.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2 2/4] mirror: allow specifying working bitmap
2024-03-07 13:47 ` [PATCH v2 2/4] mirror: allow specifying working bitmap Fiona Ebner
2024-03-08 7:41 ` Markus Armbruster
@ 2024-04-02 20:14 ` Vladimir Sementsov-Ogievskiy
2024-05-07 12:15 ` Fiona Ebner
1 sibling, 1 reply; 11+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2024-04-02 20:14 UTC (permalink / raw)
To: Fiona Ebner, qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, jsnow, f.gruenbichler,
t.lamprecht, mahaocong, xiechanglong.d, wencongyang2
On 07.03.24 16:47, Fiona Ebner wrote:
> From: John Snow <jsnow@redhat.com>
>
> for the mirror job. The bitmap's granularity is used as the job's
> granularity.
>
> The new @bitmap parameter is marked unstable in the QAPI and can
> currently only be used for @sync=full mode.
>
> Clusters initially dirty in the bitmap as well as new writes are
> copied to the target.
>
> Using block-dirty-bitmap-clear and block-dirty-bitmap-merge API,
> callers can simulate the three kinds of @BitmapSyncMode (which is used
> by backup):
> 1. always: default, just pass bitmap as working bitmap.
> 2. never: copy bitmap and pass copy to the mirror job.
> 3. on-success: copy bitmap and pass copy to the mirror job and if
> successful, merge bitmap into original afterwards.
>
> When the target image is a fresh "diff image", i.e. one that was not
> used as the target of a previous mirror and the target image's cluster
> size is larger than the bitmap's granularity, or when
> @copy-mode=write-blocking is used, there is a pitfall, because the
> cluster in the target image will be allocated, but not contain all the
> data corresponding to the same region in the source image.
>
> An idea to avoid the limitation would be to mark clusters which are
> affected by unaligned writes and are not allocated in the target image
> dirty, so they would be copied fully later. However, for migration,
> the invariant that an actively synced mirror stays actively synced
> (unless an error happens) is useful, because without that invariant,
> migration might inactivate block devices when mirror still got work
> to do and run into an assertion failure [0].
>
> Another approach would be to read the missing data from the source
> upon unaligned writes to be able to write the full target cluster
> instead.
>
> But certain targets like NBD do not allow querying the cluster size.
> To avoid limiting/breaking the use case of syncing to an existing
> target, which is arguably more common than the diff image use case,
> document the limiation in QAPI.
>
> This patch was originally based on one by Ma Haocong, but it has since
> been modified pretty heavily, first by John and then again by Fiona.
>
> [0]: https://lore.kernel.org/qemu-devel/1db7f571-cb7f-c293-04cc-cd856e060c3f@proxmox.com/
>
> Suggested-by: Ma Haocong <mahaocong@didichuxing.com>
> Signed-off-by: Ma Haocong <mahaocong@didichuxing.com>
> Signed-off-by: John Snow <jsnow@redhat.com>
> [FG: switch to bdrv_dirty_bitmap_merge_internal]
> Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
> Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
> [FE: rebase for 9.0
> get rid of bitmap mode parameter
> use caller-provided bitmap as working bitmap
> turn bitmap parameter experimental]
> Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
> ---
> block/mirror.c | 95 ++++++++++++++++++++------
> blockdev.c | 39 +++++++++--
> include/block/block_int-global-state.h | 5 +-
> qapi/block-core.json | 37 +++++++++-
> tests/unit/test-block-iothread.c | 2 +-
> 5 files changed, 146 insertions(+), 32 deletions(-)
>
> diff --git a/block/mirror.c b/block/mirror.c
> index 1609354db3..5c9a00b574 100644
> --- a/block/mirror.c
> +++ b/block/mirror.c
> @@ -51,7 +51,7 @@ typedef struct MirrorBlockJob {
> BlockDriverState *to_replace;
> /* Used to block operations on the drive-mirror-replace target */
> Error *replace_blocker;
> - bool is_none_mode;
> + MirrorSyncMode sync_mode;
Could you please split this change to separate preparation patch?
> BlockMirrorBackingMode backing_mode;
> /* Whether the target image requires explicit zero-initialization */
> bool zero_target;
> @@ -73,6 +73,11 @@ typedef struct MirrorBlockJob {
> size_t buf_size;
> int64_t bdev_length;
> unsigned long *cow_bitmap;
> + /*
> + * Whether the bitmap is created locally or provided by the caller (for
> + * incremental sync).
> + */
> + bool dirty_bitmap_is_local;
> BdrvDirtyBitmap *dirty_bitmap;
> BdrvDirtyBitmapIter *dbi;
[..]
> + if (bitmap_name) {
> + if (sync != MIRROR_SYNC_MODE_FULL) {
> + error_setg(errp, "Sync mode '%s' not supported with bitmap.",
> + MirrorSyncMode_str(sync));
> + return;
> + }
> + if (granularity) {
> + error_setg(errp, "Granularity and bitmap cannot both be set");
> + return;
> + }
> +
> + bitmap = bdrv_find_dirty_bitmap(bs, bitmap_name);
> + if (!bitmap) {
> + error_setg(errp, "Dirty bitmap '%s' not found", bitmap_name);
> + return;
> + }
> +
> + if (bdrv_dirty_bitmap_check(bitmap, BDRV_BITMAP_ALLOW_RO, errp)) {
Why allow read-only bitmaps?
> + return;
> + }
> + }
> +
> if (!bdrv_backing_chain_next(bs) && sync == MIRROR_SYNC_MODE_TOP) {
> sync = MIRROR_SYNC_MODE_FULL;
> }
> @@ -2889,10 +2913,9 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
[..]
> +# @unstable: Member @bitmap is experimental.
> +#
> # Since: 1.3
> ##
> { 'struct': 'DriveMirror',
> 'data': { '*job-id': 'str', 'device': 'str', 'target': 'str',
> '*format': 'str', '*node-name': 'str', '*replaces': 'str',
> - 'sync': 'MirrorSyncMode', '*mode': 'NewImageMode',
> + 'sync': 'MirrorSyncMode',
> + '*bitmap': { 'type': 'str', 'features': [ 'unstable' ] },
> + '*mode': 'NewImageMode',
> '*speed': 'int', '*granularity': 'uint32',
> '*buf-size': 'int', '*on-source-error': 'BlockdevOnError',
> '*on-target-error': 'BlockdevOnError',
> @@ -2513,6 +2531,18 @@
> # destination (all the disk, only the sectors allocated in the
> # topmost image, or only new I/O).
> #
> +# @bitmap: The name of a bitmap to use as a working bitmap for
> +# sync=full mode. This argument must be not be present for other
> +# sync modes and not at the same time as @granularity. The
> +# bitmap's granularity is used as the job's granularity. When
> +# the target is a diff image, i.e. one that should only contain
> +# the delta and was not synced to previously, the target's
> +# cluster size must not be larger than the bitmap's granularity.
Could we check this? Like in block_copy_calculate_cluster_size(), we can check if target does COW, and if not, we can check that we are safe with granularity.
> +# For a diff image target, using copy-mode=write-blocking should
> +# not be used, because unaligned writes will lead to allocated
> +# clusters with partial data in the target image!
Could this be checked?
> The bitmap
> +# will be enabled after the job finishes. (Since 9.0)
Hmm. That looks correct. At least for the case, when bitmap is enabled at that start of job. Suggest to require this.
> +#
> # @granularity: granularity of the dirty bitmap, default is 64K if the
> # image format doesn't have clusters, 4K if the clusters are
> # smaller than that, else the cluster size. Must be a power of 2
> @@ -2548,6 +2578,10 @@
> # disappear from the query list without user intervention.
> # Defaults to true. (Since 3.1)
> #
> +# Features:
> +#
> +# @unstable: Member @bitmap is experimental.
> +#
> # Since: 2.6
Y_MODE_BACKGROUND,
> &error_abort);
[..]
Generally looks good to me.
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2 2/4] mirror: allow specifying working bitmap
2024-04-02 20:14 ` Vladimir Sementsov-Ogievskiy
@ 2024-05-07 12:15 ` Fiona Ebner
2024-05-08 12:43 ` Fiona Ebner
0 siblings, 1 reply; 11+ messages in thread
From: Fiona Ebner @ 2024-05-07 12:15 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, jsnow, f.gruenbichler,
t.lamprecht, mahaocong, xiechanglong.d, wencongyang2
Am 02.04.24 um 22:14 schrieb Vladimir Sementsov-Ogievskiy:
> On 07.03.24 16:47, Fiona Ebner wrote:
>> diff --git a/block/mirror.c b/block/mirror.c
>> index 1609354db3..5c9a00b574 100644
>> --- a/block/mirror.c
>> +++ b/block/mirror.c
>> @@ -51,7 +51,7 @@ typedef struct MirrorBlockJob {
>> BlockDriverState *to_replace;
>> /* Used to block operations on the drive-mirror-replace target */
>> Error *replace_blocker;
>> - bool is_none_mode;
>> + MirrorSyncMode sync_mode;
>
> Could you please split this change to separate preparation patch?
>
Will do.
>> + if (bdrv_dirty_bitmap_check(bitmap, BDRV_BITMAP_ALLOW_RO,
>> errp)) {
>
> Why allow read-only bitmaps?
>
Good catch! This is a left-over from an earlier version. Now that the
bitmap shall be used as the working bitmap, it cannot be read-only. I'll
change it to BDRV_BITMAP_DEFAULT in v3 of the series.
>> +# @bitmap: The name of a bitmap to use as a working bitmap for
>> +# sync=full mode. This argument must be not be present for other
>> +# sync modes and not at the same time as @granularity. The
>> +# bitmap's granularity is used as the job's granularity. When
>> +# the target is a diff image, i.e. one that should only contain
>> +# the delta and was not synced to previously, the target's
>> +# cluster size must not be larger than the bitmap's granularity.
>
> Could we check this? Like in block_copy_calculate_cluster_size(), we can
> check if target does COW, and if not, we can check that we are safe with
> granularity.
>
The issue here is (in particular) present when the target does COW, i.e.
in qcow2 diff images, allocated clusters which end up with partial data,
when we don't have the right cluster size. Patch 4/4 adds the check for
the target's cluster size.
>> +# For a diff image target, using copy-mode=write-blocking should
>> +# not be used, because unaligned writes will lead to allocated
>> +# clusters with partial data in the target image!
>
> Could this be checked?
>
I don't think so. How should we know if the target already contains data
from a previous full sync or not?
Those caveats when using diff images are unfortunate, and users should
be warned about them of course, but the main/expected use case for the
feature is to sync to the same target multiple times, so I'd hope the
cluster size check in patch 4/4 and mentioning the edge cases in the
documentation is enough here.
>> The bitmap
>> +# will be enabled after the job finishes. (Since 9.0)
>
> Hmm. That looks correct. At least for the case, when bitmap is enabled
> at that start of job. Suggest to require this.
>
It's true for any provided bitmap: it will be disabled when the mirror
job starts, because we manually set it in bdrv_mirror_top_do_write() and
then in mirror_exit_common(), the bitmap will be enabled.
Okay, I'll require that it is enabled at the beginning.
>> +#
>> # @granularity: granularity of the dirty bitmap, default is 64K if the
>> # image format doesn't have clusters, 4K if the clusters are
>> # smaller than that, else the cluster size. Must be a power of 2
>> @@ -2548,6 +2578,10 @@
>> # disappear from the query list without user intervention.
>> # Defaults to true. (Since 3.1)
>> #
>> +# Features:
>> +#
>> +# @unstable: Member @bitmap is experimental.
>> +#
>> # Since: 2.6
>
> Y_MODE_BACKGROUND,
>> &error_abort);
>
> [..]
>
> Generally looks good to me.
>
Thank you for the review!
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2 2/4] mirror: allow specifying working bitmap
2024-05-07 12:15 ` Fiona Ebner
@ 2024-05-08 12:43 ` Fiona Ebner
0 siblings, 0 replies; 11+ messages in thread
From: Fiona Ebner @ 2024-05-08 12:43 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, jsnow, f.gruenbichler,
t.lamprecht, mahaocong, xiechanglong.d, wencongyang2
Am 07.05.24 um 14:15 schrieb Fiona Ebner:
> Am 02.04.24 um 22:14 schrieb Vladimir Sementsov-Ogievskiy:
>> On 07.03.24 16:47, Fiona Ebner wrote:
>>> +# @bitmap: The name of a bitmap to use as a working bitmap for
>>> +# sync=full mode. This argument must be not be present for other
>>> +# sync modes and not at the same time as @granularity. The
>>> +# bitmap's granularity is used as the job's granularity. When
>>> +# the target is a diff image, i.e. one that should only contain
>>> +# the delta and was not synced to previously, the target's
>>> +# cluster size must not be larger than the bitmap's granularity.
>>
>> Could we check this? Like in block_copy_calculate_cluster_size(), we can
>> check if target does COW, and if not, we can check that we are safe with
>> granularity.
>>
>
> The issue here is (in particular) present when the target does COW, i.e.
> in qcow2 diff images, allocated clusters which end up with partial data,
> when we don't have the right cluster size. Patch 4/4 adds the check for
> the target's cluster size.
>
Sorry, no. What I said is wrong. It's just that the test does something
very pathological and does not even use COW/backing files. All the
mirror targets are separate diff images there. So yes, we can do the
same as block_copy_calculate_cluster_size() and the issue only appears
in the same edge cases as for backup where we can error out early. This
also applies to copy-mode=write-blocking AFAICT.
>>> +# For a diff image target, using copy-mode=write-blocking should
>>> +# not be used, because unaligned writes will lead to allocated
>>> +# clusters with partial data in the target image!
>>
>> Could this be checked?
>>
>
> I don't think so. How should we know if the target already contains data
> from a previous full sync or not?
>
> Those caveats when using diff images are unfortunate, and users should
> be warned about them of course, but the main/expected use case for the
> feature is to sync to the same target multiple times, so I'd hope the
> cluster size check in patch 4/4 and mentioning the edge cases in the
> documentation is enough here.
>
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2024-05-08 12:44 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-03-07 13:47 [PATCH v2 0/4] mirror: allow specifying working bitmap Fiona Ebner
2024-03-07 13:47 ` [PATCH v2 1/4] qapi/block-core: avoid the re-use of MirrorSyncMode for backup Fiona Ebner
2024-03-08 7:34 ` Markus Armbruster
2024-04-01 12:51 ` Vladimir Sementsov-Ogievskiy
2024-03-07 13:47 ` [PATCH v2 2/4] mirror: allow specifying working bitmap Fiona Ebner
2024-03-08 7:41 ` Markus Armbruster
2024-04-02 20:14 ` Vladimir Sementsov-Ogievskiy
2024-05-07 12:15 ` Fiona Ebner
2024-05-08 12:43 ` Fiona Ebner
2024-03-07 13:47 ` [PATCH v2 3/4] iotests: add test for bitmap mirror Fiona Ebner
2024-03-07 13:47 ` [PATCH v2 4/4] blockdev: mirror: check for target's cluster size when using bitmap Fiona Ebner
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).