* [RFC 0/4] mirror: implement incremental and bitmap modes
@ 2024-02-16 10:55 Fiona Ebner
2024-02-16 10:55 ` [RFC 1/4] drive-mirror: add support for sync=bitmap mode=never Fiona Ebner
` (5 more replies)
0 siblings, 6 replies; 25+ messages in thread
From: Fiona Ebner @ 2024-02-16 10:55 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, vsementsov, jsnow,
f.gruenbichler, t.lamprecht, mahaocong
Previous discussion from when this was sent upstream [0] (it's been a
while). I rebased the patches and re-ordered and squashed like
suggested back then [1].
This implements two new mirror modes:
- bitmap mirror mode with always/on-success/never bitmap sync mode
- incremental mirror mode as sugar for bitmap + on-success
Use cases:
* Possibility to resume a failed mirror later.
* Possibility to only mirror deltas to a previously mirrored volume.
* Possibility to (efficiently) mirror an drive that was previously
mirrored via some external mechanism (e.g. ZFS replication).
We are using the last one in production without any issues since about
4 years now. In particular, like mentioned in [2]:
> - create bitmap(s)
> - (incrementally) replicate storage volume(s) out of band (using ZFS)
> - incrementally drive mirror as part of a live migration of VM
> - drop bitmap(s)
Now, the IO test added in patch 4/4 actually contains yet another use
case, namely doing incremental mirrors to stand-alone qcow2 "diff"
images, that only contain the delta and can be rebased later. I had to
adapt the IO test, because its output expected the mirror bitmap to
still be dirty, but nowadays the mirror is apparently already done
when the bitmaps are queried. So I thought, I'll just use
'write-blocking' mode to avoid any potential timing issues.
But this exposed an issue with the diff image approach. If a write is
not aligned to the granularity of the mirror target, then rebasing the
diff image onto a backing image will not yield the desired result,
because the full cluster is considered to be allocated and will "hide"
some part of the base/backing image. The failure can be seen by either
using 'write-blocking' mode in the IO test or setting the (bitmap)
granularity to 32 KiB rather than the current 64 KiB.
The question is how to deal with these edge cases? Some possibilities
that would make sense to me:
For 'background' mode:
* prohibit if target's cluster size is larger than the bitmap
granularity
* document the limitation
For 'write-blocking' mode:
* disallow in combination with bitmap mode (would not be happy about
it, because I'd like to use this without diff images)
* for writes that are not aligned to the target's cluster size, read
the relevant/missing parts from the source image to be able to write
whole target clusters (seems rather complex)
* document the limitation
[0]: https://lore.kernel.org/qemu-devel/20200218100740.2228521-1-f.gruenbichler@proxmox.com/
[1]: https://lore.kernel.org/qemu-devel/d35a76de-78d5-af56-0b34-f7bd2bbd3733@redhat.com/
[2]: https://lore.kernel.org/qemu-devel/1599127031.9uxdp5h9o2.astroid@nora.none/
Fabian Grünbichler (2):
mirror: move some checks to qmp
iotests: add test for bitmap mirror
John Snow (2):
drive-mirror: add support for sync=bitmap mode=never
drive-mirror: add support for conditional and always bitmap sync modes
block/mirror.c | 94 +-
blockdev.c | 70 +-
include/block/block_int-global-state.h | 4 +-
qapi/block-core.json | 25 +-
tests/qemu-iotests/tests/bitmap-sync-mirror | 550 ++++
.../qemu-iotests/tests/bitmap-sync-mirror.out | 2810 +++++++++++++++++
tests/unit/test-block-iothread.c | 4 +-
7 files changed, 3527 insertions(+), 30 deletions(-)
create mode 100755 tests/qemu-iotests/tests/bitmap-sync-mirror
create mode 100644 tests/qemu-iotests/tests/bitmap-sync-mirror.out
--
2.39.2
^ permalink raw reply [flat|nested] 25+ messages in thread
* [RFC 1/4] drive-mirror: add support for sync=bitmap mode=never
2024-02-16 10:55 [RFC 0/4] mirror: implement incremental and bitmap modes Fiona Ebner
@ 2024-02-16 10:55 ` Fiona Ebner
2024-02-21 6:55 ` Markus Armbruster
2024-02-16 10:55 ` [RFC 2/4] drive-mirror: add support for conditional and always bitmap sync modes Fiona Ebner
` (4 subsequent siblings)
5 siblings, 1 reply; 25+ messages in thread
From: Fiona Ebner @ 2024-02-16 10:55 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, vsementsov, jsnow,
f.gruenbichler, t.lamprecht, mahaocong
From: John Snow <jsnow@redhat.com>
This patch adds support for the "BITMAP" sync mode to drive-mirror and
blockdev-mirror. It adds support only for the BitmapSyncMode "never,"
because it's the simplest mode.
This mode simply uses a user-provided bitmap as an initial copy
manifest, and then does not clear any bits in the bitmap at the
conclusion of the operation.
Any new writes dirtied during the operation are copied out, in contrast
to backup. Note that whether these writes are reflected in the bitmap
at the conclusion of the operation depends on whether that bitmap is
actually recording!
This patch was originally based on one by Ma Haocong, but it has since
been modified pretty heavily.
Suggested-by: Ma Haocong <mahaocong@didichuxing.com>
Signed-off-by: Ma Haocong <mahaocong@didichuxing.com>
Signed-off-by: John Snow <jsnow@redhat.com>
[FG: switch to bdrv_dirty_bitmap_merge_internal]
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
[FE: rebase for 9.0
update version and formatting in QAPI]
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
block/mirror.c | 96 ++++++++++++++++++++------
blockdev.c | 38 +++++++++-
include/block/block_int-global-state.h | 4 +-
qapi/block-core.json | 25 ++++++-
tests/unit/test-block-iothread.c | 4 +-
5 files changed, 139 insertions(+), 28 deletions(-)
diff --git a/block/mirror.c b/block/mirror.c
index 5145eb53e1..315dff11e2 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -51,7 +51,7 @@ typedef struct MirrorBlockJob {
BlockDriverState *to_replace;
/* Used to block operations on the drive-mirror-replace target */
Error *replace_blocker;
- bool is_none_mode;
+ MirrorSyncMode sync_mode;
BlockMirrorBackingMode backing_mode;
/* Whether the target image requires explicit zero-initialization */
bool zero_target;
@@ -73,6 +73,8 @@ typedef struct MirrorBlockJob {
size_t buf_size;
int64_t bdev_length;
unsigned long *cow_bitmap;
+ BdrvDirtyBitmap *sync_bitmap;
+ BitmapSyncMode bitmap_mode;
BdrvDirtyBitmap *dirty_bitmap;
BdrvDirtyBitmapIter *dbi;
uint8_t *buf;
@@ -718,7 +720,8 @@ static int mirror_exit_common(Job *job)
&error_abort);
if (!abort && s->backing_mode == MIRROR_SOURCE_BACKING_CHAIN) {
- BlockDriverState *backing = s->is_none_mode ? src : s->base;
+ BlockDriverState *backing;
+ backing = s->sync_mode == MIRROR_SYNC_MODE_NONE ? src : s->base;
BlockDriverState *unfiltered_target = bdrv_skip_filters(target_bs);
if (bdrv_cow_bs(unfiltered_target) != backing) {
@@ -815,6 +818,16 @@ static void mirror_abort(Job *job)
assert(ret == 0);
}
+/* Always called after commit/abort. */
+static void mirror_clean(Job *job)
+{
+ MirrorBlockJob *s = container_of(job, MirrorBlockJob, common.job);
+
+ if (s->sync_bitmap) {
+ bdrv_dirty_bitmap_set_busy(s->sync_bitmap, false);
+ }
+}
+
static void coroutine_fn mirror_throttle(MirrorBlockJob *s)
{
int64_t now = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
@@ -1011,7 +1024,8 @@ static int coroutine_fn mirror_run(Job *job, Error **errp)
mirror_free_init(s);
s->last_pause_ns = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
- if (!s->is_none_mode) {
+ if ((s->sync_mode == MIRROR_SYNC_MODE_TOP) ||
+ (s->sync_mode == MIRROR_SYNC_MODE_FULL)) {
ret = mirror_dirty_init(s);
if (ret < 0 || job_is_cancelled(&s->common.job)) {
goto immediate_exit;
@@ -1302,6 +1316,7 @@ static const BlockJobDriver mirror_job_driver = {
.run = mirror_run,
.prepare = mirror_prepare,
.abort = mirror_abort,
+ .clean = mirror_clean,
.pause = mirror_pause,
.complete = mirror_complete,
.cancel = mirror_cancel,
@@ -1320,6 +1335,7 @@ static const BlockJobDriver commit_active_job_driver = {
.run = mirror_run,
.prepare = mirror_prepare,
.abort = mirror_abort,
+ .clean = mirror_clean,
.pause = mirror_pause,
.complete = mirror_complete,
.cancel = commit_active_cancel,
@@ -1712,7 +1728,10 @@ static BlockJob *mirror_start_job(
BlockCompletionFunc *cb,
void *opaque,
const BlockJobDriver *driver,
- bool is_none_mode, BlockDriverState *base,
+ MirrorSyncMode sync_mode,
+ BdrvDirtyBitmap *bitmap,
+ BitmapSyncMode bitmap_mode,
+ BlockDriverState *base,
bool auto_complete, const char *filter_node_name,
bool is_mirror, MirrorCopyMode copy_mode,
Error **errp)
@@ -1726,10 +1745,39 @@ static BlockJob *mirror_start_job(
GLOBAL_STATE_CODE();
- if (granularity == 0) {
- granularity = bdrv_get_default_bitmap_granularity(target);
+ if (sync_mode == MIRROR_SYNC_MODE_INCREMENTAL) {
+ error_setg(errp, "Sync mode '%s' not supported",
+ MirrorSyncMode_str(sync_mode));
+ return NULL;
+ } else if (sync_mode == MIRROR_SYNC_MODE_BITMAP) {
+ if (!bitmap) {
+ error_setg(errp, "Must provide a valid bitmap name for '%s'"
+ " sync mode",
+ MirrorSyncMode_str(sync_mode));
+ return NULL;
+ } else if (bitmap_mode != BITMAP_SYNC_MODE_NEVER) {
+ error_setg(errp,
+ "Bitmap Sync Mode '%s' is not supported by Mirror",
+ BitmapSyncMode_str(bitmap_mode));
+ }
+ } else if (bitmap) {
+ error_setg(errp,
+ "sync mode '%s' is not compatible with bitmaps",
+ MirrorSyncMode_str(sync_mode));
+ return NULL;
}
+ if (bitmap) {
+ if (granularity) {
+ error_setg(errp, "granularity (%d)"
+ "cannot be specified when a bitmap is provided",
+ granularity);
+ return NULL;
+ }
+ granularity = bdrv_dirty_bitmap_granularity(bitmap);
+ } else if (granularity == 0) {
+ granularity = bdrv_get_default_bitmap_granularity(target);
+ }
assert(is_power_of_2(granularity));
if (buf_size < 0) {
@@ -1869,7 +1917,9 @@ static BlockJob *mirror_start_job(
s->replaces = g_strdup(replaces);
s->on_source_error = on_source_error;
s->on_target_error = on_target_error;
- s->is_none_mode = is_none_mode;
+ s->sync_mode = sync_mode;
+ s->sync_bitmap = bitmap;
+ s->bitmap_mode = bitmap_mode;
s->backing_mode = backing_mode;
s->zero_target = zero_target;
qatomic_set(&s->copy_mode, copy_mode);
@@ -1895,6 +1945,15 @@ static BlockJob *mirror_start_job(
*/
bdrv_disable_dirty_bitmap(s->dirty_bitmap);
+ if (s->sync_bitmap) {
+ bdrv_dirty_bitmap_set_busy(s->sync_bitmap, true);
+ }
+
+ if (s->sync_mode == MIRROR_SYNC_MODE_BITMAP) {
+ bdrv_dirty_bitmap_merge_internal(s->dirty_bitmap, s->sync_bitmap,
+ NULL, true);
+ }
+
bdrv_graph_wrlock();
ret = block_job_add_bdrv(&s->common, "source", bs, 0,
BLK_PERM_WRITE_UNCHANGED | BLK_PERM_WRITE |
@@ -1977,6 +2036,9 @@ fail:
if (s->dirty_bitmap) {
bdrv_release_dirty_bitmap(s->dirty_bitmap);
}
+ if (s->sync_bitmap) {
+ bdrv_dirty_bitmap_set_busy(s->sync_bitmap, false);
+ }
job_early_fail(&s->common.job);
}
@@ -1999,35 +2061,28 @@ void mirror_start(const char *job_id, BlockDriverState *bs,
BlockDriverState *target, const char *replaces,
int creation_flags, int64_t speed,
uint32_t granularity, int64_t buf_size,
- MirrorSyncMode mode, BlockMirrorBackingMode backing_mode,
+ MirrorSyncMode mode, BdrvDirtyBitmap *bitmap,
+ BitmapSyncMode bitmap_mode,
+ BlockMirrorBackingMode backing_mode,
bool zero_target,
BlockdevOnError on_source_error,
BlockdevOnError on_target_error,
bool unmap, const char *filter_node_name,
MirrorCopyMode copy_mode, Error **errp)
{
- bool is_none_mode;
BlockDriverState *base;
GLOBAL_STATE_CODE();
- if ((mode == MIRROR_SYNC_MODE_INCREMENTAL) ||
- (mode == MIRROR_SYNC_MODE_BITMAP)) {
- error_setg(errp, "Sync mode '%s' not supported",
- MirrorSyncMode_str(mode));
- return;
- }
-
bdrv_graph_rdlock_main_loop();
- is_none_mode = mode == MIRROR_SYNC_MODE_NONE;
base = mode == MIRROR_SYNC_MODE_TOP ? bdrv_backing_chain_next(bs) : NULL;
bdrv_graph_rdunlock_main_loop();
mirror_start_job(job_id, bs, creation_flags, target, replaces,
speed, granularity, buf_size, backing_mode, zero_target,
on_source_error, on_target_error, unmap, NULL, NULL,
- &mirror_job_driver, is_none_mode, base, false,
- filter_node_name, true, copy_mode, errp);
+ &mirror_job_driver, mode, bitmap, bitmap_mode, base,
+ false, filter_node_name, true, copy_mode, errp);
}
BlockJob *commit_active_start(const char *job_id, BlockDriverState *bs,
@@ -2054,7 +2109,8 @@ BlockJob *commit_active_start(const char *job_id, BlockDriverState *bs,
job_id, bs, creation_flags, base, NULL, speed, 0, 0,
MIRROR_LEAVE_BACKING_CHAIN, false,
on_error, on_error, true, cb, opaque,
- &commit_active_job_driver, false, base, auto_complete,
+ &commit_active_job_driver, MIRROR_SYNC_MODE_FULL,
+ NULL, 0, base, auto_complete,
filter_node_name, false, MIRROR_COPY_MODE_BACKGROUND,
errp);
if (!job) {
diff --git a/blockdev.c b/blockdev.c
index f8bb0932f8..c65d9ded70 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2776,6 +2776,9 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
BlockDriverState *target,
const char *replaces,
enum MirrorSyncMode sync,
+ const char *bitmap_name,
+ bool has_bitmap_mode,
+ BitmapSyncMode bitmap_mode,
BlockMirrorBackingMode backing_mode,
bool zero_target,
bool has_speed, int64_t speed,
@@ -2794,6 +2797,7 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
{
BlockDriverState *unfiltered_bs;
int job_flags = JOB_DEFAULT;
+ BdrvDirtyBitmap *bitmap = NULL;
GLOBAL_STATE_CODE();
GRAPH_RDLOCK_GUARD_MAINLOOP();
@@ -2848,6 +2852,29 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
sync = MIRROR_SYNC_MODE_FULL;
}
+ if (bitmap_name) {
+ if (granularity) {
+ error_setg(errp, "Granularity and bitmap cannot both be set");
+ return;
+ }
+
+ if (!has_bitmap_mode) {
+ error_setg(errp, "bitmap-mode must be specified if"
+ " a bitmap is provided");
+ return;
+ }
+
+ bitmap = bdrv_find_dirty_bitmap(bs, bitmap_name);
+ if (!bitmap) {
+ error_setg(errp, "Dirty bitmap '%s' not found", bitmap_name);
+ return;
+ }
+
+ if (bdrv_dirty_bitmap_check(bitmap, BDRV_BITMAP_ALLOW_RO, errp)) {
+ return;
+ }
+ }
+
if (!replaces) {
/* We want to mirror from @bs, but keep implicit filters on top */
unfiltered_bs = bdrv_skip_implicit_filters(bs);
@@ -2889,8 +2916,8 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
* and will allow to check whether the node still exist at mirror completion
*/
mirror_start(job_id, bs, target,
- replaces, job_flags,
- speed, granularity, buf_size, sync, backing_mode, zero_target,
+ replaces, job_flags, speed, granularity, buf_size, sync,
+ bitmap, bitmap_mode, backing_mode, zero_target,
on_source_error, on_target_error, unmap, filter_node_name,
copy_mode, errp);
}
@@ -3034,6 +3061,8 @@ void qmp_drive_mirror(DriveMirror *arg, Error **errp)
blockdev_mirror_common(arg->job_id, bs, target_bs,
arg->replaces, arg->sync,
+ arg->bitmap,
+ arg->has_bitmap_mode, arg->bitmap_mode,
backing_mode, zero_target,
arg->has_speed, arg->speed,
arg->has_granularity, arg->granularity,
@@ -3053,6 +3082,8 @@ void qmp_blockdev_mirror(const char *job_id,
const char *device, const char *target,
const char *replaces,
MirrorSyncMode sync,
+ const char *bitmap,
+ bool has_bitmap_mode, BitmapSyncMode bitmap_mode,
bool has_speed, int64_t speed,
bool has_granularity, uint32_t granularity,
bool has_buf_size, int64_t buf_size,
@@ -3093,7 +3124,8 @@ void qmp_blockdev_mirror(const char *job_id,
}
blockdev_mirror_common(job_id, bs, target_bs,
- replaces, sync, backing_mode,
+ replaces, sync,
+ bitmap, has_bitmap_mode, bitmap_mode, backing_mode,
zero_target, has_speed, speed,
has_granularity, granularity,
has_buf_size, buf_size,
diff --git a/include/block/block_int-global-state.h b/include/block/block_int-global-state.h
index d2201e27f4..cc1387ae02 100644
--- a/include/block/block_int-global-state.h
+++ b/include/block/block_int-global-state.h
@@ -158,7 +158,9 @@ void mirror_start(const char *job_id, BlockDriverState *bs,
BlockDriverState *target, const char *replaces,
int creation_flags, int64_t speed,
uint32_t granularity, int64_t buf_size,
- MirrorSyncMode mode, BlockMirrorBackingMode backing_mode,
+ MirrorSyncMode mode, BdrvDirtyBitmap *bitmap,
+ BitmapSyncMode bitmap_mode,
+ BlockMirrorBackingMode backing_mode,
bool zero_target,
BlockdevOnError on_source_error,
BlockdevOnError on_target_error,
diff --git a/qapi/block-core.json b/qapi/block-core.json
index ab5a93a966..ac05483958 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2181,6 +2181,15 @@
# destination (all the disk, only the sectors allocated in the
# topmost image, or only new I/O).
#
+# @bitmap: The name of a bitmap to use for sync=bitmap mode. This
+# argument must be present for bitmap mode and absent otherwise.
+# The bitmap's granularity is used instead of @granularity.
+# (Since 9.0).
+#
+# @bitmap-mode: Specifies the type of data the bitmap should contain
+# after the operation concludes. Must be present if sync is
+# "bitmap". Must NOT be present otherwise. (Since 9.0)
+#
# @granularity: granularity of the dirty bitmap, default is 64K if the
# image format doesn't have clusters, 4K if the clusters are
# smaller than that, else the cluster size. Must be a power of 2
@@ -2223,7 +2232,9 @@
{ 'struct': 'DriveMirror',
'data': { '*job-id': 'str', 'device': 'str', 'target': 'str',
'*format': 'str', '*node-name': 'str', '*replaces': 'str',
- 'sync': 'MirrorSyncMode', '*mode': 'NewImageMode',
+ 'sync': 'MirrorSyncMode', '*bitmap': 'str',
+ '*bitmap-mode': 'BitmapSyncMode',
+ '*mode': 'NewImageMode',
'*speed': 'int', '*granularity': 'uint32',
'*buf-size': 'int', '*on-source-error': 'BlockdevOnError',
'*on-target-error': 'BlockdevOnError',
@@ -2507,6 +2518,15 @@
# destination (all the disk, only the sectors allocated in the
# topmost image, or only new I/O).
#
+# @bitmap: The name of a bitmap to use for sync=bitmap mode. This
+# argument must be present for bitmap mode and absent otherwise.
+# The bitmap's granularity is used instead of @granularity.
+# (Since 9.0).
+#
+# @bitmap-mode: Specifies the type of data the bitmap should contain
+# after the operation concludes. Must be present if sync is
+# "bitmap". Must NOT be present otherwise. (Since 9.0)
+#
# @granularity: granularity of the dirty bitmap, default is 64K if the
# image format doesn't have clusters, 4K if the clusters are
# smaller than that, else the cluster size. Must be a power of 2
@@ -2557,7 +2577,8 @@
{ 'command': 'blockdev-mirror',
'data': { '*job-id': 'str', 'device': 'str', 'target': 'str',
'*replaces': 'str',
- 'sync': 'MirrorSyncMode',
+ 'sync': 'MirrorSyncMode', '*bitmap': 'str',
+ '*bitmap-mode': 'BitmapSyncMode',
'*speed': 'int', '*granularity': 'uint32',
'*buf-size': 'int', '*on-source-error': 'BlockdevOnError',
'*on-target-error': 'BlockdevOnError',
diff --git a/tests/unit/test-block-iothread.c b/tests/unit/test-block-iothread.c
index 3766d5de6b..afa44cbd34 100644
--- a/tests/unit/test-block-iothread.c
+++ b/tests/unit/test-block-iothread.c
@@ -755,8 +755,8 @@ static void test_propagate_mirror(void)
/* Start a mirror job */
mirror_start("job0", src, target, NULL, JOB_DEFAULT, 0, 0, 0,
- MIRROR_SYNC_MODE_NONE, MIRROR_OPEN_BACKING_CHAIN, false,
- BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT,
+ MIRROR_SYNC_MODE_NONE, NULL, 0, MIRROR_OPEN_BACKING_CHAIN,
+ false, BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT,
false, "filter_node", MIRROR_COPY_MODE_BACKGROUND,
&error_abort);
--
2.39.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [RFC 2/4] drive-mirror: add support for conditional and always bitmap sync modes
2024-02-16 10:55 [RFC 0/4] mirror: implement incremental and bitmap modes Fiona Ebner
2024-02-16 10:55 ` [RFC 1/4] drive-mirror: add support for sync=bitmap mode=never Fiona Ebner
@ 2024-02-16 10:55 ` Fiona Ebner
2024-02-16 10:55 ` [RFC 3/4] mirror: move some checks to qmp Fiona Ebner
` (3 subsequent siblings)
5 siblings, 0 replies; 25+ messages in thread
From: Fiona Ebner @ 2024-02-16 10:55 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, vsementsov, jsnow,
f.gruenbichler, t.lamprecht, mahaocong
From: John Snow <jsnow@redhat.com>
Teach mirror two new tricks for using bitmaps:
Always: no matter what, we synchronize the copy_bitmap back to the
sync_bitmap. In effect, this allows us resume a failed mirror at a later
date.
Conditional: On success only, we sync the bitmap. This is akin to
incremental backup modes; we can use this bitmap to later refresh a
successfully created mirror.
Originally-by: John Snow <jsnow@redhat.com>
[FG: add check for bitmap-mode without bitmap
switch to bdrv_dirty_bitmap_merge_internal]
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
[FE: rebase for 9.0]
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
The original patch this was based on came from a WIP git branch and
thus has no Signed-off-by trailer from John, see [0]. I added an
Originally-by trailer for now. Let me know if I should drop that and
wait for John's Signed-off-by instead.
[0] https://lore.kernel.org/qemu-devel/1599140071.n44h532eeu.astroid@nora.none/
block/mirror.c | 24 ++++++++++++++++++------
blockdev.c | 3 +++
2 files changed, 21 insertions(+), 6 deletions(-)
diff --git a/block/mirror.c b/block/mirror.c
index 315dff11e2..84155b1f78 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -689,8 +689,6 @@ static int mirror_exit_common(Job *job)
bdrv_unfreeze_backing_chain(mirror_top_bs, target_bs);
}
- bdrv_release_dirty_bitmap(s->dirty_bitmap);
-
/* Make sure that the source BDS doesn't go away during bdrv_replace_node,
* before we can call bdrv_drained_end */
bdrv_ref(src);
@@ -796,6 +794,18 @@ static int mirror_exit_common(Job *job)
bdrv_drained_end(target_bs);
bdrv_unref(target_bs);
+ if (s->sync_bitmap) {
+ if (s->bitmap_mode == BITMAP_SYNC_MODE_ALWAYS ||
+ (s->bitmap_mode == BITMAP_SYNC_MODE_ON_SUCCESS &&
+ job->ret == 0 && ret == 0)) {
+ /* Success; synchronize copy back to sync. */
+ bdrv_clear_dirty_bitmap(s->sync_bitmap, NULL);
+ bdrv_dirty_bitmap_merge_internal(s->sync_bitmap, s->dirty_bitmap,
+ NULL, true);
+ }
+ }
+ bdrv_release_dirty_bitmap(s->dirty_bitmap);
+
bs_opaque->job = NULL;
bdrv_drained_end(src);
@@ -1755,10 +1765,6 @@ static BlockJob *mirror_start_job(
" sync mode",
MirrorSyncMode_str(sync_mode));
return NULL;
- } else if (bitmap_mode != BITMAP_SYNC_MODE_NEVER) {
- error_setg(errp,
- "Bitmap Sync Mode '%s' is not supported by Mirror",
- BitmapSyncMode_str(bitmap_mode));
}
} else if (bitmap) {
error_setg(errp,
@@ -1775,6 +1781,12 @@ static BlockJob *mirror_start_job(
return NULL;
}
granularity = bdrv_dirty_bitmap_granularity(bitmap);
+
+ if (bitmap_mode != BITMAP_SYNC_MODE_NEVER) {
+ if (bdrv_dirty_bitmap_check(bitmap, BDRV_BITMAP_DEFAULT, errp)) {
+ return NULL;
+ }
+ }
} else if (granularity == 0) {
granularity = bdrv_get_default_bitmap_granularity(target);
}
diff --git a/blockdev.c b/blockdev.c
index c65d9ded70..aeb9fde9f3 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2873,6 +2873,9 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
if (bdrv_dirty_bitmap_check(bitmap, BDRV_BITMAP_ALLOW_RO, errp)) {
return;
}
+ } else if (has_bitmap_mode) {
+ error_setg(errp, "Cannot specify bitmap sync mode without a bitmap");
+ return;
}
if (!replaces) {
--
2.39.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [RFC 3/4] mirror: move some checks to qmp
2024-02-16 10:55 [RFC 0/4] mirror: implement incremental and bitmap modes Fiona Ebner
2024-02-16 10:55 ` [RFC 1/4] drive-mirror: add support for sync=bitmap mode=never Fiona Ebner
2024-02-16 10:55 ` [RFC 2/4] drive-mirror: add support for conditional and always bitmap sync modes Fiona Ebner
@ 2024-02-16 10:55 ` Fiona Ebner
2024-02-16 10:55 ` [RFC 4/4] iotests: add test for bitmap mirror Fiona Ebner
` (2 subsequent siblings)
5 siblings, 0 replies; 25+ messages in thread
From: Fiona Ebner @ 2024-02-16 10:55 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, vsementsov, jsnow,
f.gruenbichler, t.lamprecht, mahaocong
From: Fabian Grünbichler <f.gruenbichler@proxmox.com>
and assert the passing conditions in block/mirror.c. while incremental
mode was never available for drive-mirror, it makes the interface more
uniform w.r.t. backup block jobs.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
[FE: rebase for 9.0]
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
block/mirror.c | 28 +++++-----------------------
blockdev.c | 29 +++++++++++++++++++++++++++++
2 files changed, 34 insertions(+), 23 deletions(-)
diff --git a/block/mirror.c b/block/mirror.c
index 84155b1f78..15d1c060eb 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -1755,31 +1755,13 @@ static BlockJob *mirror_start_job(
GLOBAL_STATE_CODE();
- if (sync_mode == MIRROR_SYNC_MODE_INCREMENTAL) {
- error_setg(errp, "Sync mode '%s' not supported",
- MirrorSyncMode_str(sync_mode));
- return NULL;
- } else if (sync_mode == MIRROR_SYNC_MODE_BITMAP) {
- if (!bitmap) {
- error_setg(errp, "Must provide a valid bitmap name for '%s'"
- " sync mode",
- MirrorSyncMode_str(sync_mode));
- return NULL;
- }
- } else if (bitmap) {
- error_setg(errp,
- "sync mode '%s' is not compatible with bitmaps",
- MirrorSyncMode_str(sync_mode));
- return NULL;
- }
+ /* QMP interface protects us from these cases */
+ assert(sync_mode != MIRROR_SYNC_MODE_INCREMENTAL);
+ assert((bitmap && sync_mode == MIRROR_SYNC_MODE_BITMAP) ||
+ (!bitmap && sync_mode != MIRROR_SYNC_MODE_BITMAP));
+ assert(!(bitmap && granularity));
if (bitmap) {
- if (granularity) {
- error_setg(errp, "granularity (%d)"
- "cannot be specified when a bitmap is provided",
- granularity);
- return NULL;
- }
granularity = bdrv_dirty_bitmap_granularity(bitmap);
if (bitmap_mode != BITMAP_SYNC_MODE_NEVER) {
diff --git a/blockdev.c b/blockdev.c
index aeb9fde9f3..519f408359 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2852,7 +2852,36 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
sync = MIRROR_SYNC_MODE_FULL;
}
+ if ((sync == MIRROR_SYNC_MODE_BITMAP) ||
+ (sync == MIRROR_SYNC_MODE_INCREMENTAL)) {
+ /* done before desugaring 'incremental' to print the right message */
+ if (!bitmap_name) {
+ error_setg(errp, "Must provide a valid bitmap name for "
+ "'%s' sync mode", MirrorSyncMode_str(sync));
+ return;
+ }
+ }
+
+ if (sync == MIRROR_SYNC_MODE_INCREMENTAL) {
+ if (has_bitmap_mode &&
+ bitmap_mode != BITMAP_SYNC_MODE_ON_SUCCESS) {
+ error_setg(errp, "Bitmap sync mode must be '%s' "
+ "when using sync mode '%s'",
+ BitmapSyncMode_str(BITMAP_SYNC_MODE_ON_SUCCESS),
+ MirrorSyncMode_str(sync));
+ return;
+ }
+ has_bitmap_mode = true;
+ sync = MIRROR_SYNC_MODE_BITMAP;
+ bitmap_mode = BITMAP_SYNC_MODE_ON_SUCCESS;
+ }
+
if (bitmap_name) {
+ if (sync != MIRROR_SYNC_MODE_BITMAP) {
+ error_setg(errp, "Sync mode '%s' not supported with bitmap.",
+ MirrorSyncMode_str(sync));
+ return;
+ }
if (granularity) {
error_setg(errp, "Granularity and bitmap cannot both be set");
return;
--
2.39.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [RFC 4/4] iotests: add test for bitmap mirror
2024-02-16 10:55 [RFC 0/4] mirror: implement incremental and bitmap modes Fiona Ebner
` (2 preceding siblings ...)
2024-02-16 10:55 ` [RFC 3/4] mirror: move some checks to qmp Fiona Ebner
@ 2024-02-16 10:55 ` Fiona Ebner
2024-02-28 16:00 ` [RFC 0/4] mirror: implement incremental and bitmap modes Vladimir Sementsov-Ogievskiy
2024-02-28 16:24 ` Vladimir Sementsov-Ogievskiy
5 siblings, 0 replies; 25+ messages in thread
From: Fiona Ebner @ 2024-02-16 10:55 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, vsementsov, jsnow,
f.gruenbichler, t.lamprecht, mahaocong
From: Fabian Grünbichler <f.gruenbichler@proxmox.com>
heavily based on/practically forked off iotest 257 for bitmap backups,
but:
- no writes to filter node 'mirror-top' between completion and
finalization, as those seem to deadlock?
- no inclusion of not-yet-available full/top sync modes in combination
with bitmaps
- extra set of reference/test mirrors to verify that writes in parallel
with active mirror work
intentionally keeping copyright and ownership of original test case to
honor provenance.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
[FE: rebase for 9.0, i.e. adapt to renames like vm.command -> vm.cmd,
specifying explicit image format for rebase,
adapt to new behavior of qemu_img(),
dropping of 'status' field in output, etc.
rename test from '384' to 'bitmap-sync-mirror']
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
tests/qemu-iotests/tests/bitmap-sync-mirror | 550 ++++
.../qemu-iotests/tests/bitmap-sync-mirror.out | 2810 +++++++++++++++++
2 files changed, 3360 insertions(+)
create mode 100755 tests/qemu-iotests/tests/bitmap-sync-mirror
create mode 100644 tests/qemu-iotests/tests/bitmap-sync-mirror.out
diff --git a/tests/qemu-iotests/tests/bitmap-sync-mirror b/tests/qemu-iotests/tests/bitmap-sync-mirror
new file mode 100755
index 0000000000..6cd9b74dac
--- /dev/null
+++ b/tests/qemu-iotests/tests/bitmap-sync-mirror
@@ -0,0 +1,550 @@
+#!/usr/bin/env python3
+# group: rw
+#
+# Test bitmap-sync mirrors (incremental, differential, and partials)
+#
+# Copyright (c) 2019 John Snow for Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+# owner=jsnow@redhat.com
+
+import math
+import os
+
+import iotests
+from iotests import log, qemu_img
+
+SIZE = 64 * 1024 * 1024
+GRANULARITY = 64 * 1024
+
+
+class Pattern:
+ def __init__(self, byte, offset, size=GRANULARITY):
+ self.byte = byte
+ self.offset = offset
+ self.size = size
+
+ def bits(self, granularity):
+ lower = self.offset // granularity
+ upper = (self.offset + self.size - 1) // granularity
+ return set(range(lower, upper + 1))
+
+
+class PatternGroup:
+ """Grouping of Pattern objects. Initialize with an iterable of Patterns."""
+ def __init__(self, patterns):
+ self.patterns = patterns
+
+ def bits(self, granularity):
+ """Calculate the unique bits dirtied by this pattern grouping"""
+ res = set()
+ for pattern in self.patterns:
+ res |= pattern.bits(granularity)
+ return res
+
+
+GROUPS = [
+ PatternGroup([
+ # Batch 0: 4 clusters
+ Pattern('0x49', 0x0000000),
+ Pattern('0x6c', 0x0100000), # 1M
+ Pattern('0x6f', 0x2000000), # 32M
+ Pattern('0x76', 0x3ff0000)]), # 64M - 64K
+ PatternGroup([
+ # Batch 1: 6 clusters (3 new)
+ Pattern('0x65', 0x0000000), # Full overwrite
+ Pattern('0x77', 0x00f8000), # Partial-left (1M-32K)
+ Pattern('0x72', 0x2008000), # Partial-right (32M+32K)
+ Pattern('0x69', 0x3fe0000)]), # Adjacent-left (64M - 128K)
+ PatternGroup([
+ # Batch 2: 7 clusters (3 new)
+ Pattern('0x74', 0x0010000), # Adjacent-right
+ Pattern('0x69', 0x00e8000), # Partial-left (1M-96K)
+ Pattern('0x6e', 0x2018000), # Partial-right (32M+96K)
+ Pattern('0x67', 0x3fe0000,
+ 2*GRANULARITY)]), # Overwrite [(64M-128K)-64M)
+ PatternGroup([
+ # Batch 3: 8 clusters (5 new)
+ # Carefully chosen such that nothing re-dirties the one cluster
+ # that copies out successfully before failure in Group #1.
+ Pattern('0xaa', 0x0010000,
+ 3*GRANULARITY), # Overwrite and 2x Adjacent-right
+ Pattern('0xbb', 0x00d8000), # Partial-left (1M-160K)
+ Pattern('0xcc', 0x2028000), # Partial-right (32M+160K)
+ Pattern('0xdd', 0x3fc0000)]), # New; leaving a gap to the right
+]
+
+
+class EmulatedBitmap:
+ def __init__(self, granularity=GRANULARITY):
+ self._bits = set()
+ self.granularity = granularity
+
+ def dirty_bits(self, bits):
+ self._bits |= set(bits)
+
+ def dirty_group(self, n):
+ self.dirty_bits(GROUPS[n].bits(self.granularity))
+
+ def clear(self):
+ self._bits = set()
+
+ def clear_bits(self, bits):
+ self._bits -= set(bits)
+
+ def clear_bit(self, bit):
+ self.clear_bits({bit})
+
+ def clear_group(self, n):
+ self.clear_bits(GROUPS[n].bits(self.granularity))
+
+ @property
+ def first_bit(self):
+ return sorted(self.bits)[0]
+
+ @property
+ def bits(self):
+ return self._bits
+
+ @property
+ def count(self):
+ return len(self.bits)
+
+ def compare(self, qmp_bitmap):
+ """
+ Print a nice human-readable message checking that a bitmap as reported
+ by the QMP interface has as many bits set as we expect it to.
+ """
+
+ name = qmp_bitmap.get('name', '(anonymous)')
+ log("= Checking Bitmap {:s} =".format(name))
+
+ want = self.count
+ have = qmp_bitmap['count'] // qmp_bitmap['granularity']
+
+ log("expecting {:d} dirty sectors; have {:d}. {:s}".format(
+ want, have, "OK!" if want == have else "ERROR!"))
+ log('')
+
+
+class Drive:
+ """Represents, vaguely, a drive attached to a VM.
+ Includes format, graph, and device information."""
+
+ def __init__(self, path, vm=None):
+ self.path = path
+ self.vm = vm
+ self.fmt = None
+ self.size = None
+ self.node = None
+
+ def img_create(self, fmt, size):
+ self.fmt = fmt
+ self.size = size
+ iotests.qemu_img_create('-f', self.fmt, self.path, str(self.size))
+
+ def create_target(self, name, fmt, size):
+ basename = os.path.basename(self.path)
+ file_node_name = "file_{}".format(basename)
+ vm = self.vm
+
+ log(vm.cmd('blockdev-create', job_id='bdc-file-job',
+ options={
+ 'driver': 'file',
+ 'filename': self.path,
+ 'size': 0,
+ }))
+ vm.run_job('bdc-file-job')
+ log(vm.cmd('blockdev-add', driver='file',
+ node_name=file_node_name, filename=self.path))
+
+ log(vm.cmd('blockdev-create', job_id='bdc-fmt-job',
+ options={
+ 'driver': fmt,
+ 'file': file_node_name,
+ 'size': size,
+ }))
+ vm.run_job('bdc-fmt-job')
+ log(vm.cmd('blockdev-add', driver=fmt,
+ node_name=name,
+ file=file_node_name))
+ self.fmt = fmt
+ self.size = size
+ self.node = name
+
+def blockdev_mirror(vm, device, target, sync, **kwargs):
+ # Strip any arguments explicitly nulled by the caller:
+ kwargs = {key: val for key, val in kwargs.items() if val is not None}
+ result = vm.qmp_log('blockdev-mirror',
+ device=device,
+ target=target,
+ sync=sync,
+ filter_node_name='mirror-top',
+ **kwargs)
+ return result
+
+def blockdev_mirror_mktarget(drive, target_id, filepath, sync, **kwargs):
+ target_drive = Drive(filepath, vm=drive.vm)
+ target_drive.create_target(target_id, drive.fmt, drive.size)
+ blockdev_mirror(drive.vm, drive.node, target_id, sync, **kwargs)
+
+def reference_mirror(drive, n, filepath):
+ log("--- Reference mirror #{:d} ---\n".format(n))
+ target_id = "ref_target_{:d}".format(n)
+ job_id = "ref_mirror_{:d}".format(n)
+ blockdev_mirror_mktarget(drive, target_id, filepath, "full",
+ job_id=job_id)
+ drive.vm.run_job(job_id, auto_dismiss=True)
+ log('')
+
+def mirror(drive, n, filepath, sync, **kwargs):
+ log("--- Test mirror #{:d} ---\n".format(n))
+ target_id = "mirror_target_{:d}".format(n)
+ job_id = "mirror_{:d}".format(n)
+ kwargs.setdefault('auto-finalize', False)
+ blockdev_mirror_mktarget(drive, target_id, filepath, sync,
+ job_id=job_id, **kwargs)
+ return job_id
+
+def perform_writes(drive, n, filter_node_name=None):
+ log("--- Write #{:d} ---\n".format(n))
+ for pattern in GROUPS[n].patterns:
+ cmd = "write -P{:s} 0x{:07x} 0x{:x}".format(
+ pattern.byte,
+ pattern.offset,
+ pattern.size)
+ log(cmd)
+ log(drive.vm.hmp_qemu_io(filter_node_name or drive.node, cmd))
+ bitmaps = drive.vm.query_bitmaps()
+ log({'bitmaps': bitmaps}, indent=2)
+ log('')
+ return bitmaps
+
+
+def compare_images(image, reference, baseimg=None, expected_match=True):
+ """
+ Print a nice human-readable message comparing these images.
+ """
+ expected_ret = 0 if expected_match else 1
+ if baseimg:
+ qemu_img("rebase", "-u", "-b", baseimg, '-F', iotests.imgfmt, image)
+
+ sub = qemu_img("compare", image, reference, check=False)
+
+ log('qemu_img compare "{:s}" "{:s}" ==> {:s}, {:s}'.format(
+ image, reference,
+ "Identical" if sub.returncode == 0 else "Mismatch",
+ "OK!" if sub.returncode == expected_ret else "ERROR!"),
+ filters=[iotests.filter_testfiles])
+
+def test_bitmap_sync(bsync_mode, msync_mode='bitmap', failure=None):
+ """
+ Test bitmap mirror routines.
+
+ :param bsync_mode: Is the Bitmap Sync mode, and can be any of:
+ - on-success: This is the "incremental" style mode. Bitmaps are
+ synchronized to what was copied out only on success.
+ (Partial images must be discarded.)
+ - never: This is the "differential" style mode.
+ Bitmaps are never synchronized.
+ - always: This is a "best effort" style mode.
+ Bitmaps are always synchronized, regardless of failure.
+ (Partial images must be kept.)
+
+ :param msync_mode: The mirror sync mode to use for the first mirror.
+ Can be any one of:
+ - bitmap: mirrors based on bitmap manifest.
+ - full: Full mirrors.
+ - top: Full mirrors of the top layer only.
+
+ :param failure: Is the (optional) failure mode, and can be any of:
+ - None: No failure. Test the normative path. Default.
+ - simulated: Cancel the job right before it completes.
+ This also tests writes "during" the job.
+ - intermediate: This tests a job that fails mid-process and produces
+ an incomplete mirror. Testing limitations prevent
+ testing competing writes.
+ """
+ with iotests.FilePath('img', 'bsync1', 'bsync2', 'bsync3',
+ 'fmirror0', 'fmirror1', 'fmirror2', 'fmirror3') as \
+ (img_path, bsync1, bsync2, bsync3,
+ fmirror0, fmirror1, fmirror2, fmirror3), \
+ iotests.VM() as vm:
+
+ mode = "Mode {:s}; Bitmap Sync {:s}".format(msync_mode, bsync_mode)
+ preposition = "with" if failure else "without"
+ cond = "{:s} {:s}".format(preposition,
+ "{:s} failure".format(failure) if failure
+ else "failure")
+ log("\n=== {:s} {:s} ===\n".format(mode, cond))
+
+ log('--- Preparing image & VM ---\n')
+ drive0 = Drive(img_path, vm=vm)
+ drive0.img_create(iotests.imgfmt, SIZE)
+ vm.add_device("virtio-scsi,id=scsi0")
+ vm.launch()
+
+ file_config = {
+ 'driver': 'file',
+ 'filename': drive0.path
+ }
+
+ if failure == 'intermediate':
+ file_config = {
+ 'driver': 'blkdebug',
+ 'image': file_config,
+ 'set-state': [{
+ 'event': 'flush_to_disk',
+ 'state': 1,
+ 'new_state': 2
+ }, {
+ 'event': 'read_aio',
+ 'state': 2,
+ 'new_state': 3
+ }, {
+ 'event': 'read_aio',
+ 'state': 3,
+ 'new_state': 4
+ }],
+ 'inject-error': [{
+ 'event': 'read_aio',
+ 'errno': 5,
+ 'state': 3,
+ 'immediately': False,
+ 'once': True
+ }, {
+ 'event': 'read_aio',
+ 'errno': 5,
+ 'state': 4,
+ 'immediately': False,
+ 'once': True
+ }]
+ }
+
+ drive0.node = 'drive0'
+ vm.qmp_log('blockdev-add',
+ filters=[iotests.filter_qmp_testfiles],
+ node_name=drive0.node,
+ driver=drive0.fmt,
+ file=file_config)
+ log('')
+
+ # 0 - Writes and Reference mirror
+ perform_writes(drive0, 0)
+ reference_mirror(drive0, 0, fmirror0)
+ log('--- Add Bitmap ---\n')
+ vm.qmp_log("block-dirty-bitmap-add", node=drive0.node,
+ name="bitmap0", granularity=GRANULARITY)
+ log('')
+ ebitmap = EmulatedBitmap()
+
+ # 1 - Writes and Reference mirror
+ bitmaps = perform_writes(drive0, 1)
+ ebitmap.dirty_group(1)
+ bitmap = vm.get_bitmap(drive0.node, 'bitmap0', bitmaps=bitmaps)
+ ebitmap.compare(bitmap)
+ reference_mirror(drive0, 1, fmirror1)
+
+ # 1 - Test mirror (w/ Optional induced failure)
+ if failure == 'intermediate':
+ # Activate blkdebug induced failure for second-to-next read
+ log(vm.hmp_qemu_io(drive0.node, 'flush'))
+ log('')
+ job = mirror(drive0, 1, bsync1, msync_mode,
+ bitmap="bitmap0", bitmap_mode=bsync_mode)
+
+ vm.run_job(job, auto_dismiss=True, auto_finalize=False,
+ cancel=(failure == 'simulated'))
+ bitmaps = vm.query_bitmaps()
+ log({'bitmaps': bitmaps}, indent=2)
+ log('')
+
+ if bsync_mode == 'always':
+ if failure == 'intermediate':
+ # We manage to copy one sector (one bit) before the error.
+ ebitmap.clear_bit(ebitmap.first_bit)
+ else:
+ # successful mirror / cancelled complete mirror
+ ebitmap.clear()
+
+ if bsync_mode == 'on-success' and not failure:
+ ebitmap.clear()
+
+ ebitmap.compare(vm.get_bitmap(drive0.node, 'bitmap0', bitmaps=bitmaps))
+
+ # 2 - Reference mirror
+ reference_mirror(drive0, 2, fmirror2)
+
+ # 2 - Bitmap mirror with writes before completion
+ job = mirror(drive0, 2, bsync2, "bitmap",
+ bitmap="bitmap0", bitmap_mode=bsync_mode)
+
+ bitmaps = perform_writes(drive0, 2, filter_node_name='mirror-top')
+ ebitmap.dirty_group(2)
+ ebitmap.compare(vm.get_bitmap(drive0.node, 'bitmap0', bitmaps=bitmaps))
+
+ # don't use run_job as that logs too much even with use_log=False
+ events = [('JOB_STATUS_CHANGE', {'data': {'id': job}})]
+ while True:
+ ev = iotests.filter_qmp_event(vm.events_wait(events, timeout=10))
+ status = ev['data']['status']
+ if status == 'ready':
+ vm.qmp('job-complete', id=job)
+ elif status == 'standby':
+ vm.qmp('job-resume', id=job)
+ elif status == 'pending':
+ vm.qmp('job-finalize', id=job)
+ elif status == 'null':
+ break
+
+ if bsync_mode != 'never':
+ ebitmap.clear()
+
+ bitmaps = vm.query_bitmaps()
+ ebitmap.compare(vm.get_bitmap(drive0.node, 'bitmap0', bitmaps=bitmaps))
+
+ # 3 - Writes and Reference mirror
+ bitmaps = perform_writes(drive0, 3)
+ ebitmap.dirty_group(3)
+ ebitmap.compare(vm.get_bitmap(drive0.node, 'bitmap0', bitmaps=bitmaps))
+ reference_mirror(drive0, 3, fmirror3)
+
+ # 3 - Bitmap mirror (In failure modes, this is a recovery.)
+ job = mirror(drive0, 3, bsync3, "bitmap",
+ bitmap="bitmap0", bitmap_mode=bsync_mode)
+
+ vm.run_job(job, auto_dismiss=True, auto_finalize=False)
+ bitmaps = vm.query_bitmaps()
+
+ log({'bitmaps': bitmaps}, indent=2)
+ log('')
+ if bsync_mode != 'never':
+ ebitmap.clear()
+ ebitmap.compare(vm.get_bitmap(drive0.node, 'bitmap0', bitmaps=bitmaps))
+
+ log('--- Cleanup ---\n')
+ vm.qmp_log("block-dirty-bitmap-remove",
+ node=drive0.node, name="bitmap0")
+ bitmaps = vm.query_bitmaps()
+ log({'bitmaps': bitmaps}, indent=2)
+ vm.shutdown()
+ log('')
+
+ log('--- Verification ---\n')
+ compare_images(bsync1, fmirror1, baseimg=fmirror0,
+ expected_match=failure != 'intermediate')
+ if not failure or bsync_mode == 'always':
+ # Always keep the last mirror on success or when using 'always'
+ base = bsync1
+ else:
+ base = fmirror1
+
+ compare_images(bsync2, fmirror2, baseimg=base, expected_match=0)
+ compare_images(bsync3, fmirror3, baseimg=bsync2)
+ compare_images(img_path, fmirror3)
+ log('')
+
+def test_mirror_api():
+ """
+ Test malformed and prohibited invocations of the mirror API.
+ """
+ with iotests.FilePath('img', 'bsync1') as \
+ (img_path, mirror_path), \
+ iotests.VM() as vm:
+
+ log("\n=== API failure tests ===\n")
+ log('--- Preparing image & VM ---\n')
+ drive0 = Drive(img_path, vm=vm)
+ drive0.img_create(iotests.imgfmt, SIZE)
+ vm.add_device("virtio-scsi,id=scsi0")
+ vm.launch()
+
+ file_config = {
+ 'driver': 'file',
+ 'filename': drive0.path
+ }
+
+ drive0.node = 'drive0'
+ vm.qmp_log('blockdev-add',
+ filters=[iotests.filter_qmp_testfiles],
+ node_name=drive0.node,
+ driver=drive0.fmt,
+ file=file_config)
+ log('')
+
+ target0 = Drive(mirror_path, vm=vm)
+ target0.create_target("mirror_target", drive0.fmt, drive0.size)
+ log('')
+
+ vm.qmp_log("block-dirty-bitmap-add", node=drive0.node,
+ name="bitmap0", granularity=GRANULARITY)
+ log('')
+
+ log('-- Testing invalid QMP commands --\n')
+
+ error_cases = {
+ 'incremental': {
+ None: ['on-success', 'always', 'never', None],
+ 'bitmap404': ['on-success', 'always', 'never', None],
+ 'bitmap0': ['always', 'never']
+ },
+ 'bitmap': {
+ None: ['on-success', 'always', 'never', None],
+ 'bitmap404': ['on-success', 'always', 'never', None],
+ 'bitmap0': [None],
+ },
+ 'full': {
+ None: ['on-success', 'always', 'never'],
+ 'bitmap404': ['on-success', 'always', 'never', None],
+ 'bitmap0': ['on-success', 'always', 'never', None],
+ },
+ 'top': {
+ None: ['on-success', 'always', 'never'],
+ 'bitmap404': ['on-success', 'always', 'never', None],
+ 'bitmap0': ['on-success', 'always', 'never', None],
+ },
+ 'none': {
+ None: ['on-success', 'always', 'never'],
+ 'bitmap404': ['on-success', 'always', 'never', None],
+ 'bitmap0': ['on-success', 'always', 'never', None],
+ }
+ }
+
+ # Dicts, as always, are not stably-ordered prior to 3.7, so use tuples:
+ for sync_mode in ('incremental', 'bitmap', 'full', 'top', 'none'):
+ log("-- Sync mode {:s} tests --\n".format(sync_mode))
+ for bitmap in (None, 'bitmap404', 'bitmap0'):
+ for policy in error_cases[sync_mode][bitmap]:
+ blockdev_mirror(drive0.vm, drive0.node, "mirror_target",
+ sync_mode, job_id='api_job',
+ bitmap=bitmap, bitmap_mode=policy)
+ log('')
+
+
+def main():
+ for bsync_mode in ("never", "on-success", "always"):
+ for failure in ("simulated", "intermediate", None):
+ test_bitmap_sync(bsync_mode, "bitmap", failure)
+
+# for sync_mode in ('full', 'top'):
+# for bsync_mode in ('on-success', 'always'):
+# for failure in ('simulated', 'intermediate', None):
+# test_bitmap_sync(bsync_mode, sync_mode, failure)
+
+ test_mirror_api()
+
+if __name__ == '__main__':
+ iotests.script_main(main, supported_fmts=['qcow2'],
+ supported_protocols=['file'])
diff --git a/tests/qemu-iotests/tests/bitmap-sync-mirror.out b/tests/qemu-iotests/tests/bitmap-sync-mirror.out
new file mode 100644
index 0000000000..96e8422fd3
--- /dev/null
+++ b/tests/qemu-iotests/tests/bitmap-sync-mirror.out
@@ -0,0 +1,2810 @@
+
+=== Mode bitmap; Bitmap Sync never with simulated failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {}
+}
+
+--- Reference mirror #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_0"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_1"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_1", "sync": "bitmap", "target": "mirror_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_1"}}
+{"return": {}}
+{"data": {"device": "mirror_1", "len": 393216, "offset": 393216, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-cancel", "arguments": {"id": "mirror_1"}}
+{"return": {}}
+{"data": {"id": "mirror_1", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_1", "len": 393216, "offset": 393216, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_2"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_2", "sync": "bitmap", "target": "mirror_target_2"}}
+{"return": {}}
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": true,
+ "count": 655360,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ],
+ "mirror-top": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": false
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 10 dirty sectors; have 10. OK!
+
+= Checking Bitmap bitmap0 =
+expecting 10 dirty sectors; have 10. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 983040,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 15 dirty sectors; have 15. OK!
+
+--- Reference mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_3", "sync": "full", "target": "ref_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_3"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_3", "sync": "bitmap", "target": "mirror_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"device": "mirror_3", "len": 983040, "offset": 983040, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"id": "mirror_3", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_3", "len": 983040, "offset": 983040, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 983040,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 15 dirty sectors; have 15. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {}
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fmirror1" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fmirror2" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync3" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+
+
+=== Mode bitmap; Bitmap Sync never with intermediate failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false, "once": true, "state": 3}, {"errno": 5, "event": "read_aio", "immediately": false, "once": true, "state": 4}], "set-state": [{"event": "flush_to_disk", "new-state": 2, "state": 1}, {"event": "read_aio", "new-state": 3, "state": 2}, {"event": "read_aio", "new-state": 4, "state": 3}]}, "node-name": "drive0"}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {}
+}
+
+--- Reference mirror #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_0"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_1"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+{"return": ""}
+
+--- Test mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_1", "sync": "bitmap", "target": "mirror_target_1"}}
+{"return": {}}
+{"data": {"action": "report", "device": "mirror_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"action": "report", "device": "mirror_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_1", "error": "Input/output error", "len": 393216, "offset": 65536, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_2"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_2", "sync": "bitmap", "target": "mirror_target_2"}}
+{"return": {}}
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": true,
+ "count": 655360,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ],
+ "mirror-top": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": false
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 10 dirty sectors; have 10. OK!
+
+= Checking Bitmap bitmap0 =
+expecting 10 dirty sectors; have 10. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 983040,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 15 dirty sectors; have 15. OK!
+
+--- Reference mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_3", "sync": "full", "target": "ref_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_3"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_3", "sync": "bitmap", "target": "mirror_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"device": "mirror_3", "len": 983040, "offset": 983040, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"id": "mirror_3", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_3", "len": 983040, "offset": 983040, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 983040,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 15 dirty sectors; have 15. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {}
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fmirror1" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fmirror2" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync3" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+
+
+=== Mode bitmap; Bitmap Sync never without failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {}
+}
+
+--- Reference mirror #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_0"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_1"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_1", "sync": "bitmap", "target": "mirror_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_1"}}
+{"return": {}}
+{"data": {"device": "mirror_1", "len": 393216, "offset": 393216, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "mirror_1"}}
+{"return": {}}
+{"data": {"id": "mirror_1", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_1", "len": 393216, "offset": 393216, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_2"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_2", "sync": "bitmap", "target": "mirror_target_2"}}
+{"return": {}}
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": true,
+ "count": 655360,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ],
+ "mirror-top": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": false
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 10 dirty sectors; have 10. OK!
+
+= Checking Bitmap bitmap0 =
+expecting 10 dirty sectors; have 10. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 983040,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 15 dirty sectors; have 15. OK!
+
+--- Reference mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_3", "sync": "full", "target": "ref_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_3"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_3", "sync": "bitmap", "target": "mirror_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"device": "mirror_3", "len": 983040, "offset": 983040, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"id": "mirror_3", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_3", "len": 983040, "offset": 983040, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 983040,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 15 dirty sectors; have 15. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {}
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fmirror1" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fmirror2" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync3" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+
+
+=== Mode bitmap; Bitmap Sync on-success with simulated failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {}
+}
+
+--- Reference mirror #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_0"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_1"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_1", "sync": "bitmap", "target": "mirror_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_1"}}
+{"return": {}}
+{"data": {"device": "mirror_1", "len": 393216, "offset": 393216, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-cancel", "arguments": {"id": "mirror_1"}}
+{"return": {}}
+{"data": {"id": "mirror_1", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_1", "len": 393216, "offset": 393216, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_2"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_2", "sync": "bitmap", "target": "mirror_target_2"}}
+{"return": {}}
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": true,
+ "count": 655360,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ],
+ "mirror-top": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": false
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 10 dirty sectors; have 10. OK!
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 524288,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 8 dirty sectors; have 8. OK!
+
+--- Reference mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_3", "sync": "full", "target": "ref_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_3"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_3", "sync": "bitmap", "target": "mirror_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"device": "mirror_3", "len": 524288, "offset": 524288, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"id": "mirror_3", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_3", "len": 524288, "offset": 524288, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {}
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fmirror1" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fmirror2" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync3" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+
+
+=== Mode bitmap; Bitmap Sync on-success with intermediate failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false, "once": true, "state": 3}, {"errno": 5, "event": "read_aio", "immediately": false, "once": true, "state": 4}], "set-state": [{"event": "flush_to_disk", "new-state": 2, "state": 1}, {"event": "read_aio", "new-state": 3, "state": 2}, {"event": "read_aio", "new-state": 4, "state": 3}]}, "node-name": "drive0"}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {}
+}
+
+--- Reference mirror #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_0"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_1"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+{"return": ""}
+
+--- Test mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_1", "sync": "bitmap", "target": "mirror_target_1"}}
+{"return": {}}
+{"data": {"action": "report", "device": "mirror_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"action": "report", "device": "mirror_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_1", "error": "Input/output error", "len": 393216, "offset": 65536, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_2"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_2", "sync": "bitmap", "target": "mirror_target_2"}}
+{"return": {}}
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": true,
+ "count": 655360,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ],
+ "mirror-top": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": false
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 10 dirty sectors; have 10. OK!
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 524288,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 8 dirty sectors; have 8. OK!
+
+--- Reference mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_3", "sync": "full", "target": "ref_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_3"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_3", "sync": "bitmap", "target": "mirror_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"device": "mirror_3", "len": 524288, "offset": 524288, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"id": "mirror_3", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_3", "len": 524288, "offset": 524288, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {}
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fmirror1" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fmirror2" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync3" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+
+
+=== Mode bitmap; Bitmap Sync on-success without failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {}
+}
+
+--- Reference mirror #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_0"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_1"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_1", "sync": "bitmap", "target": "mirror_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_1"}}
+{"return": {}}
+{"data": {"device": "mirror_1", "len": 393216, "offset": 393216, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "mirror_1"}}
+{"return": {}}
+{"data": {"id": "mirror_1", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_1", "len": 393216, "offset": 393216, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Reference mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_2"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_2", "sync": "bitmap", "target": "mirror_target_2"}}
+{"return": {}}
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": true,
+ "count": 458752,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ],
+ "mirror-top": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": false
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 7 dirty sectors; have 7. OK!
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 524288,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 8 dirty sectors; have 8. OK!
+
+--- Reference mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_3", "sync": "full", "target": "ref_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_3"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_3", "sync": "bitmap", "target": "mirror_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"device": "mirror_3", "len": 524288, "offset": 524288, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"id": "mirror_3", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_3", "len": 524288, "offset": 524288, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {}
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fmirror1" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fmirror2" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync3" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+
+
+=== Mode bitmap; Bitmap Sync always with simulated failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {}
+}
+
+--- Reference mirror #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_0"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_1"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_1", "sync": "bitmap", "target": "mirror_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_1"}}
+{"return": {}}
+{"data": {"device": "mirror_1", "len": 393216, "offset": 393216, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-cancel", "arguments": {"id": "mirror_1"}}
+{"return": {}}
+{"data": {"id": "mirror_1", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_1", "len": 393216, "offset": 393216, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Reference mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_2"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_2", "sync": "bitmap", "target": "mirror_target_2"}}
+{"return": {}}
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": true,
+ "count": 458752,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ],
+ "mirror-top": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": false
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 7 dirty sectors; have 7. OK!
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 524288,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 8 dirty sectors; have 8. OK!
+
+--- Reference mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_3", "sync": "full", "target": "ref_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_3"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_3", "sync": "bitmap", "target": "mirror_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"device": "mirror_3", "len": 524288, "offset": 524288, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"id": "mirror_3", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_3", "len": 524288, "offset": 524288, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {}
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fmirror1" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fmirror2" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync3" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+
+
+=== Mode bitmap; Bitmap Sync always with intermediate failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false, "once": true, "state": 3}, {"errno": 5, "event": "read_aio", "immediately": false, "once": true, "state": 4}], "set-state": [{"event": "flush_to_disk", "new-state": 2, "state": 1}, {"event": "read_aio", "new-state": 3, "state": 2}, {"event": "read_aio", "new-state": 4, "state": 3}]}, "node-name": "drive0"}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {}
+}
+
+--- Reference mirror #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_0"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_1"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+{"return": ""}
+
+--- Test mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_1", "sync": "bitmap", "target": "mirror_target_1"}}
+{"return": {}}
+{"data": {"action": "report", "device": "mirror_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"action": "report", "device": "mirror_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_1", "error": "Input/output error", "len": 393216, "offset": 65536, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 327680,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 5 dirty sectors; have 5. OK!
+
+--- Reference mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_2"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_2", "sync": "bitmap", "target": "mirror_target_2"}}
+{"return": {}}
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": true,
+ "count": 589824,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ],
+ "mirror-top": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": false
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 9 dirty sectors; have 9. OK!
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 524288,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 8 dirty sectors; have 8. OK!
+
+--- Reference mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_3", "sync": "full", "target": "ref_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_3"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_3", "sync": "bitmap", "target": "mirror_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"device": "mirror_3", "len": 524288, "offset": 524288, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"id": "mirror_3", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_3", "len": 524288, "offset": 524288, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {}
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fmirror1" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fmirror2" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync3" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+
+
+=== Mode bitmap; Bitmap Sync always without failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {}
+}
+
+--- Reference mirror #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_0"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_1"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test mirror #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_1", "sync": "bitmap", "target": "mirror_target_1"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_1"}}
+{"return": {}}
+{"data": {"device": "mirror_1", "len": 393216, "offset": 393216, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "mirror_1"}}
+{"return": {}}
+{"data": {"id": "mirror_1", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_1", "len": 393216, "offset": 393216, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Reference mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_2"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test mirror #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_2", "sync": "bitmap", "target": "mirror_target_2"}}
+{"return": {}}
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": true,
+ "count": 458752,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ],
+ "mirror-top": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": false
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 7 dirty sectors; have 7. OK!
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 524288,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 8 dirty sectors; have 8. OK!
+
+--- Reference mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "ref_mirror_3", "sync": "full", "target": "ref_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "ref_mirror_3"}}
+{"return": {}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "ref_mirror_3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test mirror #3 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-mirror", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "mirror_3", "sync": "bitmap", "target": "mirror_target_3"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"device": "mirror_3", "len": 524288, "offset": 524288, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "mirror_3"}}
+{"return": {}}
+{"data": {"id": "mirror_3", "type": "mirror"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "mirror_3", "len": 524288, "offset": 524288, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {}
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fmirror1" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fmirror2" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync3" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fmirror3" ==> Identical, OK!
+
+
+=== API failure tests ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+-- Testing invalid QMP commands --
+
+-- Sync mode incremental tests --
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "incremental", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Must provide a valid bitmap name for 'incremental' sync mode"}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap-mode": "always", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "incremental", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Must provide a valid bitmap name for 'incremental' sync mode"}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap-mode": "never", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "incremental", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Must provide a valid bitmap name for 'incremental' sync mode"}}
+
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "incremental", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Must provide a valid bitmap name for 'incremental' sync mode"}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "incremental", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Dirty bitmap 'bitmap404' not found"}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "incremental", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap sync mode must be 'on-success' when using sync mode 'incremental'"}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "incremental", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap sync mode must be 'on-success' when using sync mode 'incremental'"}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap404", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "incremental", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Dirty bitmap 'bitmap404' not found"}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "incremental", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap sync mode must be 'on-success' when using sync mode 'incremental'"}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "incremental", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap sync mode must be 'on-success' when using sync mode 'incremental'"}}
+
+-- Sync mode bitmap tests --
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "bitmap", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Must provide a valid bitmap name for 'bitmap' sync mode"}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap-mode": "always", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "bitmap", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Must provide a valid bitmap name for 'bitmap' sync mode"}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap-mode": "never", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "bitmap", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Must provide a valid bitmap name for 'bitmap' sync mode"}}
+
+{"execute": "blockdev-mirror", "arguments": {"device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "bitmap", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Must provide a valid bitmap name for 'bitmap' sync mode"}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "bitmap", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Dirty bitmap 'bitmap404' not found"}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "bitmap", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Dirty bitmap 'bitmap404' not found"}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "bitmap", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Dirty bitmap 'bitmap404' not found"}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap404", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "bitmap", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "bitmap-mode must be specified if a bitmap is provided"}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap0", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "bitmap", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "bitmap-mode must be specified if a bitmap is provided"}}
+
+-- Sync mode full tests --
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "full", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mode without a bitmap"}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap-mode": "always", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "full", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mode without a bitmap"}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap-mode": "never", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "full", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mode without a bitmap"}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "full", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'full' not supported with bitmap."}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "full", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'full' not supported with bitmap."}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "full", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'full' not supported with bitmap."}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap404", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "full", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'full' not supported with bitmap."}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "full", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'full' not supported with bitmap."}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "full", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'full' not supported with bitmap."}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "full", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'full' not supported with bitmap."}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap0", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "full", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'full' not supported with bitmap."}}
+
+-- Sync mode top tests --
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "top", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mode without a bitmap"}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap-mode": "always", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "top", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mode without a bitmap"}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap-mode": "never", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "top", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mode without a bitmap"}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "top", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'full' not supported with bitmap."}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "top", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'full' not supported with bitmap."}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "top", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'full' not supported with bitmap."}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap404", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "top", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'full' not supported with bitmap."}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "top", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'full' not supported with bitmap."}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "top", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'full' not supported with bitmap."}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "top", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'full' not supported with bitmap."}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap0", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "top", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'full' not supported with bitmap."}}
+
+-- Sync mode none tests --
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "none", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mode without a bitmap"}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap-mode": "always", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "none", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mode without a bitmap"}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap-mode": "never", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "none", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mode without a bitmap"}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "none", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'none' not supported with bitmap."}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "none", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'none' not supported with bitmap."}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "none", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'none' not supported with bitmap."}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap404", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "none", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'none' not supported with bitmap."}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "none", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'none' not supported with bitmap."}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "none", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'none' not supported with bitmap."}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "none", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'none' not supported with bitmap."}}
+
+{"execute": "blockdev-mirror", "arguments": {"bitmap": "bitmap0", "device": "drive0", "filter-node-name": "mirror-top", "job-id": "api_job", "sync": "none", "target": "mirror_target"}}
+{"error": {"class": "GenericError", "desc": "Sync mode 'none' not supported with bitmap."}}
+
--
2.39.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [RFC 1/4] drive-mirror: add support for sync=bitmap mode=never
2024-02-16 10:55 ` [RFC 1/4] drive-mirror: add support for sync=bitmap mode=never Fiona Ebner
@ 2024-02-21 6:55 ` Markus Armbruster
2024-02-21 9:21 ` Fiona Ebner
0 siblings, 1 reply; 25+ messages in thread
From: Markus Armbruster @ 2024-02-21 6:55 UTC (permalink / raw)
To: Fiona Ebner
Cc: qemu-devel, qemu-block, eblake, hreitz, kwolf, vsementsov, jsnow,
f.gruenbichler, t.lamprecht, mahaocong
Fiona Ebner <f.ebner@proxmox.com> writes:
> From: John Snow <jsnow@redhat.com>
>
> This patch adds support for the "BITMAP" sync mode to drive-mirror and
> blockdev-mirror. It adds support only for the BitmapSyncMode "never,"
> because it's the simplest mode.
>
> This mode simply uses a user-provided bitmap as an initial copy
> manifest, and then does not clear any bits in the bitmap at the
> conclusion of the operation.
>
> Any new writes dirtied during the operation are copied out, in contrast
> to backup. Note that whether these writes are reflected in the bitmap
> at the conclusion of the operation depends on whether that bitmap is
> actually recording!
>
> This patch was originally based on one by Ma Haocong, but it has since
> been modified pretty heavily.
>
> Suggested-by: Ma Haocong <mahaocong@didichuxing.com>
> Signed-off-by: Ma Haocong <mahaocong@didichuxing.com>
> Signed-off-by: John Snow <jsnow@redhat.com>
> [FG: switch to bdrv_dirty_bitmap_merge_internal]
> Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
> Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
> [FE: rebase for 9.0
> update version and formatting in QAPI]
> Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
[...]
> diff --git a/qapi/block-core.json b/qapi/block-core.json
> index ab5a93a966..ac05483958 100644
> --- a/qapi/block-core.json
> +++ b/qapi/block-core.json
> @@ -2181,6 +2181,15 @@
> # destination (all the disk, only the sectors allocated in the
> # topmost image, or only new I/O).
> #
> +# @bitmap: The name of a bitmap to use for sync=bitmap mode. This
> +# argument must be present for bitmap mode and absent otherwise.
> +# The bitmap's granularity is used instead of @granularity.
> +# (Since 9.0).
What happens when the user specifies @granularity anyway? Error or
silently ignored?
> +#
> +# @bitmap-mode: Specifies the type of data the bitmap should contain
> +# after the operation concludes. Must be present if sync is
> +# "bitmap". Must NOT be present otherwise. (Since 9.0)
Members that must be present when and only when some enum member has a
certain value should perhaps be in a union branch. Perhaps the block
maintainers have an opinion here.
> +#
> # @granularity: granularity of the dirty bitmap, default is 64K if the
> # image format doesn't have clusters, 4K if the clusters are
> # smaller than that, else the cluster size. Must be a power of 2
> @@ -2223,7 +2232,9 @@
> { 'struct': 'DriveMirror',
> 'data': { '*job-id': 'str', 'device': 'str', 'target': 'str',
> '*format': 'str', '*node-name': 'str', '*replaces': 'str',
> - 'sync': 'MirrorSyncMode', '*mode': 'NewImageMode',
> + 'sync': 'MirrorSyncMode', '*bitmap': 'str',
> + '*bitmap-mode': 'BitmapSyncMode',
> + '*mode': 'NewImageMode',
> '*speed': 'int', '*granularity': 'uint32',
> '*buf-size': 'int', '*on-source-error': 'BlockdevOnError',
> '*on-target-error': 'BlockdevOnError',
> @@ -2507,6 +2518,15 @@
> # destination (all the disk, only the sectors allocated in the
> # topmost image, or only new I/O).
> #
> +# @bitmap: The name of a bitmap to use for sync=bitmap mode. This
> +# argument must be present for bitmap mode and absent otherwise.
> +# The bitmap's granularity is used instead of @granularity.
> +# (Since 9.0).
> +#
> +# @bitmap-mode: Specifies the type of data the bitmap should contain
> +# after the operation concludes. Must be present if sync is
> +# "bitmap". Must NOT be present otherwise. (Since 9.0)
> +#
> # @granularity: granularity of the dirty bitmap, default is 64K if the
> # image format doesn't have clusters, 4K if the clusters are
> # smaller than that, else the cluster size. Must be a power of 2
> @@ -2557,7 +2577,8 @@
> { 'command': 'blockdev-mirror',
> 'data': { '*job-id': 'str', 'device': 'str', 'target': 'str',
> '*replaces': 'str',
> - 'sync': 'MirrorSyncMode',
> + 'sync': 'MirrorSyncMode', '*bitmap': 'str',
> + '*bitmap-mode': 'BitmapSyncMode',
> '*speed': 'int', '*granularity': 'uint32',
> '*buf-size': 'int', '*on-source-error': 'BlockdevOnError',
> '*on-target-error': 'BlockdevOnError',
[...]
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC 1/4] drive-mirror: add support for sync=bitmap mode=never
2024-02-21 6:55 ` Markus Armbruster
@ 2024-02-21 9:21 ` Fiona Ebner
0 siblings, 0 replies; 25+ messages in thread
From: Fiona Ebner @ 2024-02-21 9:21 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, qemu-block, eblake, hreitz, kwolf, vsementsov, jsnow,
f.gruenbichler, t.lamprecht, mahaocong
Am 21.02.24 um 07:55 schrieb Markus Armbruster:
>> diff --git a/qapi/block-core.json b/qapi/block-core.json
>> index ab5a93a966..ac05483958 100644
>> --- a/qapi/block-core.json
>> +++ b/qapi/block-core.json
>> @@ -2181,6 +2181,15 @@
>> # destination (all the disk, only the sectors allocated in the
>> # topmost image, or only new I/O).
>> #
>> +# @bitmap: The name of a bitmap to use for sync=bitmap mode. This
>> +# argument must be present for bitmap mode and absent otherwise.
>> +# The bitmap's granularity is used instead of @granularity.
>> +# (Since 9.0).
>
> What happens when the user specifies @granularity anyway? Error or
> silently ignored?
>
It's an error:
>> + if (bitmap) {
>> + if (granularity) {
>> + error_setg(errp, "granularity (%d)"
>> + "cannot be specified when a bitmap is provided",
>> + granularity);
>> + return NULL;
>> + }
>> +#
>> +# @bitmap-mode: Specifies the type of data the bitmap should contain
>> +# after the operation concludes. Must be present if sync is
>> +# "bitmap". Must NOT be present otherwise. (Since 9.0)
>
> Members that must be present when and only when some enum member has a
> certain value should perhaps be in a union branch. Perhaps the block
> maintainers have an opinion here.
>
Sounds sensible to me. Considering also the next patches, in the end it
could be a union discriminated by the @sync which contains @bitmap and
@bitmap-mode when it's the 'bitmap' sync mode, @bitmap when it's the
'incremental' sync mode (@bitmap-sync mode needs to be 'on-success'
then, so there is no choice for the user) and which contains
@granularity for the other sync modes.
Best Regards,
Fiona
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC 0/4] mirror: implement incremental and bitmap modes
2024-02-16 10:55 [RFC 0/4] mirror: implement incremental and bitmap modes Fiona Ebner
` (3 preceding siblings ...)
2024-02-16 10:55 ` [RFC 4/4] iotests: add test for bitmap mirror Fiona Ebner
@ 2024-02-28 16:00 ` Vladimir Sementsov-Ogievskiy
2024-02-28 16:06 ` Vladimir Sementsov-Ogievskiy
2024-02-28 16:24 ` Vladimir Sementsov-Ogievskiy
5 siblings, 1 reply; 25+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2024-02-28 16:00 UTC (permalink / raw)
To: Fiona Ebner, qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, jsnow, f.gruenbichler,
t.lamprecht, mahaocong
On 16.02.24 13:55, Fiona Ebner wrote:
> Previous discussion from when this was sent upstream [0] (it's been a
> while). I rebased the patches and re-ordered and squashed like
> suggested back then [1].
>
> This implements two new mirror modes:
>
> - bitmap mirror mode with always/on-success/never bitmap sync mode
> - incremental mirror mode as sugar for bitmap + on-success
>
> Use cases:
> * Possibility to resume a failed mirror later.
> * Possibility to only mirror deltas to a previously mirrored volume.
> * Possibility to (efficiently) mirror an drive that was previously
> mirrored via some external mechanism (e.g. ZFS replication).
>
> We are using the last one in production without any issues since about
> 4 years now. In particular, like mentioned in [2]:
>
>> - create bitmap(s)
>> - (incrementally) replicate storage volume(s) out of band (using ZFS)
>> - incrementally drive mirror as part of a live migration of VM
>> - drop bitmap(s)
>
>
> Now, the IO test added in patch 4/4 actually contains yet another use
> case, namely doing incremental mirrors to stand-alone qcow2 "diff"
> images, that only contain the delta and can be rebased later. I had to
> adapt the IO test, because its output expected the mirror bitmap to
> still be dirty, but nowadays the mirror is apparently already done
> when the bitmaps are queried. So I thought, I'll just use
> 'write-blocking' mode to avoid any potential timing issues.
>
> But this exposed an issue with the diff image approach. If a write is
> not aligned to the granularity of the mirror target, then rebasing the
> diff image onto a backing image will not yield the desired result,
> because the full cluster is considered to be allocated and will "hide"
> some part of the base/backing image. The failure can be seen by either
> using 'write-blocking' mode in the IO test or setting the (bitmap)
> granularity to 32 KiB rather than the current 64 KiB.
>
> The question is how to deal with these edge cases? Some possibilities
> that would make sense to me:
>
> For 'background' mode:
> * prohibit if target's cluster size is larger than the bitmap
> granularity
> * document the limitation
>
> For 'write-blocking' mode:
> * disallow in combination with bitmap mode (would not be happy about
> it, because I'd like to use this without diff images)
why not just require the same: bitmap granularity must be >= target granularity
> * for writes that are not aligned to the target's cluster size, read
> the relevant/missing parts from the source image to be able to write
> whole target clusters (seems rather complex)
There is another approach: consider and unaligned part of the request, fit in one cluster (we can always split any request to "aligned" middle part, and at most two small "unligned" parts, each fit into one cluster).
We have two possibilities:
1. the cluster is dirty (marked dirty in the bitmap used by background process)
We can simply ignore this part and rely on background process. This will not affect the convergence of the mirror job.
2. the cluster is clear (i.e. background process, or some previous write already copied it)
In this case, we are safe to do unaligned write, as target cluster must be allocated.
(for bitmap-mode, I don't consider here clusters that are clear from the start, which we shouldn't copy in any case)
> * document the limitation
>
>
> [0]: https://lore.kernel.org/qemu-devel/20200218100740.2228521-1-f.gruenbichler@proxmox.com/
> [1]: https://lore.kernel.org/qemu-devel/d35a76de-78d5-af56-0b34-f7bd2bbd3733@redhat.com/
> [2]: https://lore.kernel.org/qemu-devel/1599127031.9uxdp5h9o2.astroid@nora.none/
>
>
> Fabian Grünbichler (2):
> mirror: move some checks to qmp
> iotests: add test for bitmap mirror
>
> John Snow (2):
> drive-mirror: add support for sync=bitmap mode=never
> drive-mirror: add support for conditional and always bitmap sync modes
>
> block/mirror.c | 94 +-
> blockdev.c | 70 +-
> include/block/block_int-global-state.h | 4 +-
> qapi/block-core.json | 25 +-
> tests/qemu-iotests/tests/bitmap-sync-mirror | 550 ++++
> .../qemu-iotests/tests/bitmap-sync-mirror.out | 2810 +++++++++++++++++
> tests/unit/test-block-iothread.c | 4 +-
> 7 files changed, 3527 insertions(+), 30 deletions(-)
> create mode 100755 tests/qemu-iotests/tests/bitmap-sync-mirror
> create mode 100644 tests/qemu-iotests/tests/bitmap-sync-mirror.out
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC 0/4] mirror: implement incremental and bitmap modes
2024-02-28 16:00 ` [RFC 0/4] mirror: implement incremental and bitmap modes Vladimir Sementsov-Ogievskiy
@ 2024-02-28 16:06 ` Vladimir Sementsov-Ogievskiy
2024-02-29 10:11 ` Fiona Ebner
0 siblings, 1 reply; 25+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2024-02-28 16:06 UTC (permalink / raw)
To: Fiona Ebner, qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, jsnow, f.gruenbichler,
t.lamprecht, mahaocong
On 28.02.24 19:00, Vladimir Sementsov-Ogievskiy wrote:
> On 16.02.24 13:55, Fiona Ebner wrote:
>> Previous discussion from when this was sent upstream [0] (it's been a
>> while). I rebased the patches and re-ordered and squashed like
>> suggested back then [1].
>>
>> This implements two new mirror modes:
>>
>> - bitmap mirror mode with always/on-success/never bitmap sync mode
>> - incremental mirror mode as sugar for bitmap + on-success
>>
>> Use cases:
>> * Possibility to resume a failed mirror later.
>> * Possibility to only mirror deltas to a previously mirrored volume.
>> * Possibility to (efficiently) mirror an drive that was previously
>> mirrored via some external mechanism (e.g. ZFS replication).
>>
>> We are using the last one in production without any issues since about
>> 4 years now. In particular, like mentioned in [2]:
>>
>>> - create bitmap(s)
>>> - (incrementally) replicate storage volume(s) out of band (using ZFS)
>>> - incrementally drive mirror as part of a live migration of VM
>>> - drop bitmap(s)
>>
>>
>> Now, the IO test added in patch 4/4 actually contains yet another use
>> case, namely doing incremental mirrors to stand-alone qcow2 "diff"
>> images, that only contain the delta and can be rebased later. I had to
>> adapt the IO test, because its output expected the mirror bitmap to
>> still be dirty, but nowadays the mirror is apparently already done
>> when the bitmaps are queried. So I thought, I'll just use
>> 'write-blocking' mode to avoid any potential timing issues.
>>
>> But this exposed an issue with the diff image approach. If a write is
>> not aligned to the granularity of the mirror target, then rebasing the
>> diff image onto a backing image will not yield the desired result,
>> because the full cluster is considered to be allocated and will "hide"
>> some part of the base/backing image. The failure can be seen by either
>> using 'write-blocking' mode in the IO test or setting the (bitmap)
>> granularity to 32 KiB rather than the current 64 KiB.
>>
>> The question is how to deal with these edge cases? Some possibilities
>> that would make sense to me:
>>
>> For 'background' mode:
>> * prohibit if target's cluster size is larger than the bitmap
>> granularity
>> * document the limitation
>>
>> For 'write-blocking' mode:
>> * disallow in combination with bitmap mode (would not be happy about
>> it, because I'd like to use this without diff images)
>
> why not just require the same: bitmap granularity must be >= target granularity
>
>> * for writes that are not aligned to the target's cluster size, read
>> the relevant/missing parts from the source image to be able to write
>> whole target clusters (seems rather complex)
>
> There is another approach: consider and unaligned part of the request, fit in one cluster (we can always split any request to "aligned" middle part, and at most two small "unligned" parts, each fit into one cluster).
>
> We have two possibilities:
>
> 1. the cluster is dirty (marked dirty in the bitmap used by background process)
>
> We can simply ignore this part and rely on background process. This will not affect the convergence of the mirror job.
>
> 2. the cluster is clear (i.e. background process, or some previous write already copied it)
>
> In this case, we are safe to do unaligned write, as target cluster must be allocated.
>
> (for bitmap-mode, I don't consider here clusters that are clear from the start, which we shouldn't copy in any case)
>
Hmm, right, and that's exactly the logic we already have in do_sync_target_write(). So that's enough just to require that bitmap_granularity >= target_granularity
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC 0/4] mirror: implement incremental and bitmap modes
2024-02-16 10:55 [RFC 0/4] mirror: implement incremental and bitmap modes Fiona Ebner
` (4 preceding siblings ...)
2024-02-28 16:00 ` [RFC 0/4] mirror: implement incremental and bitmap modes Vladimir Sementsov-Ogievskiy
@ 2024-02-28 16:24 ` Vladimir Sementsov-Ogievskiy
2024-02-29 10:41 ` Fiona Ebner
5 siblings, 1 reply; 25+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2024-02-28 16:24 UTC (permalink / raw)
To: Fiona Ebner, qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, jsnow, f.gruenbichler,
t.lamprecht, mahaocong
On 16.02.24 13:55, Fiona Ebner wrote:
> Previous discussion from when this was sent upstream [0] (it's been a
> while). I rebased the patches and re-ordered and squashed like
> suggested back then [1].
>
> This implements two new mirror modes:
>
> - bitmap mirror mode with always/on-success/never bitmap sync mode
> - incremental mirror mode as sugar for bitmap + on-success
>
> Use cases:
> * Possibility to resume a failed mirror later.
> * Possibility to only mirror deltas to a previously mirrored volume.
> * Possibility to (efficiently) mirror an drive that was previously
> mirrored via some external mechanism (e.g. ZFS replication).
>
> We are using the last one in production without any issues since about
> 4 years now. In particular, like mentioned in [2]:
>
>> - create bitmap(s)
>> - (incrementally) replicate storage volume(s) out of band (using ZFS)
>> - incrementally drive mirror as part of a live migration of VM
>> - drop bitmap(s)
Actually which mode you use, "never", "always" or "conditional"? Or in downstream you have different approach?
Why am I asking:
These modes (for backup) were developed prior to block-dirty-bitmap-merge command, which allowed to copy bitmaps as you want. With that API, we actually don't need all these modes, instead it's enough to pass a bitmap, which would be _actually_ used by mirror.
So, if you need "never" mode, you just copy your bitmap by block-dirty-bitmap-add + block-dirty-bitmap-merge, and pass a copy to mirror job.
Or, you pass your bitmap to mirror-job, and have a "always" mode.
And I don't see, why we need a "conditional" mode, which actually just drops away the progress we actually made. (OK, we failed, but why to drop the progress of successfully copied clusters?)
Using user-given bitmap in the mirror job has also an additional advantage of live progress: up to visualization of disk copying by visualization of the dirty bitmap contents.
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC 0/4] mirror: implement incremental and bitmap modes
2024-02-28 16:06 ` Vladimir Sementsov-Ogievskiy
@ 2024-02-29 10:11 ` Fiona Ebner
2024-02-29 11:48 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 25+ messages in thread
From: Fiona Ebner @ 2024-02-29 10:11 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, jsnow, f.gruenbichler,
t.lamprecht, mahaocong
Am 28.02.24 um 17:06 schrieb Vladimir Sementsov-Ogievskiy:
> On 28.02.24 19:00, Vladimir Sementsov-Ogievskiy wrote:
>> On 16.02.24 13:55, Fiona Ebner wrote:
>>> Now, the IO test added in patch 4/4 actually contains yet another use
>>> case, namely doing incremental mirrors to stand-alone qcow2 "diff"
>>> images, that only contain the delta and can be rebased later. I had to
>>> adapt the IO test, because its output expected the mirror bitmap to
>>> still be dirty, but nowadays the mirror is apparently already done
>>> when the bitmaps are queried. So I thought, I'll just use
>>> 'write-blocking' mode to avoid any potential timing issues.
>>>
>>> But this exposed an issue with the diff image approach. If a write is
>>> not aligned to the granularity of the mirror target, then rebasing the
>>> diff image onto a backing image will not yield the desired result,
>>> because the full cluster is considered to be allocated and will "hide"
>>> some part of the base/backing image. The failure can be seen by either
>>> using 'write-blocking' mode in the IO test or setting the (bitmap)
>>> granularity to 32 KiB rather than the current 64 KiB.
>>>
>>> The question is how to deal with these edge cases? Some possibilities
>>> that would make sense to me:
>>>
>>> For 'background' mode:
>>> * prohibit if target's cluster size is larger than the bitmap
>>> granularity
>>> * document the limitation
>>>
>>> For 'write-blocking' mode:
>>> * disallow in combination with bitmap mode (would not be happy about
>>> it, because I'd like to use this without diff images)
>>
>> why not just require the same: bitmap granularity must be >= target
>> granularity
>>
For the iotest's use-case, that only works for background mode. I'll
explain below.
>>> * for writes that are not aligned to the target's cluster size, read
>>> the relevant/missing parts from the source image to be able to write
>>> whole target clusters (seems rather complex)
>>
>> There is another approach: consider and unaligned part of the request,
>> fit in one cluster (we can always split any request to "aligned"
>> middle part, and at most two small "unligned" parts, each fit into one
>> cluster).
>>
>> We have two possibilities:
>>
>> 1. the cluster is dirty (marked dirty in the bitmap used by background
>> process)
>>
>> We can simply ignore this part and rely on background process. This
>> will not affect the convergence of the mirror job.
>>
Agreed.
>> 2. the cluster is clear (i.e. background process, or some previous
>> write already copied it)
>>
The iotest creates a new target image for each incremental sync which
only records the diff relative to the previous mirror and those diff
images are later rebased onto each other to get the full picture.
Thus, it can be that a previous mirror job (not just background process
or previous write) already copied a cluster, and in particular, copied
it to a different target!
>> In this case, we are safe to do unaligned write, as target cluster
>> must be allocated.
Because the diff image is new, the target's cluster is not necessarily
allocated. When using write-blocking and a write of, e.g., 9 bytes to a
clear source cluster comes in, only those 9 bytes are written to the
target. Now the target's cluster is allocated but with only those 9
bytes of data. When rebasing, the previously copied cluster is "masked"
and when reading the rebased image, we only see the cluster with those 9
bytes (and IIRC, zeroes for the rest of the cluster rather than the
previously copied data).
>>
>> (for bitmap-mode, I don't consider here clusters that are clear from
>> the start, which we shouldn't copy in any case)
>>
We do need to copy new writes to any cluster, and with a clear cluster
and write-blocking, the issue can manifest.
>
> Hmm, right, and that's exactly the logic we already have in
> do_sync_target_write(). So that's enough just to require that
> bitmap_granularity >= target_granularity
>
Best Regards,
Fiona
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC 0/4] mirror: implement incremental and bitmap modes
2024-02-28 16:24 ` Vladimir Sementsov-Ogievskiy
@ 2024-02-29 10:41 ` Fiona Ebner
2024-02-29 12:00 ` Vladimir Sementsov-Ogievskiy
2024-03-04 9:02 ` Fabian Grünbichler
0 siblings, 2 replies; 25+ messages in thread
From: Fiona Ebner @ 2024-02-29 10:41 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, jsnow, f.gruenbichler,
t.lamprecht, mahaocong
Am 28.02.24 um 17:24 schrieb Vladimir Sementsov-Ogievskiy:
> On 16.02.24 13:55, Fiona Ebner wrote:
>> Previous discussion from when this was sent upstream [0] (it's been a
>> while). I rebased the patches and re-ordered and squashed like
>> suggested back then [1].
>>
>> This implements two new mirror modes:
>>
>> - bitmap mirror mode with always/on-success/never bitmap sync mode
>> - incremental mirror mode as sugar for bitmap + on-success
>>
>> Use cases:
>> * Possibility to resume a failed mirror later.
>> * Possibility to only mirror deltas to a previously mirrored volume.
>> * Possibility to (efficiently) mirror an drive that was previously
>> mirrored via some external mechanism (e.g. ZFS replication).
>>
>> We are using the last one in production without any issues since about
>> 4 years now. In particular, like mentioned in [2]:
>>
>>> - create bitmap(s)
>>> - (incrementally) replicate storage volume(s) out of band (using ZFS)
>>> - incrementally drive mirror as part of a live migration of VM
>>> - drop bitmap(s)
>
> Actually which mode you use, "never", "always" or "conditional"? Or in
> downstream you have different approach?
>
We are using "conditional", but I think we don't really require any
specific mode, because we drop the bitmaps after mirroring (even in
failure case). Fabian, please correct me if I'm wrong.
> Why am I asking:
>
> These modes (for backup) were developed prior to
> block-dirty-bitmap-merge command, which allowed to copy bitmaps as you
> want. With that API, we actually don't need all these modes, instead
> it's enough to pass a bitmap, which would be _actually_ used by mirror.
>
> So, if you need "never" mode, you just copy your bitmap by
> block-dirty-bitmap-add + block-dirty-bitmap-merge, and pass a copy to
> mirror job.
>
> Or, you pass your bitmap to mirror-job, and have a "always" mode.
>
> And I don't see, why we need a "conditional" mode, which actually just
> drops away the progress we actually made. (OK, we failed, but why to
> drop the progress of successfully copied clusters?)
>
I'm not sure actually. Maybe John remembers?
I see, I'll drop the 'bitmap-mode' in the next version if nobody
complains :)
>
> Using user-given bitmap in the mirror job has also an additional
> advantage of live progress: up to visualization of disk copying by
> visualization of the dirty bitmap contents.
>
Best Regards,
Fiona
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC 0/4] mirror: implement incremental and bitmap modes
2024-02-29 10:11 ` Fiona Ebner
@ 2024-02-29 11:48 ` Vladimir Sementsov-Ogievskiy
2024-02-29 12:47 ` Fiona Ebner
0 siblings, 1 reply; 25+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2024-02-29 11:48 UTC (permalink / raw)
To: Fiona Ebner, qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, jsnow, f.gruenbichler,
t.lamprecht, mahaocong
On 29.02.24 13:11, Fiona Ebner wrote:
> Am 28.02.24 um 17:06 schrieb Vladimir Sementsov-Ogievskiy:
>> On 28.02.24 19:00, Vladimir Sementsov-Ogievskiy wrote:
>>> On 16.02.24 13:55, Fiona Ebner wrote:
>>>> Now, the IO test added in patch 4/4 actually contains yet another use
>>>> case, namely doing incremental mirrors to stand-alone qcow2 "diff"
>>>> images, that only contain the delta and can be rebased later. I had to
>>>> adapt the IO test, because its output expected the mirror bitmap to
>>>> still be dirty, but nowadays the mirror is apparently already done
>>>> when the bitmaps are queried. So I thought, I'll just use
>>>> 'write-blocking' mode to avoid any potential timing issues.
>>>>
>>>> But this exposed an issue with the diff image approach. If a write is
>>>> not aligned to the granularity of the mirror target, then rebasing the
>>>> diff image onto a backing image will not yield the desired result,
>>>> because the full cluster is considered to be allocated and will "hide"
>>>> some part of the base/backing image. The failure can be seen by either
>>>> using 'write-blocking' mode in the IO test or setting the (bitmap)
>>>> granularity to 32 KiB rather than the current 64 KiB.
>>>>
>>>> The question is how to deal with these edge cases? Some possibilities
>>>> that would make sense to me:
>>>>
>>>> For 'background' mode:
>>>> * prohibit if target's cluster size is larger than the bitmap
>>>> granularity
>>>> * document the limitation
>>>>
>>>> For 'write-blocking' mode:
>>>> * disallow in combination with bitmap mode (would not be happy about
>>>> it, because I'd like to use this without diff images)
>>>
>>> why not just require the same: bitmap granularity must be >= target
>>> granularity
>>>
>
> For the iotest's use-case, that only works for background mode. I'll
> explain below.
>
>>>> * for writes that are not aligned to the target's cluster size, read
>>>> the relevant/missing parts from the source image to be able to write
>>>> whole target clusters (seems rather complex)
>>>
>>> There is another approach: consider and unaligned part of the request,
>>> fit in one cluster (we can always split any request to "aligned"
>>> middle part, and at most two small "unligned" parts, each fit into one
>>> cluster).
>>>
>>> We have two possibilities:
>>>
>>> 1. the cluster is dirty (marked dirty in the bitmap used by background
>>> process)
>>>
>>> We can simply ignore this part and rely on background process. This
>>> will not affect the convergence of the mirror job.
>>>
>
> Agreed.
>
>>> 2. the cluster is clear (i.e. background process, or some previous
>>> write already copied it)
>>>
>
> The iotest creates a new target image for each incremental sync which
> only records the diff relative to the previous mirror and those diff
> images are later rebased onto each other to get the full picture.
>
> Thus, it can be that a previous mirror job (not just background process
> or previous write) already copied a cluster, and in particular, copied
> it to a different target!
Aha understand.
For simplicity, let's consider case, when source "cluster size" = "job cluster size" = "bitmap granularity" = "target cluster size".
Which types of clusters we should consider, when we want to handle guest write?
1. Clusters, that should be copied by background process
These are dirty clusters from user-given bitmap, or if we do a full-disk mirror, all clusters, not yet copied by background process.
For such clusters we simply ignore the unaligned write. We can even ignore the aligned write too: less disturbing the guest by delays.
2. Clusters, already copied by background process during this mirror job and not dirtied by guest since this time.
For such clusters we are safe to do unaligned write, as target cluster must be allocated.
3. Clusters, not marked initially by dirty bitmap.
What to do with them? We can't do unaligned write. I see two variants:
- do additional read from source, to fill the whole cluster, which seems a bit too heavy
- just mark the cluster as dirty for background job. So we behave like in "background" mode. But why not? The maximum count of such "hacks" is limited to number of "clear" clusters at start of mirror job, which means that we don't seriously affect the convergence. Mirror is guaranteed to converge anyway. And the whole sense of "write-blocking" mode is to have a guaranteed convergence. What do you think?
----
Of course, we can't distinguish 3 types by on dirty bitmap, so we need the second one. For example "done_bitmap", where we can mark clusters that were successfully copied. That would be a kind of block-status of target image. But using bitmap is a lot better than querying block-status from target.
>
>>> In this case, we are safe to do unaligned write, as target cluster
>>> must be allocated.
>
> Because the diff image is new, the target's cluster is not necessarily
> allocated. When using write-blocking and a write of, e.g., 9 bytes to a
> clear source cluster comes in, only those 9 bytes are written to the
> target. Now the target's cluster is allocated but with only those 9
> bytes of data. When rebasing, the previously copied cluster is "masked"
> and when reading the rebased image, we only see the cluster with those 9
> bytes (and IIRC, zeroes for the rest of the cluster rather than the
> previously copied data).
>
>>>
>>> (for bitmap-mode, I don't consider here clusters that are clear from
>>> the start, which we shouldn't copy in any case)
>>>
>
> We do need to copy new writes to any cluster, and with a clear cluster
> and write-blocking, the issue can manifest.
OK right, I was misunderstanding bitmap-mode for mirror. Now I understand.
>
>>
>> Hmm, right, and that's exactly the logic we already have in
>> do_sync_target_write(). So that's enough just to require that
>> bitmap_granularity >= target_granularity
>>
>
> Best Regards,
> Fiona
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC 0/4] mirror: implement incremental and bitmap modes
2024-02-29 10:41 ` Fiona Ebner
@ 2024-02-29 12:00 ` Vladimir Sementsov-Ogievskiy
2024-02-29 14:50 ` Fiona Ebner
2024-03-04 9:02 ` Fabian Grünbichler
1 sibling, 1 reply; 25+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2024-02-29 12:00 UTC (permalink / raw)
To: Fiona Ebner, qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, jsnow, f.gruenbichler,
t.lamprecht, mahaocong
On 29.02.24 13:41, Fiona Ebner wrote:
> Am 28.02.24 um 17:24 schrieb Vladimir Sementsov-Ogievskiy:
>> On 16.02.24 13:55, Fiona Ebner wrote:
>>> Previous discussion from when this was sent upstream [0] (it's been a
>>> while). I rebased the patches and re-ordered and squashed like
>>> suggested back then [1].
>>>
>>> This implements two new mirror modes:
>>>
>>> - bitmap mirror mode with always/on-success/never bitmap sync mode
>>> - incremental mirror mode as sugar for bitmap + on-success
>>>
>>> Use cases:
>>> * Possibility to resume a failed mirror later.
>>> * Possibility to only mirror deltas to a previously mirrored volume.
>>> * Possibility to (efficiently) mirror an drive that was previously
>>> mirrored via some external mechanism (e.g. ZFS replication).
>>>
>>> We are using the last one in production without any issues since about
>>> 4 years now. In particular, like mentioned in [2]:
>>>
>>>> - create bitmap(s)
>>>> - (incrementally) replicate storage volume(s) out of band (using ZFS)
>>>> - incrementally drive mirror as part of a live migration of VM
>>>> - drop bitmap(s)
>>
>> Actually which mode you use, "never", "always" or "conditional"? Or in
>> downstream you have different approach?
>>
>
> We are using "conditional", but I think we don't really require any
> specific mode, because we drop the bitmaps after mirroring (even in
> failure case). Fabian, please correct me if I'm wrong.
>
>> Why am I asking:
>>
>> These modes (for backup) were developed prior to
>> block-dirty-bitmap-merge command, which allowed to copy bitmaps as you
>> want. With that API, we actually don't need all these modes, instead
>> it's enough to pass a bitmap, which would be _actually_ used by mirror.
>>
>> So, if you need "never" mode, you just copy your bitmap by
>> block-dirty-bitmap-add + block-dirty-bitmap-merge, and pass a copy to
>> mirror job.
>>
>> Or, you pass your bitmap to mirror-job, and have a "always" mode.
>>
>> And I don't see, why we need a "conditional" mode, which actually just
>> drops away the progress we actually made. (OK, we failed, but why to
>> drop the progress of successfully copied clusters?)
>>
>
> I'm not sure actually. Maybe John remembers?
Ah, I understand. Conditional just make sense if you don't support "partial success", and you want to delete target image in case of failure. And create a new one, to restart incremental job.
But anyway, this all could be simply achieved with bitmap-copying/merging API, if we allow to pass user-given bitmap to the mirror as working bitmap.
>
> I see, I'll drop the 'bitmap-mode' in the next version if nobody
> complains :)
>
Good. It's a golden rule: never make public interfaces which you don't actually need for production. I myself sometimes violate it and spend extra time on developing features, which we later have to just drop as "not needed downstream, no sense in upstreaming".
>>
>> Using user-given bitmap in the mirror job has also an additional
>> advantage of live progress: up to visualization of disk copying by
>> visualization of the dirty bitmap contents.
>>
>
> Best Regards,
> Fiona
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC 0/4] mirror: implement incremental and bitmap modes
2024-02-29 11:48 ` Vladimir Sementsov-Ogievskiy
@ 2024-02-29 12:47 ` Fiona Ebner
2024-03-06 13:44 ` Fiona Ebner
0 siblings, 1 reply; 25+ messages in thread
From: Fiona Ebner @ 2024-02-29 12:47 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, jsnow, f.gruenbichler,
t.lamprecht, mahaocong
Am 29.02.24 um 12:48 schrieb Vladimir Sementsov-Ogievskiy:
> On 29.02.24 13:11, Fiona Ebner wrote:
>>
>> The iotest creates a new target image for each incremental sync which
>> only records the diff relative to the previous mirror and those diff
>> images are later rebased onto each other to get the full picture.
>>
>> Thus, it can be that a previous mirror job (not just background process
>> or previous write) already copied a cluster, and in particular, copied
>> it to a different target!
>
> Aha understand.
>
> For simplicity, let's consider case, when source "cluster size" = "job
> cluster size" = "bitmap granularity" = "target cluster size".
>
> Which types of clusters we should consider, when we want to handle guest
> write?
>
> 1. Clusters, that should be copied by background process
>
> These are dirty clusters from user-given bitmap, or if we do a full-disk
> mirror, all clusters, not yet copied by background process.
>
> For such clusters we simply ignore the unaligned write. We can even
> ignore the aligned write too: less disturbing the guest by delays.
>
Since do_sync_target_write() currently doesn't ignore aligned writes, I
wouldn't change it. Of course they can count towards the "done_bitmap"
you propose below.
> 2. Clusters, already copied by background process during this mirror job
> and not dirtied by guest since this time.
>
> For such clusters we are safe to do unaligned write, as target cluster
> must be allocated.
>
Right.
> 3. Clusters, not marked initially by dirty bitmap.
>
> What to do with them? We can't do unaligned write. I see two variants:
>
> - do additional read from source, to fill the whole cluster, which seems
> a bit too heavy
>
Yes, I'd rather only do that as a last resort.
> - just mark the cluster as dirty for background job. So we behave like
> in "background" mode. But why not? The maximum count of such "hacks" is
> limited to number of "clear" clusters at start of mirror job, which
> means that we don't seriously affect the convergence. Mirror is
> guaranteed to converge anyway. And the whole sense of "write-blocking"
> mode is to have a guaranteed convergence. What do you think?
>
It could lead to a lot of flips between job->actively_synced == true and
== false. AFAIU, currently, we only switch back from true to false when
an error happens. While I don't see a concrete issue with it, at least
it might be unexpected to users, so it better be documented.
I'll try going with this approach, thanks!
>
> ----
>
> Of course, we can't distinguish 3 types by on dirty bitmap, so we need
> the second one. For example "done_bitmap", where we can mark clusters
> that were successfully copied. That would be a kind of block-status of
> target image. But using bitmap is a lot better than querying
> block-status from target.
Best Regards,
Fiona
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC 0/4] mirror: implement incremental and bitmap modes
2024-02-29 12:00 ` Vladimir Sementsov-Ogievskiy
@ 2024-02-29 14:50 ` Fiona Ebner
2024-03-01 14:14 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 25+ messages in thread
From: Fiona Ebner @ 2024-02-29 14:50 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, jsnow, f.gruenbichler,
t.lamprecht, mahaocong
Am 29.02.24 um 13:00 schrieb Vladimir Sementsov-Ogievskiy:
>
> But anyway, this all could be simply achieved with
> bitmap-copying/merging API, if we allow to pass user-given bitmap to the
> mirror as working bitmap.
>
>>
>> I see, I'll drop the 'bitmap-mode' in the next version if nobody
>> complains :)
>>
>
> Good. It's a golden rule: never make public interfaces which you don't
> actually need for production. I myself sometimes violate it and spend
> extra time on developing features, which we later have to just drop as
> "not needed downstream, no sense in upstreaming".
>
Just wondering which new mode I should allow for the @MirrorSyncMode
then? The documentation states:
> # @incremental: only copy data described by the dirty bitmap.
> # (since: 2.4)
> #
> # @bitmap: only copy data described by the dirty bitmap. (since: 4.2)
> # Behavior on completion is determined by the BitmapSyncMode.
For backup, do_backup_common() just maps @incremental to @bitmap +
@bitmap-mode == @on-success.
Using @bitmap for mirror would lead to being at odds with the
documentation, because it mentions the BitmapSyncMode, which mirror
won't have.
Using @incremental for mirror would be consistent with the
documentation, but behave a bit differently from backup.
Opinions?
Best Regards,
Fiona
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC 0/4] mirror: implement incremental and bitmap modes
2024-02-29 14:50 ` Fiona Ebner
@ 2024-03-01 14:14 ` Vladimir Sementsov-Ogievskiy
2024-03-01 14:52 ` Fiona Ebner
0 siblings, 1 reply; 25+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2024-03-01 14:14 UTC (permalink / raw)
To: Fiona Ebner, qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, jsnow, f.gruenbichler,
t.lamprecht, mahaocong
On 29.02.24 17:50, Fiona Ebner wrote:
> Am 29.02.24 um 13:00 schrieb Vladimir Sementsov-Ogievskiy:
>>
>> But anyway, this all could be simply achieved with
>> bitmap-copying/merging API, if we allow to pass user-given bitmap to the
>> mirror as working bitmap.
>>
>>>
>>> I see, I'll drop the 'bitmap-mode' in the next version if nobody
>>> complains :)
>>>
>>
>> Good. It's a golden rule: never make public interfaces which you don't
>> actually need for production. I myself sometimes violate it and spend
>> extra time on developing features, which we later have to just drop as
>> "not needed downstream, no sense in upstreaming".
>>
>
> Just wondering which new mode I should allow for the @MirrorSyncMode
> then? The documentation states:
>
>> # @incremental: only copy data described by the dirty bitmap.
>> # (since: 2.4)
>> #
>> # @bitmap: only copy data described by the dirty bitmap. (since: 4.2)
>> # Behavior on completion is determined by the BitmapSyncMode.
>
> For backup, do_backup_common() just maps @incremental to @bitmap +
> @bitmap-mode == @on-success.
>
> Using @bitmap for mirror would lead to being at odds with the
> documentation, because it mentions the BitmapSyncMode, which mirror
> won't have.
>
> Using @incremental for mirror would be consistent with the
> documentation, but behave a bit differently from backup.
>
> Opinions?
>
Good question.
As we already understood, (block-)job-api needs some spring-cleaning. Unfortunately I don't have much time on it, but still I decided to start from finally depreacting block-job-* API and moving to job-*.. Probably bitmap/bitmap-mode/sync APIs also need some optimization, keeping in mind new block-dirty-bitmap-merge api.
So, what I could advice in this situation for newc interfaces:
1. be minimalistic
2. add `x-` prefix when unsure
So, following these two rules, what about x-bitmap field, which may be combined only with 'full' mode, and do what you need?
About documentation: actually, I never liked that we use for backup job "MirrorSyncMode". Now it looks more like "BackupSyncMode", having two values supported only by backup.
I'm also unsure how mode=full&bitmap=some_bitmap differs from mode=bitmap&bitmap=some_bitmap..
So, I'd suggest simply rename MirrorSyncMode to BackupSyncMode, and add separate MirrorSyncMode with only "full", "top" and "none" values.
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC 0/4] mirror: implement incremental and bitmap modes
2024-03-01 14:14 ` Vladimir Sementsov-Ogievskiy
@ 2024-03-01 14:52 ` Fiona Ebner
2024-03-01 15:02 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 25+ messages in thread
From: Fiona Ebner @ 2024-03-01 14:52 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, jsnow, f.gruenbichler,
t.lamprecht, mahaocong
Am 01.03.24 um 15:14 schrieb Vladimir Sementsov-Ogievskiy:
>
> As we already understood, (block-)job-api needs some spring-cleaning.
> Unfortunately I don't have much time on it, but still I decided to start
> from finally depreacting block-job-* API and moving to job-*.. Probably
> bitmap/bitmap-mode/sync APIs also need some optimization, keeping in
> mind new block-dirty-bitmap-merge api.
>
> So, what I could advice in this situation for newc interfaces:
>
> 1. be minimalistic
> 2. add `x-` prefix when unsure
>
> So, following these two rules, what about x-bitmap field, which may be
> combined only with 'full' mode, and do what you need?
>
AFAIU, it should rather be marked as @unstable in QAPI [0]? Then it
doesn't need to be renamed if it becomes stable later.
> About documentation: actually, I never liked that we use for backup job
> "MirrorSyncMode". Now it looks more like "BackupSyncMode", having two
> values supported only by backup.
>
> I'm also unsure how mode=full&bitmap=some_bitmap differs from
> mode=bitmap&bitmap=some_bitmap..
>
With the current patches, it was an error to specify @bitmap for other
modes than 'incremental' and 'bitmap'.
> So, I'd suggest simply rename MirrorSyncMode to BackupSyncMode, and add
> separate MirrorSyncMode with only "full", "top" and "none" values.
>
Sounds good to me!
[0]:
https://gitlab.com/qemu-project/qemu/-/commit/a3c45b3e62962f99338716b1347cfb0d427cea44
Best Regards,
Fiona
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC 0/4] mirror: implement incremental and bitmap modes
2024-03-01 14:52 ` Fiona Ebner
@ 2024-03-01 15:02 ` Vladimir Sementsov-Ogievskiy
2024-03-01 15:14 ` Fiona Ebner
0 siblings, 1 reply; 25+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2024-03-01 15:02 UTC (permalink / raw)
To: Fiona Ebner, qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, jsnow, f.gruenbichler,
t.lamprecht, mahaocong
On 01.03.24 17:52, Fiona Ebner wrote:
> Am 01.03.24 um 15:14 schrieb Vladimir Sementsov-Ogievskiy:
>>
>> As we already understood, (block-)job-api needs some spring-cleaning.
>> Unfortunately I don't have much time on it, but still I decided to start
>> from finally depreacting block-job-* API and moving to job-*.. Probably
>> bitmap/bitmap-mode/sync APIs also need some optimization, keeping in
>> mind new block-dirty-bitmap-merge api.
>>
>> So, what I could advice in this situation for newc interfaces:
>>
>> 1. be minimalistic
>> 2. add `x-` prefix when unsure
>>
>> So, following these two rules, what about x-bitmap field, which may be
>> combined only with 'full' mode, and do what you need?
>>
>
> AFAIU, it should rather be marked as @unstable in QAPI [0]? Then it
> doesn't need to be renamed if it becomes stable later.
Right, unstable feature is needed, using "x-" is optional.
Recent discussion about it was in my "vhost-user-blk: live resize additional APIs" series:
https://patchew.org/QEMU/20231006202045.1161543-1-vsementsov@yandex-team.ru/20231006202045.1161543-5-vsementsov@yandex-team.ru/
Following it, I think it's OK to not care anymore with "x-" prefixes, and rely on unstable feature.
>
>> About documentation: actually, I never liked that we use for backup job
>> "MirrorSyncMode". Now it looks more like "BackupSyncMode", having two
>> values supported only by backup.
>>
>> I'm also unsure how mode=full&bitmap=some_bitmap differs from
>> mode=bitmap&bitmap=some_bitmap..
>>
>
> With the current patches, it was an error to specify @bitmap for other
> modes than 'incremental' and 'bitmap'.
Current documentation says:
# @bitmap: The name of a dirty bitmap to use. Must be present if sync
# is "bitmap" or "incremental". Can be present if sync is "full"
# or "top". Must not be present otherwise.
# (Since 2.4 (drive-backup), 3.1 (blockdev-backup))
>
>> So, I'd suggest simply rename MirrorSyncMode to BackupSyncMode, and add
>> separate MirrorSyncMode with only "full", "top" and "none" values.
>>
>
> Sounds good to me!
>
> [0]:
> https://gitlab.com/qemu-project/qemu/-/commit/a3c45b3e62962f99338716b1347cfb0d427cea44
>
> Best Regards,
> Fiona
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC 0/4] mirror: implement incremental and bitmap modes
2024-03-01 15:02 ` Vladimir Sementsov-Ogievskiy
@ 2024-03-01 15:14 ` Fiona Ebner
2024-03-01 15:46 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 25+ messages in thread
From: Fiona Ebner @ 2024-03-01 15:14 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, jsnow, f.gruenbichler,
t.lamprecht, mahaocong
Am 01.03.24 um 16:02 schrieb Vladimir Sementsov-Ogievskiy:
> On 01.03.24 17:52, Fiona Ebner wrote:
>> Am 01.03.24 um 15:14 schrieb Vladimir Sementsov-Ogievskiy:
>>>
>>> As we already understood, (block-)job-api needs some spring-cleaning.
>>> Unfortunately I don't have much time on it, but still I decided to start
>>> from finally depreacting block-job-* API and moving to job-*.. Probably
>>> bitmap/bitmap-mode/sync APIs also need some optimization, keeping in
>>> mind new block-dirty-bitmap-merge api.
>>>
>>> So, what I could advice in this situation for newc interfaces:
>>>
>>> 1. be minimalistic
>>> 2. add `x-` prefix when unsure
>>>
>>> So, following these two rules, what about x-bitmap field, which may be
>>> combined only with 'full' mode, and do what you need?
>>>
>>
>> AFAIU, it should rather be marked as @unstable in QAPI [0]? Then it
>> doesn't need to be renamed if it becomes stable later.
>
> Right, unstable feature is needed, using "x-" is optional.
>
> Recent discussion about it was in my "vhost-user-blk: live resize
> additional APIs" series:
>
> https://patchew.org/QEMU/20231006202045.1161543-1-vsementsov@yandex-team.ru/20231006202045.1161543-5-vsementsov@yandex-team.ru/
>
> Following it, I think it's OK to not care anymore with "x-" prefixes,
> and rely on unstable feature.
>
Thanks for the confirmation! I'll go without the prefix in the name then.
>>
>>> About documentation: actually, I never liked that we use for backup job
>>> "MirrorSyncMode". Now it looks more like "BackupSyncMode", having two
>>> values supported only by backup.
>>>
>>> I'm also unsure how mode=full&bitmap=some_bitmap differs from
>>> mode=bitmap&bitmap=some_bitmap..
>>>
>>
>> With the current patches, it was an error to specify @bitmap for other
>> modes than 'incremental' and 'bitmap'.
>
> Current documentation says:
> # @bitmap: The name of a dirty bitmap to use. Must be present if sync
> # is "bitmap" or "incremental". Can be present if sync is "full"
> # or "top". Must not be present otherwise.
> # (Since 2.4 (drive-backup), 3.1 (blockdev-backup))
>
>
This is for backup. The documentation (and behavior) for @bitmap added
by these patches for mirror is different ;)
Best Regards,
Fiona
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC 0/4] mirror: implement incremental and bitmap modes
2024-03-01 15:14 ` Fiona Ebner
@ 2024-03-01 15:46 ` Vladimir Sementsov-Ogievskiy
2024-03-04 7:50 ` Fiona Ebner
0 siblings, 1 reply; 25+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2024-03-01 15:46 UTC (permalink / raw)
To: Fiona Ebner, qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, jsnow, f.gruenbichler,
t.lamprecht, mahaocong
On 01.03.24 18:14, Fiona Ebner wrote:
> Am 01.03.24 um 16:02 schrieb Vladimir Sementsov-Ogievskiy:
>> On 01.03.24 17:52, Fiona Ebner wrote:
>>> Am 01.03.24 um 15:14 schrieb Vladimir Sementsov-Ogievskiy:
>>>>
>>>> As we already understood, (block-)job-api needs some spring-cleaning.
>>>> Unfortunately I don't have much time on it, but still I decided to start
>>>> from finally depreacting block-job-* API and moving to job-*.. Probably
>>>> bitmap/bitmap-mode/sync APIs also need some optimization, keeping in
>>>> mind new block-dirty-bitmap-merge api.
>>>>
>>>> So, what I could advice in this situation for newc interfaces:
>>>>
>>>> 1. be minimalistic
>>>> 2. add `x-` prefix when unsure
>>>>
>>>> So, following these two rules, what about x-bitmap field, which may be
>>>> combined only with 'full' mode, and do what you need?
>>>>
>>>
>>> AFAIU, it should rather be marked as @unstable in QAPI [0]? Then it
>>> doesn't need to be renamed if it becomes stable later.
>>
>> Right, unstable feature is needed, using "x-" is optional.
>>
>> Recent discussion about it was in my "vhost-user-blk: live resize
>> additional APIs" series:
>>
>> https://patchew.org/QEMU/20231006202045.1161543-1-vsementsov@yandex-team.ru/20231006202045.1161543-5-vsementsov@yandex-team.ru/
>>
>> Following it, I think it's OK to not care anymore with "x-" prefixes,
>> and rely on unstable feature.
>>
>
> Thanks for the confirmation! I'll go without the prefix in the name then.
>
>>>
>>>> About documentation: actually, I never liked that we use for backup job
>>>> "MirrorSyncMode". Now it looks more like "BackupSyncMode", having two
>>>> values supported only by backup.
>>>>
>>>> I'm also unsure how mode=full&bitmap=some_bitmap differs from
>>>> mode=bitmap&bitmap=some_bitmap..
>>>>
>>>
>>> With the current patches, it was an error to specify @bitmap for other
>>> modes than 'incremental' and 'bitmap'.
>>
>> Current documentation says:
>> # @bitmap: The name of a dirty bitmap to use. Must be present if sync
>> # is "bitmap" or "incremental". Can be present if sync is "full"
>> # or "top". Must not be present otherwise.
>> # (Since 2.4 (drive-backup), 3.1 (blockdev-backup))
>>
>>
>
> This is for backup. The documentation (and behavior) for @bitmap added
> by these patches for mirror is different ;)
I meant backup in "I'm also unsure", just as one more point not consider backup-bitmap-API as a prototype for mirror-bitmap-API.
>
> Best Regards,
> Fiona
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC 0/4] mirror: implement incremental and bitmap modes
2024-03-01 15:46 ` Vladimir Sementsov-Ogievskiy
@ 2024-03-04 7:50 ` Fiona Ebner
0 siblings, 0 replies; 25+ messages in thread
From: Fiona Ebner @ 2024-03-04 7:50 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, jsnow, f.gruenbichler,
t.lamprecht, mahaocong
Am 01.03.24 um 16:46 schrieb Vladimir Sementsov-Ogievskiy:
> On 01.03.24 18:14, Fiona Ebner wrote:
>> Am 01.03.24 um 16:02 schrieb Vladimir Sementsov-Ogievskiy:
>>>>> About documentation: actually, I never liked that we use for backup
>>>>> job
>>>>> "MirrorSyncMode". Now it looks more like "BackupSyncMode", having two
>>>>> values supported only by backup.
>>>>>
>>>>> I'm also unsure how mode=full&bitmap=some_bitmap differs from
>>>>> mode=bitmap&bitmap=some_bitmap..
>>>>>
>>>>
>>>> With the current patches, it was an error to specify @bitmap for other
>>>> modes than 'incremental' and 'bitmap'.
>>>
>>> Current documentation says:
>>> # @bitmap: The name of a dirty bitmap to use. Must be present if
>>> sync
>>> # is "bitmap" or "incremental". Can be present if sync is "full"
>>> # or "top". Must not be present otherwise.
>>> # (Since 2.4 (drive-backup), 3.1 (blockdev-backup))
>>>
>>>
>>
>> This is for backup. The documentation (and behavior) for @bitmap added
>> by these patches for mirror is different ;)
>
> I meant backup in "I'm also unsure", just as one more point not consider
> backup-bitmap-API as a prototype for mirror-bitmap-API.
>
Oh, I see. Sorry for the confusion!
Best Regards,
Fiona
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC 0/4] mirror: implement incremental and bitmap modes
2024-02-29 10:41 ` Fiona Ebner
2024-02-29 12:00 ` Vladimir Sementsov-Ogievskiy
@ 2024-03-04 9:02 ` Fabian Grünbichler
1 sibling, 0 replies; 25+ messages in thread
From: Fabian Grünbichler @ 2024-03-04 9:02 UTC (permalink / raw)
To: Fiona Ebner, qemu-devel, Vladimir Sementsov-Ogievskiy
Cc: armbru, eblake, hreitz, jsnow, kwolf, mahaocong, qemu-block,
t.lamprecht
On February 29, 2024 11:41 am, Fiona Ebner wrote:
> Am 28.02.24 um 17:24 schrieb Vladimir Sementsov-Ogievskiy:
>> On 16.02.24 13:55, Fiona Ebner wrote:
>>> Previous discussion from when this was sent upstream [0] (it's been a
>>> while). I rebased the patches and re-ordered and squashed like
>>> suggested back then [1].
>>>
>>> This implements two new mirror modes:
>>>
>>> - bitmap mirror mode with always/on-success/never bitmap sync mode
>>> - incremental mirror mode as sugar for bitmap + on-success
>>>
>>> Use cases:
>>> * Possibility to resume a failed mirror later.
>>> * Possibility to only mirror deltas to a previously mirrored volume.
>>> * Possibility to (efficiently) mirror an drive that was previously
>>> mirrored via some external mechanism (e.g. ZFS replication).
>>>
>>> We are using the last one in production without any issues since about
>>> 4 years now. In particular, like mentioned in [2]:
>>>
>>>> - create bitmap(s)
>>>> - (incrementally) replicate storage volume(s) out of band (using ZFS)
>>>> - incrementally drive mirror as part of a live migration of VM
>>>> - drop bitmap(s)
>>
>> Actually which mode you use, "never", "always" or "conditional"? Or in
>> downstream you have different approach?
>>
>
> We are using "conditional", but I think we don't really require any
> specific mode, because we drop the bitmaps after mirroring (even in
> failure case). Fabian, please correct me if I'm wrong.
indeed, we don't really care for our current use case (and don't have
any other planned either, AFAIK), the bitmap is used only for the
duration of a single mirror, and always discarded at the end.
>> Why am I asking:
>>
>> These modes (for backup) were developed prior to
>> block-dirty-bitmap-merge command, which allowed to copy bitmaps as you
>> want. With that API, we actually don't need all these modes, instead
>> it's enough to pass a bitmap, which would be _actually_ used by mirror.
>>
>> So, if you need "never" mode, you just copy your bitmap by
>> block-dirty-bitmap-add + block-dirty-bitmap-merge, and pass a copy to
>> mirror job.
>>
>> Or, you pass your bitmap to mirror-job, and have a "always" mode.
>>
>> And I don't see, why we need a "conditional" mode, which actually just
>> drops away the progress we actually made. (OK, we failed, but why to
>> drop the progress of successfully copied clusters?)
>>
>
> I'm not sure actually. Maybe John remembers?
>
> I see, I'll drop the 'bitmap-mode' in the next version if nobody
> complains :)
it was probably just done to mimic the backup interface, if that is not
desired, dropping it is probably a good idea.
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC 0/4] mirror: implement incremental and bitmap modes
2024-02-29 12:47 ` Fiona Ebner
@ 2024-03-06 13:44 ` Fiona Ebner
2024-03-07 9:20 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 25+ messages in thread
From: Fiona Ebner @ 2024-03-06 13:44 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, jsnow, f.gruenbichler,
t.lamprecht, mahaocong
Am 29.02.24 um 13:47 schrieb Fiona Ebner:
> Am 29.02.24 um 12:48 schrieb Vladimir Sementsov-Ogievskiy:
>> On 29.02.24 13:11, Fiona Ebner wrote:
>>>
>>> The iotest creates a new target image for each incremental sync which
>>> only records the diff relative to the previous mirror and those diff
>>> images are later rebased onto each other to get the full picture.
>>>
>>> Thus, it can be that a previous mirror job (not just background process
>>> or previous write) already copied a cluster, and in particular, copied
>>> it to a different target!
>>
>> Aha understand.
>>
>> For simplicity, let's consider case, when source "cluster size" = "job
>> cluster size" = "bitmap granularity" = "target cluster size".
>>
>> Which types of clusters we should consider, when we want to handle guest
>> write?
>>
>> 1. Clusters, that should be copied by background process
>>
>> These are dirty clusters from user-given bitmap, or if we do a full-disk
>> mirror, all clusters, not yet copied by background process.
>>
>> For such clusters we simply ignore the unaligned write. We can even
>> ignore the aligned write too: less disturbing the guest by delays.
>>
>
> Since do_sync_target_write() currently doesn't ignore aligned writes, I
> wouldn't change it. Of course they can count towards the "done_bitmap"
> you propose below.
>
>> 2. Clusters, already copied by background process during this mirror job
>> and not dirtied by guest since this time.
>>
>> For such clusters we are safe to do unaligned write, as target cluster
>> must be allocated.
>>
>
> Right.
>
>> 3. Clusters, not marked initially by dirty bitmap.
>>
>> What to do with them? We can't do unaligned write. I see two variants:
>>
>> - do additional read from source, to fill the whole cluster, which seems
>> a bit too heavy
>>
>
> Yes, I'd rather only do that as a last resort.
>
>> - just mark the cluster as dirty for background job. So we behave like
>> in "background" mode. But why not? The maximum count of such "hacks" is
>> limited to number of "clear" clusters at start of mirror job, which
>> means that we don't seriously affect the convergence. Mirror is
>> guaranteed to converge anyway. And the whole sense of "write-blocking"
>> mode is to have a guaranteed convergence. What do you think?
>>
>
> It could lead to a lot of flips between job->actively_synced == true and
> == false. AFAIU, currently, we only switch back from true to false when
> an error happens. While I don't see a concrete issue with it, at least
> it might be unexpected to users, so it better be documented.
>
> I'll try going with this approach, thanks!
>
These flips are actually a problem. When using live-migration with disk
mirroring, it's good that an actively synced image stays actively
synced. Otherwise, migration could finish at an inconvenient time and
try to inactivate the block device while mirror still got something to
do which would lead to an assertion failure [0].
The IO test added by this series is what uses the possibility to sync to
"diff images" which contain only the delta. In production, we are only
syncing to a previously mirrored target image. Non-aligned writes are
not an issue later like with a diff image. (Even if the initial
mirroring happened via ZFS replication outside of QEMU).
So copy-mode=write-blocking would work fine for our use case, but if I
go with the "mark clusters for unaligned writes dirty"-approach, it
would not work fine anymore.
Should I rather just document the limitation for the combination "target
is a diff image" and copy-mode=write-blocking?
I'd still add the check for the granularity and target cluster size.
While also only needed for diff images, it would allow using background
mode safely for those.
Best Regards,
Fiona
[0]:
https://lore.kernel.org/qemu-devel/1db7f571-cb7f-c293-04cc-cd856e060c3f@proxmox.com/
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC 0/4] mirror: implement incremental and bitmap modes
2024-03-06 13:44 ` Fiona Ebner
@ 2024-03-07 9:20 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 25+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2024-03-07 9:20 UTC (permalink / raw)
To: Fiona Ebner, qemu-devel
Cc: qemu-block, armbru, eblake, hreitz, kwolf, jsnow, f.gruenbichler,
t.lamprecht, mahaocong
On 06.03.24 16:44, Fiona Ebner wrote:
> Am 29.02.24 um 13:47 schrieb Fiona Ebner:
>> Am 29.02.24 um 12:48 schrieb Vladimir Sementsov-Ogievskiy:
>>> On 29.02.24 13:11, Fiona Ebner wrote:
>>>>
>>>> The iotest creates a new target image for each incremental sync which
>>>> only records the diff relative to the previous mirror and those diff
>>>> images are later rebased onto each other to get the full picture.
>>>>
>>>> Thus, it can be that a previous mirror job (not just background process
>>>> or previous write) already copied a cluster, and in particular, copied
>>>> it to a different target!
>>>
>>> Aha understand.
>>>
>>> For simplicity, let's consider case, when source "cluster size" = "job
>>> cluster size" = "bitmap granularity" = "target cluster size".
>>>
>>> Which types of clusters we should consider, when we want to handle guest
>>> write?
>>>
>>> 1. Clusters, that should be copied by background process
>>>
>>> These are dirty clusters from user-given bitmap, or if we do a full-disk
>>> mirror, all clusters, not yet copied by background process.
>>>
>>> For such clusters we simply ignore the unaligned write. We can even
>>> ignore the aligned write too: less disturbing the guest by delays.
>>>
>>
>> Since do_sync_target_write() currently doesn't ignore aligned writes, I
>> wouldn't change it. Of course they can count towards the "done_bitmap"
>> you propose below.
>>
>>> 2. Clusters, already copied by background process during this mirror job
>>> and not dirtied by guest since this time.
>>>
>>> For such clusters we are safe to do unaligned write, as target cluster
>>> must be allocated.
>>>
>>
>> Right.
>>
>>> 3. Clusters, not marked initially by dirty bitmap.
>>>
>>> What to do with them? We can't do unaligned write. I see two variants:
>>>
>>> - do additional read from source, to fill the whole cluster, which seems
>>> a bit too heavy
>>>
>>
>> Yes, I'd rather only do that as a last resort.
>>
>>> - just mark the cluster as dirty for background job. So we behave like
>>> in "background" mode. But why not? The maximum count of such "hacks" is
>>> limited to number of "clear" clusters at start of mirror job, which
>>> means that we don't seriously affect the convergence. Mirror is
>>> guaranteed to converge anyway. And the whole sense of "write-blocking"
>>> mode is to have a guaranteed convergence. What do you think?
>>>
>>
>> It could lead to a lot of flips between job->actively_synced == true and
>> == false. AFAIU, currently, we only switch back from true to false when
>> an error happens. While I don't see a concrete issue with it, at least
>> it might be unexpected to users, so it better be documented.
>>
>> I'll try going with this approach, thanks!
>>
>
> These flips are actually a problem. When using live-migration with disk
> mirroring, it's good that an actively synced image stays actively
> synced. Otherwise, migration could finish at an inconvenient time and
> try to inactivate the block device while mirror still got something to
> do which would lead to an assertion failure [0].
Hmm right. So, when mirror is actively-synced, we have to read the whole cluster from source to make an aligned write on target.
>
> The IO test added by this series is what uses the possibility to sync to
> "diff images" which contain only the delta. In production, we are only
> syncing to a previously mirrored target image. Non-aligned writes are
> not an issue later like with a diff image. (Even if the initial
> mirroring happened via ZFS replication outside of QEMU).
>
> So copy-mode=write-blocking would work fine for our use case, but if I
> go with the "mark clusters for unaligned writes dirty"-approach, it
> would not work fine anymore.
>
> Should I rather just document the limitation for the combination "target
> is a diff image" and copy-mode=write-blocking?
Of course, simply documenting the limitation is better than implementing a new feature, if you don't need the feature for production)
>
> I'd still add the check for the granularity and target cluster size.
Check is good too.
> While also only needed for diff images, it would allow using background
> mode safely for those.
>
> Best Regards,
> Fiona
>
> [0]:
> https://lore.kernel.org/qemu-devel/1db7f571-cb7f-c293-04cc-cd856e060c3f@proxmox.com/
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2024-03-07 9:21 UTC | newest]
Thread overview: 25+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-02-16 10:55 [RFC 0/4] mirror: implement incremental and bitmap modes Fiona Ebner
2024-02-16 10:55 ` [RFC 1/4] drive-mirror: add support for sync=bitmap mode=never Fiona Ebner
2024-02-21 6:55 ` Markus Armbruster
2024-02-21 9:21 ` Fiona Ebner
2024-02-16 10:55 ` [RFC 2/4] drive-mirror: add support for conditional and always bitmap sync modes Fiona Ebner
2024-02-16 10:55 ` [RFC 3/4] mirror: move some checks to qmp Fiona Ebner
2024-02-16 10:55 ` [RFC 4/4] iotests: add test for bitmap mirror Fiona Ebner
2024-02-28 16:00 ` [RFC 0/4] mirror: implement incremental and bitmap modes Vladimir Sementsov-Ogievskiy
2024-02-28 16:06 ` Vladimir Sementsov-Ogievskiy
2024-02-29 10:11 ` Fiona Ebner
2024-02-29 11:48 ` Vladimir Sementsov-Ogievskiy
2024-02-29 12:47 ` Fiona Ebner
2024-03-06 13:44 ` Fiona Ebner
2024-03-07 9:20 ` Vladimir Sementsov-Ogievskiy
2024-02-28 16:24 ` Vladimir Sementsov-Ogievskiy
2024-02-29 10:41 ` Fiona Ebner
2024-02-29 12:00 ` Vladimir Sementsov-Ogievskiy
2024-02-29 14:50 ` Fiona Ebner
2024-03-01 14:14 ` Vladimir Sementsov-Ogievskiy
2024-03-01 14:52 ` Fiona Ebner
2024-03-01 15:02 ` Vladimir Sementsov-Ogievskiy
2024-03-01 15:14 ` Fiona Ebner
2024-03-01 15:46 ` Vladimir Sementsov-Ogievskiy
2024-03-04 7:50 ` Fiona Ebner
2024-03-04 9:02 ` Fabian Grünbichler
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).