* [PULL 01/53] iotests: fix _check_o_direct
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 02/53] copy-on-read: support preadv/pwritev_part functions Max Reitz
` (52 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Unfortunately commit "iotests: handle tmpfs" breaks running iotests
with -nbd -nocache, as _check_o_direct tries to create
$TEST_IMG.test_o_direct, but in case of nbd TEST_IMG is something like
nbd+unix:///... , and test fails with message
qemu-img: nbd+unix:///?socket[...]test_o_direct: Protocol driver
'nbd' does not support image creation, and opening the image
failed: Failed to connect to '/tmp/tmp.[...]/nbd/test_o_direct': No
such file or directory
Use TEST_DIR instead.
Fixes: cfdca2b9f9d4ca26bb2b2dfe8de3149092e39170
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201218182012.47607-1-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
tests/qemu-iotests/common.rc | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index 29354654cc..297acf9b6a 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -821,9 +821,10 @@ _supported_cache_modes()
# Check whether the filesystem supports O_DIRECT
_check_o_direct()
{
- $QEMU_IMG create -f raw "$TEST_IMG".test_o_direct 1M > /dev/null
- out=$($QEMU_IO -f raw -t none -c quit "$TEST_IMG".test_o_direct 2>&1)
- rm -f "$TEST_IMG".test_o_direct
+ testfile="$TEST_DIR"/_check_o_direct
+ $QEMU_IMG create -f raw "$testfile" 1M > /dev/null
+ out=$($QEMU_IO -f raw -t none -c quit "$testfile" 2>&1)
+ rm -f "$testfile"
[[ "$out" != *"O_DIRECT"* ]]
}
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 02/53] copy-on-read: support preadv/pwritev_part functions
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
2021-01-26 14:19 ` [PULL 01/53] iotests: fix _check_o_direct Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 03/53] block: add API function to insert a node Max Reitz
` (51 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Add support for the recently introduced functions
bdrv_co_preadv_part()
and
bdrv_co_pwritev_part()
to the COR-filter driver.
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201216061703.70908-2-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/copy-on-read.c | 28 ++++++++++++++++------------
1 file changed, 16 insertions(+), 12 deletions(-)
diff --git a/block/copy-on-read.c b/block/copy-on-read.c
index 2816e61afe..cb03e0f2d3 100644
--- a/block/copy-on-read.c
+++ b/block/copy-on-read.c
@@ -74,21 +74,25 @@ static int64_t cor_getlength(BlockDriverState *bs)
}
-static int coroutine_fn cor_co_preadv(BlockDriverState *bs,
- uint64_t offset, uint64_t bytes,
- QEMUIOVector *qiov, int flags)
+static int coroutine_fn cor_co_preadv_part(BlockDriverState *bs,
+ uint64_t offset, uint64_t bytes,
+ QEMUIOVector *qiov,
+ size_t qiov_offset,
+ int flags)
{
- return bdrv_co_preadv(bs->file, offset, bytes, qiov,
- flags | BDRV_REQ_COPY_ON_READ);
+ return bdrv_co_preadv_part(bs->file, offset, bytes, qiov, qiov_offset,
+ flags | BDRV_REQ_COPY_ON_READ);
}
-static int coroutine_fn cor_co_pwritev(BlockDriverState *bs,
- uint64_t offset, uint64_t bytes,
- QEMUIOVector *qiov, int flags)
+static int coroutine_fn cor_co_pwritev_part(BlockDriverState *bs,
+ uint64_t offset,
+ uint64_t bytes,
+ QEMUIOVector *qiov,
+ size_t qiov_offset, int flags)
{
-
- return bdrv_co_pwritev(bs->file, offset, bytes, qiov, flags);
+ return bdrv_co_pwritev_part(bs->file, offset, bytes, qiov, qiov_offset,
+ flags);
}
@@ -137,8 +141,8 @@ static BlockDriver bdrv_copy_on_read = {
.bdrv_getlength = cor_getlength,
- .bdrv_co_preadv = cor_co_preadv,
- .bdrv_co_pwritev = cor_co_pwritev,
+ .bdrv_co_preadv_part = cor_co_preadv_part,
+ .bdrv_co_pwritev_part = cor_co_pwritev_part,
.bdrv_co_pwrite_zeroes = cor_co_pwrite_zeroes,
.bdrv_co_pdiscard = cor_co_pdiscard,
.bdrv_co_pwritev_compressed = cor_co_pwritev_compressed,
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 03/53] block: add API function to insert a node
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
2021-01-26 14:19 ` [PULL 01/53] iotests: fix _check_o_direct Max Reitz
2021-01-26 14:19 ` [PULL 02/53] copy-on-read: support preadv/pwritev_part functions Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 04/53] copy-on-read: add filter drop function Max Reitz
` (50 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Provide API for insertion a node to backing chain.
Suggested-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201216061703.70908-3-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
include/block/block.h | 2 ++
block.c | 25 +++++++++++++++++++++++++
2 files changed, 27 insertions(+)
diff --git a/include/block/block.h b/include/block/block.h
index a193545b6a..127bdf3392 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -358,6 +358,8 @@ void bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top,
Error **errp);
void bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
Error **errp);
+BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options,
+ int flags, Error **errp);
int bdrv_parse_aio(const char *mode, int *flags);
int bdrv_parse_cache_mode(const char *mode, int *flags, bool *writethrough);
diff --git a/block.c b/block.c
index 8b9d457546..91a66d4f3e 100644
--- a/block.c
+++ b/block.c
@@ -4660,6 +4660,31 @@ static void bdrv_delete(BlockDriverState *bs)
g_free(bs);
}
+BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options,
+ int flags, Error **errp)
+{
+ BlockDriverState *new_node_bs;
+ Error *local_err = NULL;
+
+ new_node_bs = bdrv_open(NULL, NULL, node_options, flags, errp);
+ if (new_node_bs == NULL) {
+ error_prepend(errp, "Could not create node: ");
+ return NULL;
+ }
+
+ bdrv_drained_begin(bs);
+ bdrv_replace_node(bs, new_node_bs, &local_err);
+ bdrv_drained_end(bs);
+
+ if (local_err) {
+ bdrv_unref(new_node_bs);
+ error_propagate(errp, local_err);
+ return NULL;
+ }
+
+ return new_node_bs;
+}
+
/*
* Run consistency checks on an image
*
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 04/53] copy-on-read: add filter drop function
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (2 preceding siblings ...)
2021-01-26 14:19 ` [PULL 03/53] block: add API function to insert a node Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 05/53] qapi: add filter-node-name to block-stream Max Reitz
` (49 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Provide API for the COR-filter removal. Also, drop the filter child
permissions for an inactive state when the filter node is being
removed.
To insert the filter, the block generic layer function
bdrv_insert_node() can be used.
The new function bdrv_cor_filter_drop() may be considered as an
intermediate solution before the QEMU permission update system has
overhauled. Then we are able to implement the API function
bdrv_remove_node() on the block generic layer.
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201216061703.70908-4-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/copy-on-read.h | 32 +++++++++++++++++++++++++
block/copy-on-read.c | 56 ++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 88 insertions(+)
create mode 100644 block/copy-on-read.h
diff --git a/block/copy-on-read.h b/block/copy-on-read.h
new file mode 100644
index 0000000000..7bf405dccd
--- /dev/null
+++ b/block/copy-on-read.h
@@ -0,0 +1,32 @@
+/*
+ * Copy-on-read filter block driver
+ *
+ * The filter driver performs Copy-On-Read (COR) operations
+ *
+ * Copyright (c) 2018-2020 Virtuozzo International GmbH.
+ *
+ * Author:
+ * Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef BLOCK_COPY_ON_READ
+#define BLOCK_COPY_ON_READ
+
+#include "block/block_int.h"
+
+void bdrv_cor_filter_drop(BlockDriverState *cor_filter_bs);
+
+#endif /* BLOCK_COPY_ON_READ */
diff --git a/block/copy-on-read.c b/block/copy-on-read.c
index cb03e0f2d3..618c4c4f43 100644
--- a/block/copy-on-read.c
+++ b/block/copy-on-read.c
@@ -23,11 +23,20 @@
#include "qemu/osdep.h"
#include "block/block_int.h"
#include "qemu/module.h"
+#include "qapi/error.h"
+#include "block/copy-on-read.h"
+
+
+typedef struct BDRVStateCOR {
+ bool active;
+} BDRVStateCOR;
static int cor_open(BlockDriverState *bs, QDict *options, int flags,
Error **errp)
{
+ BDRVStateCOR *state = bs->opaque;
+
bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
false, errp);
@@ -42,6 +51,13 @@ static int cor_open(BlockDriverState *bs, QDict *options, int flags,
((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
bs->file->bs->supported_zero_flags);
+ state->active = true;
+
+ /*
+ * We don't need to call bdrv_child_refresh_perms() now as the permissions
+ * will be updated later when the filter node gets its parent.
+ */
+
return 0;
}
@@ -57,6 +73,17 @@ static void cor_child_perm(BlockDriverState *bs, BdrvChild *c,
uint64_t perm, uint64_t shared,
uint64_t *nperm, uint64_t *nshared)
{
+ BDRVStateCOR *s = bs->opaque;
+
+ if (!s->active) {
+ /*
+ * While the filter is being removed
+ */
+ *nperm = 0;
+ *nshared = BLK_PERM_ALL;
+ return;
+ }
+
*nperm = perm & PERM_PASSTHROUGH;
*nshared = (shared & PERM_PASSTHROUGH) | PERM_UNCHANGED;
@@ -135,6 +162,7 @@ static void cor_lock_medium(BlockDriverState *bs, bool locked)
static BlockDriver bdrv_copy_on_read = {
.format_name = "copy-on-read",
+ .instance_size = sizeof(BDRVStateCOR),
.bdrv_open = cor_open,
.bdrv_child_perm = cor_child_perm,
@@ -154,6 +182,34 @@ static BlockDriver bdrv_copy_on_read = {
.is_filter = true,
};
+
+void bdrv_cor_filter_drop(BlockDriverState *cor_filter_bs)
+{
+ BdrvChild *child;
+ BlockDriverState *bs;
+ BDRVStateCOR *s = cor_filter_bs->opaque;
+
+ child = bdrv_filter_child(cor_filter_bs);
+ if (!child) {
+ return;
+ }
+ bs = child->bs;
+
+ /* Retain the BDS until we complete the graph change. */
+ bdrv_ref(bs);
+ /* Hold a guest back from writing while permissions are being reset. */
+ bdrv_drained_begin(bs);
+ /* Drop permissions before the graph change. */
+ s->active = false;
+ bdrv_child_refresh_perms(cor_filter_bs, child, &error_abort);
+ bdrv_replace_node(cor_filter_bs, bs, &error_abort);
+
+ bdrv_drained_end(bs);
+ bdrv_unref(bs);
+ bdrv_unref(cor_filter_bs);
+}
+
+
static void bdrv_copy_on_read_init(void)
{
bdrv_register(&bdrv_copy_on_read);
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 05/53] qapi: add filter-node-name to block-stream
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (3 preceding siblings ...)
2021-01-26 14:19 ` [PULL 04/53] copy-on-read: add filter drop function Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 06/53] qapi: copy-on-read filter: add 'bottom' option Max Reitz
` (48 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Provide the possibility to pass the 'filter-node-name' parameter to the
block-stream job as it is done for the commit block job.
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[vsementsov: comment indentation, s/Since: 5.2/Since: 6.0/]
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201216061703.70908-5-vsementsov@virtuozzo.com>
[mreitz: s/commit/stream/]
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
qapi/block-core.json | 6 ++++++
include/block/block_int.h | 7 ++++++-
block/monitor/block-hmp-cmds.c | 4 ++--
block/stream.c | 4 +++-
blockdev.c | 4 +++-
5 files changed, 20 insertions(+), 5 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 3484986d1c..b55732d802 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2543,6 +2543,11 @@
# 'stop' and 'enospc' can only be used if the block device
# supports io-status (see BlockInfo). Since 1.3.
#
+# @filter-node-name: the node name that should be assigned to the
+# filter driver that the stream job inserts into the graph
+# above @device. If this option is not given, a node name is
+# autogenerated. (Since: 6.0)
+#
# @auto-finalize: When false, this job will wait in a PENDING state after it has
# finished its work, waiting for @block-job-finalize before
# making any block graph changes.
@@ -2573,6 +2578,7 @@
'data': { '*job-id': 'str', 'device': 'str', '*base': 'str',
'*base-node': 'str', '*backing-file': 'str', '*speed': 'int',
'*on-error': 'BlockdevOnError',
+ '*filter-node-name': 'str',
'*auto-finalize': 'bool', '*auto-dismiss': 'bool' } }
##
diff --git a/include/block/block_int.h b/include/block/block_int.h
index b9ef61fe4d..f3d0a0e60a 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -1143,6 +1143,9 @@ int is_windows_drive(const char *filename);
* See @BlockJobCreateFlags
* @speed: The maximum speed, in bytes per second, or 0 for unlimited.
* @on_error: The action to take upon error.
+ * @filter_node_name: The node name that should be assigned to the filter
+ * driver that the stream job inserts into the graph above
+ * @bs. NULL means that a node name should be autogenerated.
* @errp: Error object.
*
* Start a streaming operation on @bs. Clusters that are unallocated
@@ -1155,7 +1158,9 @@ int is_windows_drive(const char *filename);
void stream_start(const char *job_id, BlockDriverState *bs,
BlockDriverState *base, const char *backing_file_str,
int creation_flags, int64_t speed,
- BlockdevOnError on_error, Error **errp);
+ BlockdevOnError on_error,
+ const char *filter_node_name,
+ Error **errp);
/**
* commit_start:
diff --git a/block/monitor/block-hmp-cmds.c b/block/monitor/block-hmp-cmds.c
index d15a2be827..e8a58f326e 100644
--- a/block/monitor/block-hmp-cmds.c
+++ b/block/monitor/block-hmp-cmds.c
@@ -508,8 +508,8 @@ void hmp_block_stream(Monitor *mon, const QDict *qdict)
qmp_block_stream(true, device, device, base != NULL, base, false, NULL,
false, NULL, qdict_haskey(qdict, "speed"), speed, true,
- BLOCKDEV_ON_ERROR_REPORT, false, false, false, false,
- &error);
+ BLOCKDEV_ON_ERROR_REPORT, false, NULL, false, false, false,
+ false, &error);
hmp_handle_error(mon, error);
}
diff --git a/block/stream.c b/block/stream.c
index 236384f2f7..6e281c71ac 100644
--- a/block/stream.c
+++ b/block/stream.c
@@ -221,7 +221,9 @@ static const BlockJobDriver stream_job_driver = {
void stream_start(const char *job_id, BlockDriverState *bs,
BlockDriverState *base, const char *backing_file_str,
int creation_flags, int64_t speed,
- BlockdevOnError on_error, Error **errp)
+ BlockdevOnError on_error,
+ const char *filter_node_name,
+ Error **errp)
{
StreamBlockJob *s;
BlockDriverState *iter;
diff --git a/blockdev.c b/blockdev.c
index 2431448c5d..e496356e10 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2502,6 +2502,7 @@ void qmp_block_stream(bool has_job_id, const char *job_id, const char *device,
bool has_backing_file, const char *backing_file,
bool has_speed, int64_t speed,
bool has_on_error, BlockdevOnError on_error,
+ bool has_filter_node_name, const char *filter_node_name,
bool has_auto_finalize, bool auto_finalize,
bool has_auto_dismiss, bool auto_dismiss,
Error **errp)
@@ -2584,7 +2585,8 @@ void qmp_block_stream(bool has_job_id, const char *job_id, const char *device,
}
stream_start(has_job_id ? job_id : NULL, bs, base_bs, base_name,
- job_flags, has_speed ? speed : 0, on_error, &local_err);
+ job_flags, has_speed ? speed : 0, on_error,
+ filter_node_name, &local_err);
if (local_err) {
error_propagate(errp, local_err);
goto out;
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 06/53] qapi: copy-on-read filter: add 'bottom' option
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (4 preceding siblings ...)
2021-01-26 14:19 ` [PULL 05/53] qapi: add filter-node-name to block-stream Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 07/53] iotests: add #310 to test bottom node in COR driver Max Reitz
` (47 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Add an option to limit copy-on-read operations to specified sub-chain
of backing-chain, to make copy-on-read filter useful for block-stream
job.
Suggested-by: Max Reitz <mreitz@redhat.com>
Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[vsementsov: change subject, modified to freeze the chain,
do some fixes]
Message-Id: <20201216061703.70908-6-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
qapi/block-core.json | 20 ++++++++-
block/copy-on-read.c | 98 +++++++++++++++++++++++++++++++++++++++++++-
2 files changed, 115 insertions(+), 3 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index b55732d802..65167ebf56 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3959,6 +3959,24 @@
'data': { 'throttle-group': 'str',
'file' : 'BlockdevRef'
} }
+
+##
+# @BlockdevOptionsCor:
+#
+# Driver specific block device options for the copy-on-read driver.
+#
+# @bottom: The name of a non-filter node (allocation-bearing layer) that
+# limits the COR operations in the backing chain (inclusive), so
+# that no data below this node will be copied by this filter.
+# If option is absent, the limit is not applied, so that data
+# from all backing layers may be copied.
+#
+# Since: 6.0
+##
+{ 'struct': 'BlockdevOptionsCor',
+ 'base': 'BlockdevOptionsGenericFormat',
+ 'data': { '*bottom': 'str' } }
+
##
# @BlockdevOptions:
#
@@ -4011,7 +4029,7 @@
'bochs': 'BlockdevOptionsGenericFormat',
'cloop': 'BlockdevOptionsGenericFormat',
'compress': 'BlockdevOptionsGenericFormat',
- 'copy-on-read':'BlockdevOptionsGenericFormat',
+ 'copy-on-read':'BlockdevOptionsCor',
'dmg': 'BlockdevOptionsGenericFormat',
'file': 'BlockdevOptionsFile',
'ftp': 'BlockdevOptionsCurlFtp',
diff --git a/block/copy-on-read.c b/block/copy-on-read.c
index 618c4c4f43..71560984f6 100644
--- a/block/copy-on-read.c
+++ b/block/copy-on-read.c
@@ -24,18 +24,24 @@
#include "block/block_int.h"
#include "qemu/module.h"
#include "qapi/error.h"
+#include "qapi/qmp/qdict.h"
#include "block/copy-on-read.h"
typedef struct BDRVStateCOR {
bool active;
+ BlockDriverState *bottom_bs;
+ bool chain_frozen;
} BDRVStateCOR;
static int cor_open(BlockDriverState *bs, QDict *options, int flags,
Error **errp)
{
+ BlockDriverState *bottom_bs = NULL;
BDRVStateCOR *state = bs->opaque;
+ /* Find a bottom node name, if any */
+ const char *bottom_node = qdict_get_try_str(options, "bottom");
bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
@@ -51,7 +57,38 @@ static int cor_open(BlockDriverState *bs, QDict *options, int flags,
((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
bs->file->bs->supported_zero_flags);
+ if (bottom_node) {
+ bottom_bs = bdrv_find_node(bottom_node);
+ if (!bottom_bs) {
+ error_setg(errp, "Bottom node '%s' not found", bottom_node);
+ qdict_del(options, "bottom");
+ return -EINVAL;
+ }
+ qdict_del(options, "bottom");
+
+ if (!bottom_bs->drv) {
+ error_setg(errp, "Bottom node '%s' not opened", bottom_node);
+ return -EINVAL;
+ }
+
+ if (bottom_bs->drv->is_filter) {
+ error_setg(errp, "Bottom node '%s' is a filter", bottom_node);
+ return -EINVAL;
+ }
+
+ if (bdrv_freeze_backing_chain(bs, bottom_bs, errp) < 0) {
+ return -EINVAL;
+ }
+ state->chain_frozen = true;
+
+ /*
+ * We do freeze the chain, so it shouldn't be removed. Still, storing a
+ * pointer worth bdrv_ref().
+ */
+ bdrv_ref(bottom_bs);
+ }
state->active = true;
+ state->bottom_bs = bottom_bs;
/*
* We don't need to call bdrv_child_refresh_perms() now as the permissions
@@ -107,8 +144,46 @@ static int coroutine_fn cor_co_preadv_part(BlockDriverState *bs,
size_t qiov_offset,
int flags)
{
- return bdrv_co_preadv_part(bs->file, offset, bytes, qiov, qiov_offset,
- flags | BDRV_REQ_COPY_ON_READ);
+ int64_t n;
+ int local_flags;
+ int ret;
+ BDRVStateCOR *state = bs->opaque;
+
+ if (!state->bottom_bs) {
+ return bdrv_co_preadv_part(bs->file, offset, bytes, qiov, qiov_offset,
+ flags | BDRV_REQ_COPY_ON_READ);
+ }
+
+ while (bytes) {
+ local_flags = flags;
+
+ /* In case of failure, try to copy-on-read anyway */
+ ret = bdrv_is_allocated(bs->file->bs, offset, bytes, &n);
+ if (ret <= 0) {
+ ret = bdrv_is_allocated_above(bdrv_backing_chain_next(bs->file->bs),
+ state->bottom_bs, true, offset,
+ n, &n);
+ if (ret > 0 || ret < 0) {
+ local_flags |= BDRV_REQ_COPY_ON_READ;
+ }
+ /* Finish earlier if the end of a backing file has been reached */
+ if (n == 0) {
+ break;
+ }
+ }
+
+ ret = bdrv_co_preadv_part(bs->file, offset, n, qiov, qiov_offset,
+ local_flags);
+ if (ret < 0) {
+ return ret;
+ }
+
+ offset += n;
+ qiov_offset += n;
+ bytes -= n;
+ }
+
+ return 0;
}
@@ -160,11 +235,25 @@ static void cor_lock_medium(BlockDriverState *bs, bool locked)
}
+static void cor_close(BlockDriverState *bs)
+{
+ BDRVStateCOR *s = bs->opaque;
+
+ if (s->chain_frozen) {
+ s->chain_frozen = false;
+ bdrv_unfreeze_backing_chain(bs, s->bottom_bs);
+ }
+
+ bdrv_unref(s->bottom_bs);
+}
+
+
static BlockDriver bdrv_copy_on_read = {
.format_name = "copy-on-read",
.instance_size = sizeof(BDRVStateCOR),
.bdrv_open = cor_open,
+ .bdrv_close = cor_close,
.bdrv_child_perm = cor_child_perm,
.bdrv_getlength = cor_getlength,
@@ -201,6 +290,11 @@ void bdrv_cor_filter_drop(BlockDriverState *cor_filter_bs)
bdrv_drained_begin(bs);
/* Drop permissions before the graph change. */
s->active = false;
+ /* unfreeze, as otherwise bdrv_replace_node() will fail */
+ if (s->chain_frozen) {
+ s->chain_frozen = false;
+ bdrv_unfreeze_backing_chain(cor_filter_bs, s->bottom_bs);
+ }
bdrv_child_refresh_perms(cor_filter_bs, child, &error_abort);
bdrv_replace_node(cor_filter_bs, bs, &error_abort);
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 07/53] iotests: add #310 to test bottom node in COR driver
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (5 preceding siblings ...)
2021-01-26 14:19 ` [PULL 06/53] qapi: copy-on-read filter: add 'bottom' option Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 08/53] block: include supported_read_flags into BDS structure Max Reitz
` (46 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
The test case #310 is similar to #216 by Max Reitz. The difference is
that the test #310 involves a bottom node to the COR filter driver.
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[vsementsov: detach backing to test reads from top, limit to qcow2]
Message-Id: <20201216061703.70908-7-vsementsov@virtuozzo.com>
[mreitz: Add "# group:" line]
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
tests/qemu-iotests/310 | 117 +++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/310.out | 15 +++++
tests/qemu-iotests/group | 1 +
3 files changed, 133 insertions(+)
create mode 100755 tests/qemu-iotests/310
create mode 100644 tests/qemu-iotests/310.out
diff --git a/tests/qemu-iotests/310 b/tests/qemu-iotests/310
new file mode 100755
index 0000000000..9d9c928c4b
--- /dev/null
+++ b/tests/qemu-iotests/310
@@ -0,0 +1,117 @@
+#!/usr/bin/env python3
+# group: rw quick
+#
+# Copy-on-read tests using a COR filter with a bottom node
+#
+# Copyright (C) 2018 Red Hat, Inc.
+# Copyright (c) 2020 Virtuozzo International GmbH
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+import iotests
+from iotests import log, qemu_img, qemu_io_silent
+
+# Need backing file support
+iotests.script_initialize(supported_fmts=['qcow2'],
+ supported_platforms=['linux'])
+
+log('')
+log('=== Copy-on-read across nodes ===')
+log('')
+
+# This test is similar to the 216 one by Max Reitz <mreitz@redhat.com>
+# The difference is that this test case involves a bottom node to the
+# COR filter driver.
+
+with iotests.FilePath('base.img') as base_img_path, \
+ iotests.FilePath('mid.img') as mid_img_path, \
+ iotests.FilePath('top.img') as top_img_path, \
+ iotests.VM() as vm:
+
+ log('--- Setting up images ---')
+ log('')
+
+ assert qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') == 0
+ assert qemu_io_silent(base_img_path, '-c', 'write -P 1 0M 1M') == 0
+ assert qemu_io_silent(base_img_path, '-c', 'write -P 1 3M 1M') == 0
+ assert qemu_img('create', '-f', iotests.imgfmt, '-b', base_img_path,
+ '-F', iotests.imgfmt, mid_img_path) == 0
+ assert qemu_io_silent(mid_img_path, '-c', 'write -P 3 2M 1M') == 0
+ assert qemu_io_silent(mid_img_path, '-c', 'write -P 3 4M 1M') == 0
+ assert qemu_img('create', '-f', iotests.imgfmt, '-b', mid_img_path,
+ '-F', iotests.imgfmt, top_img_path) == 0
+ assert qemu_io_silent(top_img_path, '-c', 'write -P 2 1M 1M') == 0
+
+# 0 1 2 3 4
+# top 2
+# mid 3 3
+# base 1 1
+
+ log('Done')
+
+ log('')
+ log('--- Doing COR ---')
+ log('')
+
+ vm.launch()
+
+ log(vm.qmp('blockdev-add',
+ node_name='node0',
+ driver='copy-on-read',
+ bottom='node2',
+ file={
+ 'driver': iotests.imgfmt,
+ 'file': {
+ 'driver': 'file',
+ 'filename': top_img_path
+ },
+ 'backing': {
+ 'node-name': 'node2',
+ 'driver': iotests.imgfmt,
+ 'file': {
+ 'driver': 'file',
+ 'filename': mid_img_path
+ },
+ 'backing': {
+ 'driver': iotests.imgfmt,
+ 'file': {
+ 'driver': 'file',
+ 'filename': base_img_path
+ }
+ },
+ }
+ }))
+
+ # Trigger COR
+ log(vm.qmp('human-monitor-command',
+ command_line='qemu-io node0 "read 0 5M"'))
+
+ vm.shutdown()
+
+ log('')
+ log('--- Checking COR result ---')
+ log('')
+
+ # Detach backing to check that we can read the data from the top level now
+ assert qemu_img('rebase', '-u', '-b', '', '-f', iotests.imgfmt,
+ top_img_path) == 0
+
+ assert qemu_io_silent(top_img_path, '-c', 'read -P 0 0 1M') == 0
+ assert qemu_io_silent(top_img_path, '-c', 'read -P 2 1M 1M') == 0
+ assert qemu_io_silent(top_img_path, '-c', 'read -P 3 2M 1M') == 0
+ assert qemu_io_silent(top_img_path, '-c', 'read -P 0 3M 1M') == 0
+ assert qemu_io_silent(top_img_path, '-c', 'read -P 3 4M 1M') == 0
+
+ log('Done')
diff --git a/tests/qemu-iotests/310.out b/tests/qemu-iotests/310.out
new file mode 100644
index 0000000000..a70aa5cdae
--- /dev/null
+++ b/tests/qemu-iotests/310.out
@@ -0,0 +1,15 @@
+
+=== Copy-on-read across nodes ===
+
+--- Setting up images ---
+
+Done
+
+--- Doing COR ---
+
+{"return": {}}
+{"return": ""}
+
+--- Checking COR result ---
+
+Done
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index bc5bc324fe..d4a3e36a9a 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -318,4 +318,5 @@
307 rw quick export
308 rw
309 rw auto quick
+310 rw quick
312 rw quick
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 08/53] block: include supported_read_flags into BDS structure
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (6 preceding siblings ...)
2021-01-26 14:19 ` [PULL 07/53] iotests: add #310 to test bottom node in COR driver Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 09/53] copy-on-read: skip non-guest reads if no copy needed Max Reitz
` (45 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Add the new member supported_read_flags to the BlockDriverState
structure. It will control the flags set for copy-on-read operations.
Make the block generic layer evaluate supported read flags before they
go to a block driver.
Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[vsementsov: use assert instead of abort]
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201216061703.70908-8-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
include/block/block_int.h | 4 ++++
block/io.c | 10 ++++++++--
2 files changed, 12 insertions(+), 2 deletions(-)
diff --git a/include/block/block_int.h b/include/block/block_int.h
index f3d0a0e60a..92d3754ead 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -881,6 +881,10 @@ struct BlockDriverState {
/* I/O Limits */
BlockLimits bl;
+ /*
+ * Flags honored during pread
+ */
+ unsigned int supported_read_flags;
/* Flags honored during pwrite (so far: BDRV_REQ_FUA,
* BDRV_REQ_WRITE_UNCHANGED).
* If a driver does not support BDRV_REQ_WRITE_UNCHANGED, those
diff --git a/block/io.c b/block/io.c
index 95b1c56c06..d203435a73 100644
--- a/block/io.c
+++ b/block/io.c
@@ -1453,6 +1453,9 @@ static int coroutine_fn bdrv_aligned_preadv(BdrvChild *child,
if (flags & BDRV_REQ_COPY_ON_READ) {
int64_t pnum;
+ /* The flag BDRV_REQ_COPY_ON_READ has reached its addressee */
+ flags &= ~BDRV_REQ_COPY_ON_READ;
+
ret = bdrv_is_allocated(bs, offset, bytes, &pnum);
if (ret < 0) {
goto out;
@@ -1474,9 +1477,11 @@ static int coroutine_fn bdrv_aligned_preadv(BdrvChild *child,
goto out;
}
+ assert(!(flags & ~bs->supported_read_flags));
+
max_bytes = ROUND_UP(MAX(0, total_bytes - offset), align);
if (bytes <= max_bytes && bytes <= max_transfer) {
- ret = bdrv_driver_preadv(bs, offset, bytes, qiov, qiov_offset, 0);
+ ret = bdrv_driver_preadv(bs, offset, bytes, qiov, qiov_offset, flags);
goto out;
}
@@ -1489,7 +1494,8 @@ static int coroutine_fn bdrv_aligned_preadv(BdrvChild *child,
ret = bdrv_driver_preadv(bs, offset + bytes - bytes_remaining,
num, qiov,
- qiov_offset + bytes - bytes_remaining, 0);
+ qiov_offset + bytes - bytes_remaining,
+ flags);
max_bytes -= num;
} else {
num = bytes_remaining;
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 09/53] copy-on-read: skip non-guest reads if no copy needed
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (7 preceding siblings ...)
2021-01-26 14:19 ` [PULL 08/53] block: include supported_read_flags into BDS structure Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 10/53] stream: rework backing-file changing Max Reitz
` (44 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
If the flag BDRV_REQ_PREFETCH was set, skip idling read/write
operations in COR-driver. It can be taken into account for the
COR-algorithms optimization. That check is being made during the
block stream job by the moment.
Add the BDRV_REQ_PREFETCH flag to the supported_read_flags of the
COR-filter.
block: Modify the comment for the flag BDRV_REQ_PREFETCH as we are
going to use it alone and pass it to the COR-filter driver for further
processing.
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201216061703.70908-9-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
include/block/block.h | 8 +++++---
block/copy-on-read.c | 14 ++++++++++----
2 files changed, 15 insertions(+), 7 deletions(-)
diff --git a/include/block/block.h b/include/block/block.h
index 127bdf3392..81fcaad5ac 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -72,9 +72,11 @@ typedef enum {
BDRV_REQ_NO_FALLBACK = 0x100,
/*
- * BDRV_REQ_PREFETCH may be used only together with BDRV_REQ_COPY_ON_READ
- * on read request and means that caller doesn't really need data to be
- * written to qiov parameter which may be NULL.
+ * BDRV_REQ_PREFETCH makes sense only in the context of copy-on-read
+ * (i.e., together with the BDRV_REQ_COPY_ON_READ flag or when a COR
+ * filter is involved), in which case it signals that the COR operation
+ * need not read the data into memory (qiov) but only ensure they are
+ * copied to the top layer (i.e., that COR operation is done).
*/
BDRV_REQ_PREFETCH = 0x200,
diff --git a/block/copy-on-read.c b/block/copy-on-read.c
index 71560984f6..9cad9e1b8c 100644
--- a/block/copy-on-read.c
+++ b/block/copy-on-read.c
@@ -50,6 +50,8 @@ static int cor_open(BlockDriverState *bs, QDict *options, int flags,
return -EINVAL;
}
+ bs->supported_read_flags = BDRV_REQ_PREFETCH;
+
bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED |
(BDRV_REQ_FUA & bs->file->bs->supported_write_flags);
@@ -172,10 +174,14 @@ static int coroutine_fn cor_co_preadv_part(BlockDriverState *bs,
}
}
- ret = bdrv_co_preadv_part(bs->file, offset, n, qiov, qiov_offset,
- local_flags);
- if (ret < 0) {
- return ret;
+ /* Skip if neither read nor write are needed */
+ if ((local_flags & (BDRV_REQ_PREFETCH | BDRV_REQ_COPY_ON_READ)) !=
+ BDRV_REQ_PREFETCH) {
+ ret = bdrv_co_preadv_part(bs->file, offset, n, qiov, qiov_offset,
+ local_flags);
+ if (ret < 0) {
+ return ret;
+ }
}
offset += n;
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 10/53] stream: rework backing-file changing
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (8 preceding siblings ...)
2021-01-26 14:19 ` [PULL 09/53] copy-on-read: skip non-guest reads if no copy needed Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 11/53] qapi: block-stream: add "bottom" argument Max Reitz
` (43 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Stream in stream_prepare calls bdrv_change_backing_file() to change
backing-file in the metadata of bs.
It may use either backing-file parameter given by user or just take
filename of base on job start.
Backing file format is determined by base on job finish.
There are some problems with this design, we solve only two by this
patch:
1. Consider scenario with backing-file unset. Current concept of stream
supports changing of the base during the job (we don't freeze link to
the base). So, we should not save base filename at job start,
- let's determine name of the base on job finish.
2. Using direct base to determine filename and format is not very good:
base node may be a filter, so its filename may be JSON, and format_name
is not good for storing into qcow2 metadata as backing file format.
- let's use unfiltered_base
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[vsementsov: change commit subject, change logic in stream_prepare]
Message-Id: <20201216061703.70908-10-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/stream.c | 9 +++++----
blockdev.c | 8 +-------
2 files changed, 6 insertions(+), 11 deletions(-)
diff --git a/block/stream.c b/block/stream.c
index 6e281c71ac..6a525a5edf 100644
--- a/block/stream.c
+++ b/block/stream.c
@@ -65,6 +65,7 @@ static int stream_prepare(Job *job)
BlockDriverState *bs = blk_bs(bjob->blk);
BlockDriverState *unfiltered_bs = bdrv_skip_filters(bs);
BlockDriverState *base = bdrv_filter_or_cow_bs(s->above_base);
+ BlockDriverState *unfiltered_base = bdrv_skip_filters(base);
Error *local_err = NULL;
int ret = 0;
@@ -73,10 +74,10 @@ static int stream_prepare(Job *job)
if (bdrv_cow_child(unfiltered_bs)) {
const char *base_id = NULL, *base_fmt = NULL;
- if (base) {
- base_id = s->backing_file_str;
- if (base->drv) {
- base_fmt = base->drv->format_name;
+ if (unfiltered_base) {
+ base_id = s->backing_file_str ?: unfiltered_base->filename;
+ if (unfiltered_base->drv) {
+ base_fmt = unfiltered_base->drv->format_name;
}
}
bdrv_set_backing_hd(unfiltered_bs, base, &local_err);
diff --git a/blockdev.c b/blockdev.c
index e496356e10..8c03de582c 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2511,7 +2511,6 @@ void qmp_block_stream(bool has_job_id, const char *job_id, const char *device,
BlockDriverState *base_bs = NULL;
AioContext *aio_context;
Error *local_err = NULL;
- const char *base_name = NULL;
int job_flags = JOB_DEFAULT;
if (!has_on_error) {
@@ -2539,7 +2538,6 @@ void qmp_block_stream(bool has_job_id, const char *job_id, const char *device,
goto out;
}
assert(bdrv_get_aio_context(base_bs) == aio_context);
- base_name = base;
}
if (has_base_node) {
@@ -2554,7 +2552,6 @@ void qmp_block_stream(bool has_job_id, const char *job_id, const char *device,
}
assert(bdrv_get_aio_context(base_bs) == aio_context);
bdrv_refresh_filename(base_bs);
- base_name = base_bs->filename;
}
/* Check for op blockers in the whole chain between bs and base */
@@ -2574,9 +2571,6 @@ void qmp_block_stream(bool has_job_id, const char *job_id, const char *device,
goto out;
}
- /* backing_file string overrides base bs filename */
- base_name = has_backing_file ? backing_file : base_name;
-
if (has_auto_finalize && !auto_finalize) {
job_flags |= JOB_MANUAL_FINALIZE;
}
@@ -2584,7 +2578,7 @@ void qmp_block_stream(bool has_job_id, const char *job_id, const char *device,
job_flags |= JOB_MANUAL_DISMISS;
}
- stream_start(has_job_id ? job_id : NULL, bs, base_bs, base_name,
+ stream_start(has_job_id ? job_id : NULL, bs, base_bs, backing_file,
job_flags, has_speed ? speed : 0, on_error,
filter_node_name, &local_err);
if (local_err) {
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 11/53] qapi: block-stream: add "bottom" argument
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (9 preceding siblings ...)
2021-01-26 14:19 ` [PULL 10/53] stream: rework backing-file changing Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 12/53] iotests: 30: prepare to COR filter insertion by stream job Max Reitz
` (42 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
The code already don't freeze base node and we try to make it prepared
for the situation when base node is changed during the operation. In
other words, block-stream doesn't own base node.
Let's introduce a new interface which should replace the current one,
which will in better relations with the code. Specifying bottom node
instead of base, and requiring it to be non-filter gives us the
following benefits:
- drop difference between above_base and base_overlay, which will be
renamed to just bottom, when old interface dropped
- clean way to work with parallel streams/commits on the same backing
chain, which otherwise become a problem when we introduce a filter
for stream job
- cleaner interface. Nobody will surprised the fact that base node may
disappear during block-stream, when there is no word about "base" in
the interface.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201216061703.70908-11-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
qapi/block-core.json | 12 ++++---
include/block/block_int.h | 1 +
block/monitor/block-hmp-cmds.c | 3 +-
block/stream.c | 50 +++++++++++++++++++---------
blockdev.c | 59 ++++++++++++++++++++++++++++------
5 files changed, 94 insertions(+), 31 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 65167ebf56..1d9dcd7d30 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2517,10 +2517,14 @@
# @device: the device or node name of the top image
#
# @base: the common backing file name.
-# It cannot be set if @base-node is also set.
+# It cannot be set if @base-node or @bottom is also set.
#
# @base-node: the node name of the backing file.
-# It cannot be set if @base is also set. (Since 2.8)
+# It cannot be set if @base or @bottom is also set. (Since 2.8)
+#
+# @bottom: the last node in the chain that should be streamed into
+# top. It cannot be set if @base or @base-node is also set.
+# It cannot be filter node. (Since 6.0)
#
# @backing-file: The backing file string to write into the top
# image. This filename is not validated.
@@ -2576,8 +2580,8 @@
##
{ 'command': 'block-stream',
'data': { '*job-id': 'str', 'device': 'str', '*base': 'str',
- '*base-node': 'str', '*backing-file': 'str', '*speed': 'int',
- '*on-error': 'BlockdevOnError',
+ '*base-node': 'str', '*backing-file': 'str', '*bottom': 'str',
+ '*speed': 'int', '*on-error': 'BlockdevOnError',
'*filter-node-name': 'str',
'*auto-finalize': 'bool', '*auto-dismiss': 'bool' } }
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 92d3754ead..f4b844f310 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -1161,6 +1161,7 @@ int is_windows_drive(const char *filename);
*/
void stream_start(const char *job_id, BlockDriverState *bs,
BlockDriverState *base, const char *backing_file_str,
+ BlockDriverState *bottom,
int creation_flags, int64_t speed,
BlockdevOnError on_error,
const char *filter_node_name,
diff --git a/block/monitor/block-hmp-cmds.c b/block/monitor/block-hmp-cmds.c
index e8a58f326e..afd75ab628 100644
--- a/block/monitor/block-hmp-cmds.c
+++ b/block/monitor/block-hmp-cmds.c
@@ -507,7 +507,8 @@ void hmp_block_stream(Monitor *mon, const QDict *qdict)
int64_t speed = qdict_get_try_int(qdict, "speed", 0);
qmp_block_stream(true, device, device, base != NULL, base, false, NULL,
- false, NULL, qdict_haskey(qdict, "speed"), speed, true,
+ false, NULL, false, NULL,
+ qdict_haskey(qdict, "speed"), speed, true,
BLOCKDEV_ON_ERROR_REPORT, false, NULL, false, false, false,
false, &error);
diff --git a/block/stream.c b/block/stream.c
index 6a525a5edf..045d6bc76b 100644
--- a/block/stream.c
+++ b/block/stream.c
@@ -221,6 +221,7 @@ static const BlockJobDriver stream_job_driver = {
void stream_start(const char *job_id, BlockDriverState *bs,
BlockDriverState *base, const char *backing_file_str,
+ BlockDriverState *bottom,
int creation_flags, int64_t speed,
BlockdevOnError on_error,
const char *filter_node_name,
@@ -230,25 +231,42 @@ void stream_start(const char *job_id, BlockDriverState *bs,
BlockDriverState *iter;
bool bs_read_only;
int basic_flags = BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE_UNCHANGED;
- BlockDriverState *base_overlay = bdrv_find_overlay(bs, base);
+ BlockDriverState *base_overlay;
BlockDriverState *above_base;
- if (!base_overlay) {
- error_setg(errp, "'%s' is not in the backing chain of '%s'",
- base->node_name, bs->node_name);
- return;
- }
+ assert(!(base && bottom));
+ assert(!(backing_file_str && bottom));
+
+ if (bottom) {
+ /*
+ * New simple interface. The code is written in terms of old interface
+ * with @base parameter (still, it doesn't freeze link to base, so in
+ * this mean old code is correct for new interface). So, for now, just
+ * emulate base_overlay and above_base. Still, when old interface
+ * finally removed, we should refactor code to use only "bottom", but
+ * not "*base*" things.
+ */
+ assert(!bottom->drv->is_filter);
+ base_overlay = above_base = bottom;
+ } else {
+ base_overlay = bdrv_find_overlay(bs, base);
+ if (!base_overlay) {
+ error_setg(errp, "'%s' is not in the backing chain of '%s'",
+ base->node_name, bs->node_name);
+ return;
+ }
- /*
- * Find the node directly above @base. @base_overlay is a COW overlay, so
- * it must have a bdrv_cow_child(), but it is the immediate overlay of
- * @base, so between the two there can only be filters.
- */
- above_base = base_overlay;
- if (bdrv_cow_bs(above_base) != base) {
- above_base = bdrv_cow_bs(above_base);
- while (bdrv_filter_bs(above_base) != base) {
- above_base = bdrv_filter_bs(above_base);
+ /*
+ * Find the node directly above @base. @base_overlay is a COW overlay,
+ * so it must have a bdrv_cow_child(), but it is the immediate overlay
+ * of @base, so between the two there can only be filters.
+ */
+ above_base = base_overlay;
+ if (bdrv_cow_bs(above_base) != base) {
+ above_base = bdrv_cow_bs(above_base);
+ while (bdrv_filter_bs(above_base) != base) {
+ above_base = bdrv_filter_bs(above_base);
+ }
}
}
diff --git a/blockdev.c b/blockdev.c
index 8c03de582c..0540c621da 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2500,6 +2500,7 @@ void qmp_block_stream(bool has_job_id, const char *job_id, const char *device,
bool has_base, const char *base,
bool has_base_node, const char *base_node,
bool has_backing_file, const char *backing_file,
+ bool has_bottom, const char *bottom,
bool has_speed, int64_t speed,
bool has_on_error, BlockdevOnError on_error,
bool has_filter_node_name, const char *filter_node_name,
@@ -2507,12 +2508,31 @@ void qmp_block_stream(bool has_job_id, const char *job_id, const char *device,
bool has_auto_dismiss, bool auto_dismiss,
Error **errp)
{
- BlockDriverState *bs, *iter;
+ BlockDriverState *bs, *iter, *iter_end;
BlockDriverState *base_bs = NULL;
+ BlockDriverState *bottom_bs = NULL;
AioContext *aio_context;
Error *local_err = NULL;
int job_flags = JOB_DEFAULT;
+ if (has_base && has_base_node) {
+ error_setg(errp, "'base' and 'base-node' cannot be specified "
+ "at the same time");
+ return;
+ }
+
+ if (has_base && has_bottom) {
+ error_setg(errp, "'base' and 'bottom' cannot be specified "
+ "at the same time");
+ return;
+ }
+
+ if (has_bottom && has_base_node) {
+ error_setg(errp, "'bottom' and 'base-node' cannot be specified "
+ "at the same time");
+ return;
+ }
+
if (!has_on_error) {
on_error = BLOCKDEV_ON_ERROR_REPORT;
}
@@ -2525,12 +2545,6 @@ void qmp_block_stream(bool has_job_id, const char *job_id, const char *device,
aio_context = bdrv_get_aio_context(bs);
aio_context_acquire(aio_context);
- if (has_base && has_base_node) {
- error_setg(errp, "'base' and 'base-node' cannot be specified "
- "at the same time");
- goto out;
- }
-
if (has_base) {
base_bs = bdrv_find_backing_image(bs, base);
if (base_bs == NULL) {
@@ -2554,8 +2568,33 @@ void qmp_block_stream(bool has_job_id, const char *job_id, const char *device,
bdrv_refresh_filename(base_bs);
}
- /* Check for op blockers in the whole chain between bs and base */
- for (iter = bs; iter && iter != base_bs;
+ if (has_bottom) {
+ bottom_bs = bdrv_lookup_bs(NULL, bottom, errp);
+ if (!bottom_bs) {
+ goto out;
+ }
+ if (!bottom_bs->drv) {
+ error_setg(errp, "Node '%s' is not open", bottom);
+ goto out;
+ }
+ if (bottom_bs->drv->is_filter) {
+ error_setg(errp, "Node '%s' is a filter, use a non-filter node "
+ "as 'bottom'", bottom);
+ goto out;
+ }
+ if (!bdrv_chain_contains(bs, bottom_bs)) {
+ error_setg(errp, "Node '%s' is not in a chain starting from '%s'",
+ bottom, device);
+ goto out;
+ }
+ assert(bdrv_get_aio_context(bottom_bs) == aio_context);
+ }
+
+ /*
+ * Check for op blockers in the whole chain between bs and base (or bottom)
+ */
+ iter_end = has_bottom ? bdrv_filter_or_cow_bs(bottom_bs) : base_bs;
+ for (iter = bs; iter && iter != iter_end;
iter = bdrv_filter_or_cow_bs(iter))
{
if (bdrv_op_is_blocked(iter, BLOCK_OP_TYPE_STREAM, errp)) {
@@ -2579,7 +2618,7 @@ void qmp_block_stream(bool has_job_id, const char *job_id, const char *device,
}
stream_start(has_job_id ? job_id : NULL, bs, base_bs, backing_file,
- job_flags, has_speed ? speed : 0, on_error,
+ bottom_bs, job_flags, has_speed ? speed : 0, on_error,
filter_node_name, &local_err);
if (local_err) {
error_propagate(errp, local_err);
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 12/53] iotests: 30: prepare to COR filter insertion by stream job
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (10 preceding siblings ...)
2021-01-26 14:19 ` [PULL 11/53] qapi: block-stream: add "bottom" argument Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 13/53] block/stream: add s->target_bs Max Reitz
` (41 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
test_stream_parallel run parallel stream jobs, intersecting so that top
of one is base of another. It's OK now, but it would be a problem if
insert the filter, as one job will want to use another job's filter as
above_base node.
Correct thing to do is move to new interface: "bottom" argument instead
of base. This guarantees that jobs don't intersect by their actions.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201216061703.70908-12-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
tests/qemu-iotests/030 | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030
index 890784b116..f8a692432c 100755
--- a/tests/qemu-iotests/030
+++ b/tests/qemu-iotests/030
@@ -246,7 +246,9 @@ class TestParallelOps(iotests.QMPTestCase):
node_name = 'node%d' % i
job_id = 'stream-%s' % node_name
pending_jobs.append(job_id)
- result = self.vm.qmp('block-stream', device=node_name, job_id=job_id, base=self.imgs[i-2], speed=1024)
+ result = self.vm.qmp('block-stream', device=node_name,
+ job_id=job_id, bottom=f'node{i-1}',
+ speed=1024)
self.assert_qmp(result, 'return', {})
for job in pending_jobs:
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 13/53] block/stream: add s->target_bs
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (11 preceding siblings ...)
2021-01-26 14:19 ` [PULL 12/53] iotests: 30: prepare to COR filter insertion by stream job Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 14/53] block: apply COR-filter to block-stream jobs Max Reitz
` (40 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Add a direct link to target bs for convenience and to simplify
following commit which will insert COR filter above target bs.
This is a part of original commit written by Andrey.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201216061703.70908-13-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/stream.c | 23 ++++++++++-------------
1 file changed, 10 insertions(+), 13 deletions(-)
diff --git a/block/stream.c b/block/stream.c
index 045d6bc76b..626dfa2b22 100644
--- a/block/stream.c
+++ b/block/stream.c
@@ -33,6 +33,7 @@ typedef struct StreamBlockJob {
BlockJob common;
BlockDriverState *base_overlay; /* COW overlay (stream from this) */
BlockDriverState *above_base; /* Node directly above the base */
+ BlockDriverState *target_bs;
BlockdevOnError on_error;
char *backing_file_str;
bool bs_read_only;
@@ -53,23 +54,20 @@ static void stream_abort(Job *job)
StreamBlockJob *s = container_of(job, StreamBlockJob, common.job);
if (s->chain_frozen) {
- BlockJob *bjob = &s->common;
- bdrv_unfreeze_backing_chain(blk_bs(bjob->blk), s->above_base);
+ bdrv_unfreeze_backing_chain(s->target_bs, s->above_base);
}
}
static int stream_prepare(Job *job)
{
StreamBlockJob *s = container_of(job, StreamBlockJob, common.job);
- BlockJob *bjob = &s->common;
- BlockDriverState *bs = blk_bs(bjob->blk);
- BlockDriverState *unfiltered_bs = bdrv_skip_filters(bs);
+ BlockDriverState *unfiltered_bs = bdrv_skip_filters(s->target_bs);
BlockDriverState *base = bdrv_filter_or_cow_bs(s->above_base);
BlockDriverState *unfiltered_base = bdrv_skip_filters(base);
Error *local_err = NULL;
int ret = 0;
- bdrv_unfreeze_backing_chain(bs, s->above_base);
+ bdrv_unfreeze_backing_chain(s->target_bs, s->above_base);
s->chain_frozen = false;
if (bdrv_cow_child(unfiltered_bs)) {
@@ -95,13 +93,12 @@ static void stream_clean(Job *job)
{
StreamBlockJob *s = container_of(job, StreamBlockJob, common.job);
BlockJob *bjob = &s->common;
- BlockDriverState *bs = blk_bs(bjob->blk);
/* Reopen the image back in read-only mode if necessary */
if (s->bs_read_only) {
/* Give up write permissions before making it read-only */
blk_set_perm(bjob->blk, 0, BLK_PERM_ALL, &error_abort);
- bdrv_reopen_set_read_only(bs, true, NULL);
+ bdrv_reopen_set_read_only(s->target_bs, true, NULL);
}
g_free(s->backing_file_str);
@@ -111,8 +108,7 @@ static int coroutine_fn stream_run(Job *job, Error **errp)
{
StreamBlockJob *s = container_of(job, StreamBlockJob, common.job);
BlockBackend *blk = s->common.blk;
- BlockDriverState *bs = blk_bs(blk);
- BlockDriverState *unfiltered_bs = bdrv_skip_filters(bs);
+ BlockDriverState *unfiltered_bs = bdrv_skip_filters(s->target_bs);
bool enable_cor = !bdrv_cow_child(s->base_overlay);
int64_t len;
int64_t offset = 0;
@@ -125,7 +121,7 @@ static int coroutine_fn stream_run(Job *job, Error **errp)
return 0;
}
- len = bdrv_getlength(bs);
+ len = bdrv_getlength(s->target_bs);
if (len < 0) {
return len;
}
@@ -137,7 +133,7 @@ static int coroutine_fn stream_run(Job *job, Error **errp)
* account.
*/
if (enable_cor) {
- bdrv_enable_copy_on_read(bs);
+ bdrv_enable_copy_on_read(s->target_bs);
}
for ( ; offset < len; offset += n) {
@@ -199,7 +195,7 @@ static int coroutine_fn stream_run(Job *job, Error **errp)
}
if (enable_cor) {
- bdrv_disable_copy_on_read(bs);
+ bdrv_disable_copy_on_read(s->target_bs);
}
/* Do not remove the backing file if an error was there but ignored. */
@@ -314,6 +310,7 @@ void stream_start(const char *job_id, BlockDriverState *bs,
s->base_overlay = base_overlay;
s->above_base = above_base;
s->backing_file_str = g_strdup(backing_file_str);
+ s->target_bs = bs;
s->bs_read_only = bs_read_only;
s->chain_frozen = true;
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 14/53] block: apply COR-filter to block-stream jobs
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (12 preceding siblings ...)
2021-01-26 14:19 ` [PULL 13/53] block/stream: add s->target_bs Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-28 18:38 ` Philippe Mathieu-Daudé
2021-01-26 14:19 ` [PULL 15/53] iotests.py: Assume a couple of variables as given Max Reitz
` (39 subsequent siblings)
53 siblings, 1 reply; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
This patch completes the series with the COR-filter applied to
block-stream operations.
Adding the filter makes it possible in future implement discarding
copied regions in backing files during the block-stream job, to reduce
the disk overuse (we need control on permissions).
Also, the filter now is smart enough to do copy-on-read with specified
base, so we have benefit on guest reads even when doing block-stream of
the part of the backing chain.
Several iotests are slightly modified due to filter insertion.
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201216061703.70908-14-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/stream.c | 105 ++++++++++++++++++++++---------------
tests/qemu-iotests/030 | 8 +--
tests/qemu-iotests/141.out | 2 +-
tests/qemu-iotests/245 | 20 ++++---
4 files changed, 80 insertions(+), 55 deletions(-)
diff --git a/block/stream.c b/block/stream.c
index 626dfa2b22..1fa742b0db 100644
--- a/block/stream.c
+++ b/block/stream.c
@@ -17,8 +17,10 @@
#include "block/blockjob_int.h"
#include "qapi/error.h"
#include "qapi/qmp/qerror.h"
+#include "qapi/qmp/qdict.h"
#include "qemu/ratelimit.h"
#include "sysemu/block-backend.h"
+#include "block/copy-on-read.h"
enum {
/*
@@ -33,11 +35,11 @@ typedef struct StreamBlockJob {
BlockJob common;
BlockDriverState *base_overlay; /* COW overlay (stream from this) */
BlockDriverState *above_base; /* Node directly above the base */
+ BlockDriverState *cor_filter_bs;
BlockDriverState *target_bs;
BlockdevOnError on_error;
char *backing_file_str;
bool bs_read_only;
- bool chain_frozen;
} StreamBlockJob;
static int coroutine_fn stream_populate(BlockBackend *blk,
@@ -45,17 +47,7 @@ static int coroutine_fn stream_populate(BlockBackend *blk,
{
assert(bytes < SIZE_MAX);
- return blk_co_preadv(blk, offset, bytes, NULL,
- BDRV_REQ_COPY_ON_READ | BDRV_REQ_PREFETCH);
-}
-
-static void stream_abort(Job *job)
-{
- StreamBlockJob *s = container_of(job, StreamBlockJob, common.job);
-
- if (s->chain_frozen) {
- bdrv_unfreeze_backing_chain(s->target_bs, s->above_base);
- }
+ return blk_co_preadv(blk, offset, bytes, NULL, BDRV_REQ_PREFETCH);
}
static int stream_prepare(Job *job)
@@ -67,8 +59,9 @@ static int stream_prepare(Job *job)
Error *local_err = NULL;
int ret = 0;
- bdrv_unfreeze_backing_chain(s->target_bs, s->above_base);
- s->chain_frozen = false;
+ /* We should drop filter at this point, as filter hold the backing chain */
+ bdrv_cor_filter_drop(s->cor_filter_bs);
+ s->cor_filter_bs = NULL;
if (bdrv_cow_child(unfiltered_bs)) {
const char *base_id = NULL, *base_fmt = NULL;
@@ -94,6 +87,11 @@ static void stream_clean(Job *job)
StreamBlockJob *s = container_of(job, StreamBlockJob, common.job);
BlockJob *bjob = &s->common;
+ if (s->cor_filter_bs) {
+ bdrv_cor_filter_drop(s->cor_filter_bs);
+ s->cor_filter_bs = NULL;
+ }
+
/* Reopen the image back in read-only mode if necessary */
if (s->bs_read_only) {
/* Give up write permissions before making it read-only */
@@ -109,7 +107,6 @@ static int coroutine_fn stream_run(Job *job, Error **errp)
StreamBlockJob *s = container_of(job, StreamBlockJob, common.job);
BlockBackend *blk = s->common.blk;
BlockDriverState *unfiltered_bs = bdrv_skip_filters(s->target_bs);
- bool enable_cor = !bdrv_cow_child(s->base_overlay);
int64_t len;
int64_t offset = 0;
uint64_t delay_ns = 0;
@@ -127,15 +124,6 @@ static int coroutine_fn stream_run(Job *job, Error **errp)
}
job_progress_set_remaining(&s->common.job, len);
- /* Turn on copy-on-read for the whole block device so that guest read
- * requests help us make progress. Only do this when copying the entire
- * backing chain since the copy-on-read operation does not take base into
- * account.
- */
- if (enable_cor) {
- bdrv_enable_copy_on_read(s->target_bs);
- }
-
for ( ; offset < len; offset += n) {
bool copy;
int ret;
@@ -194,10 +182,6 @@ static int coroutine_fn stream_run(Job *job, Error **errp)
}
}
- if (enable_cor) {
- bdrv_disable_copy_on_read(s->target_bs);
- }
-
/* Do not remove the backing file if an error was there but ignored. */
return error;
}
@@ -209,7 +193,6 @@ static const BlockJobDriver stream_job_driver = {
.free = block_job_free,
.run = stream_run,
.prepare = stream_prepare,
- .abort = stream_abort,
.clean = stream_clean,
.user_resume = block_job_user_resume,
},
@@ -228,7 +211,9 @@ void stream_start(const char *job_id, BlockDriverState *bs,
bool bs_read_only;
int basic_flags = BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE_UNCHANGED;
BlockDriverState *base_overlay;
+ BlockDriverState *cor_filter_bs = NULL;
BlockDriverState *above_base;
+ QDict *opts;
assert(!(base && bottom));
assert(!(backing_file_str && bottom));
@@ -266,30 +251,62 @@ void stream_start(const char *job_id, BlockDriverState *bs,
}
}
- if (bdrv_freeze_backing_chain(bs, above_base, errp) < 0) {
- return;
- }
-
/* Make sure that the image is opened in read-write mode */
bs_read_only = bdrv_is_read_only(bs);
if (bs_read_only) {
- if (bdrv_reopen_set_read_only(bs, false, errp) != 0) {
- bs_read_only = false;
- goto fail;
+ int ret;
+ /* Hold the chain during reopen */
+ if (bdrv_freeze_backing_chain(bs, above_base, errp) < 0) {
+ return;
+ }
+
+ ret = bdrv_reopen_set_read_only(bs, false, errp);
+
+ /* failure, or cor-filter will hold the chain */
+ bdrv_unfreeze_backing_chain(bs, above_base);
+
+ if (ret < 0) {
+ return;
}
}
- /* Prevent concurrent jobs trying to modify the graph structure here, we
- * already have our own plans. Also don't allow resize as the image size is
- * queried only at the job start and then cached. */
- s = block_job_create(job_id, &stream_job_driver, NULL, bs,
- basic_flags | BLK_PERM_GRAPH_MOD,
+ opts = qdict_new();
+
+ qdict_put_str(opts, "driver", "copy-on-read");
+ qdict_put_str(opts, "file", bdrv_get_node_name(bs));
+ /* Pass the base_overlay node name as 'bottom' to COR driver */
+ qdict_put_str(opts, "bottom", base_overlay->node_name);
+ if (filter_node_name) {
+ qdict_put_str(opts, "node-name", filter_node_name);
+ }
+
+ cor_filter_bs = bdrv_insert_node(bs, opts, BDRV_O_RDWR, errp);
+ if (!cor_filter_bs) {
+ goto fail;
+ }
+
+ if (!filter_node_name) {
+ cor_filter_bs->implicit = true;
+ }
+
+ s = block_job_create(job_id, &stream_job_driver, NULL, cor_filter_bs,
+ BLK_PERM_CONSISTENT_READ,
basic_flags | BLK_PERM_WRITE,
speed, creation_flags, NULL, NULL, errp);
if (!s) {
goto fail;
}
+ /*
+ * Prevent concurrent jobs trying to modify the graph structure here, we
+ * already have our own plans. Also don't allow resize as the image size is
+ * queried only at the job start and then cached.
+ */
+ if (block_job_add_bdrv(&s->common, "active node", bs, 0,
+ basic_flags | BLK_PERM_WRITE, &error_abort)) {
+ goto fail;
+ }
+
/* Block all intermediate nodes between bs and base, because they will
* disappear from the chain after this operation. The streaming job reads
* every block only once, assuming that it doesn't change, so forbid writes
@@ -310,9 +327,9 @@ void stream_start(const char *job_id, BlockDriverState *bs,
s->base_overlay = base_overlay;
s->above_base = above_base;
s->backing_file_str = g_strdup(backing_file_str);
+ s->cor_filter_bs = cor_filter_bs;
s->target_bs = bs;
s->bs_read_only = bs_read_only;
- s->chain_frozen = true;
s->on_error = on_error;
trace_stream_start(bs, base, s);
@@ -320,8 +337,10 @@ void stream_start(const char *job_id, BlockDriverState *bs,
return;
fail:
+ if (cor_filter_bs) {
+ bdrv_cor_filter_drop(cor_filter_bs);
+ }
if (bs_read_only) {
bdrv_reopen_set_read_only(bs, true, NULL);
}
- bdrv_unfreeze_backing_chain(bs, above_base);
}
diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030
index f8a692432c..832fe4a1e2 100755
--- a/tests/qemu-iotests/030
+++ b/tests/qemu-iotests/030
@@ -279,12 +279,14 @@ class TestParallelOps(iotests.QMPTestCase):
self.assert_no_active_block_jobs()
# Set a speed limit to make sure that this job blocks the rest
- result = self.vm.qmp('block-stream', device='node4', job_id='stream-node4', base=self.imgs[1], speed=1024*1024)
+ result = self.vm.qmp('block-stream', device='node4',
+ job_id='stream-node4', base=self.imgs[1],
+ filter_node_name='stream-filter', speed=1024*1024)
self.assert_qmp(result, 'return', {})
result = self.vm.qmp('block-stream', device='node5', job_id='stream-node5', base=self.imgs[2])
self.assert_qmp(result, 'error/desc',
- "Node 'node4' is busy: block device is in use by block job: stream")
+ "Node 'stream-filter' is busy: block device is in use by block job: stream")
result = self.vm.qmp('block-stream', device='node3', job_id='stream-node3', base=self.imgs[2])
self.assert_qmp(result, 'error/desc',
@@ -297,7 +299,7 @@ class TestParallelOps(iotests.QMPTestCase):
# block-commit should also fail if it touches nodes used by the stream job
result = self.vm.qmp('block-commit', device='drive0', base=self.imgs[4], job_id='commit-node4')
self.assert_qmp(result, 'error/desc',
- "Node 'node4' is busy: block device is in use by block job: stream")
+ "Node 'stream-filter' is busy: block device is in use by block job: stream")
result = self.vm.qmp('block-commit', device='drive0', base=self.imgs[1], top=self.imgs[3], job_id='commit-node1')
self.assert_qmp(result, 'error/desc',
diff --git a/tests/qemu-iotests/141.out b/tests/qemu-iotests/141.out
index 6d8652e22b..c4c15fb275 100644
--- a/tests/qemu-iotests/141.out
+++ b/tests/qemu-iotests/141.out
@@ -165,7 +165,7 @@ wrote 1048576/1048576 bytes at offset 0
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}}
{'execute': 'blockdev-del',
'arguments': {'node-name': 'drv0'}}
-{"error": {"class": "GenericError", "desc": "Node drv0 is in use"}}
+{"error": {"class": "GenericError", "desc": "Node 'drv0' is busy: block device is in use by block job: stream"}}
{'execute': 'block-job-cancel',
'arguments': {'device': 'job0'}}
{"return": {}}
diff --git a/tests/qemu-iotests/245 b/tests/qemu-iotests/245
index 86f00f290f..cfdeb902be 100755
--- a/tests/qemu-iotests/245
+++ b/tests/qemu-iotests/245
@@ -893,20 +893,24 @@ class TestBlockdevReopen(iotests.QMPTestCase):
# hd1 <- hd0
result = self.vm.qmp('block-stream', conv_keys = True, job_id = 'stream0',
- device = 'hd1', auto_finalize = False)
+ device = 'hd1', filter_node_name='cor',
+ auto_finalize = False)
self.assert_qmp(result, 'return', {})
- # We can't reopen with the original options because that would
- # make hd1 read-only and block-stream requires it to be read-write
- # (Which error message appears depends on whether the stream job is
- # already done with copying at this point.)
+ # We can't reopen with the original options because there is a filter
+ # inserted by stream job above hd1.
self.reopen(opts, {},
- ["Can't set node 'hd1' to r/o with copy-on-read enabled",
- "Cannot make block node read-only, there is a writer on it"])
+ "Cannot change the option 'backing.backing.file.node-name'")
+
+ # We can't reopen hd1 to read-only, as block-stream requires it to be
+ # read-write
+ self.reopen(opts['backing'], {'read-only': True},
+ "Cannot make block node read-only, there is a writer on it")
# We can't remove hd2 while the stream job is ongoing
opts['backing']['backing'] = None
- self.reopen(opts, {'backing.read-only': False}, "Cannot change 'backing' link from 'hd1' to 'hd2'")
+ self.reopen(opts['backing'], {'read-only': False},
+ "Cannot change 'backing' link from 'hd1' to 'hd2'")
# We can detach hd1 from hd0 because it doesn't affect the stream job
opts['backing'] = None
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* Re: [PULL 14/53] block: apply COR-filter to block-stream jobs
2021-01-26 14:19 ` [PULL 14/53] block: apply COR-filter to block-stream jobs Max Reitz
@ 2021-01-28 18:38 ` Philippe Mathieu-Daudé
2021-01-29 5:26 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 59+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-01-28 18:38 UTC (permalink / raw)
To: Max Reitz, qemu-block, Andrey Shinkevich
Cc: Kevin Wolf, Peter Maydell, qemu-devel
Hi Andrey,
On 1/26/21 3:19 PM, Max Reitz wrote:
> From: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
>
> This patch completes the series with the COR-filter applied to
> block-stream operations.
>
> Adding the filter makes it possible in future implement discarding
> copied regions in backing files during the block-stream job, to reduce
> the disk overuse (we need control on permissions).
>
> Also, the filter now is smart enough to do copy-on-read with specified
> base, so we have benefit on guest reads even when doing block-stream of
> the part of the backing chain.
>
> Several iotests are slightly modified due to filter insertion.
>
> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> Message-Id: <20201216061703.70908-14-vsementsov@virtuozzo.com>
> Reviewed-by: Max Reitz <mreitz@redhat.com>
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> block/stream.c | 105 ++++++++++++++++++++++---------------
> tests/qemu-iotests/030 | 8 +--
> tests/qemu-iotests/141.out | 2 +-
> tests/qemu-iotests/245 | 20 ++++---
> 4 files changed, 80 insertions(+), 55 deletions(-)
>
> diff --git a/block/stream.c b/block/stream.c
...
> @@ -228,7 +211,9 @@ void stream_start(const char *job_id, BlockDriverState *bs,
> bool bs_read_only;
> int basic_flags = BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE_UNCHANGED;
> BlockDriverState *base_overlay;
> + BlockDriverState *cor_filter_bs = NULL;
> BlockDriverState *above_base;
> + QDict *opts;
>
> assert(!(base && bottom));
> assert(!(backing_file_str && bottom));
> @@ -266,30 +251,62 @@ void stream_start(const char *job_id, BlockDriverState *bs,
> }
> }
>
> - if (bdrv_freeze_backing_chain(bs, above_base, errp) < 0) {
> - return;
> - }
> -
> /* Make sure that the image is opened in read-write mode */
> bs_read_only = bdrv_is_read_only(bs);
> if (bs_read_only) {
> - if (bdrv_reopen_set_read_only(bs, false, errp) != 0) {
> - bs_read_only = false;
> - goto fail;
> + int ret;
> + /* Hold the chain during reopen */
> + if (bdrv_freeze_backing_chain(bs, above_base, errp) < 0) {
> + return;
> + }
> +
> + ret = bdrv_reopen_set_read_only(bs, false, errp);
> +
> + /* failure, or cor-filter will hold the chain */
> + bdrv_unfreeze_backing_chain(bs, above_base);
> +
> + if (ret < 0) {
> + return;
> }
> }
>
> - /* Prevent concurrent jobs trying to modify the graph structure here, we
> - * already have our own plans. Also don't allow resize as the image size is
> - * queried only at the job start and then cached. */
> - s = block_job_create(job_id, &stream_job_driver, NULL, bs,
> - basic_flags | BLK_PERM_GRAPH_MOD,
> + opts = qdict_new();
Coverity reported (CID 1445793) that this resource could be leaked...
> +
> + qdict_put_str(opts, "driver", "copy-on-read");
> + qdict_put_str(opts, "file", bdrv_get_node_name(bs));
> + /* Pass the base_overlay node name as 'bottom' to COR driver */
> + qdict_put_str(opts, "bottom", base_overlay->node_name);
> + if (filter_node_name) {
> + qdict_put_str(opts, "node-name", filter_node_name);
> + }
> +
> + cor_filter_bs = bdrv_insert_node(bs, opts, BDRV_O_RDWR, errp);
> + if (!cor_filter_bs) {
... probably here.
Should we call g_free(opts) here?
> + goto fail;
> + }
> +
> + if (!filter_node_name) {
> + cor_filter_bs->implicit = true;
> + }
> +
> + s = block_job_create(job_id, &stream_job_driver, NULL, cor_filter_bs,
> + BLK_PERM_CONSISTENT_READ,
> basic_flags | BLK_PERM_WRITE,
> speed, creation_flags, NULL, NULL, errp);
> if (!s) {
> goto fail;
> }
>
> + /*
> + * Prevent concurrent jobs trying to modify the graph structure here, we
> + * already have our own plans. Also don't allow resize as the image size is
> + * queried only at the job start and then cached.
> + */
> + if (block_job_add_bdrv(&s->common, "active node", bs, 0,
> + basic_flags | BLK_PERM_WRITE, &error_abort)) {
> + goto fail;
> + }
> +
> /* Block all intermediate nodes between bs and base, because they will
> * disappear from the chain after this operation. The streaming job reads
> * every block only once, assuming that it doesn't change, so forbid writes
> @@ -310,9 +327,9 @@ void stream_start(const char *job_id, BlockDriverState *bs,
> s->base_overlay = base_overlay;
> s->above_base = above_base;
> s->backing_file_str = g_strdup(backing_file_str);
> + s->cor_filter_bs = cor_filter_bs;
> s->target_bs = bs;
> s->bs_read_only = bs_read_only;
> - s->chain_frozen = true;
>
> s->on_error = on_error;
> trace_stream_start(bs, base, s);
> @@ -320,8 +337,10 @@ void stream_start(const char *job_id, BlockDriverState *bs,
> return;
>
> fail:
> + if (cor_filter_bs) {
> + bdrv_cor_filter_drop(cor_filter_bs);
> + }
> if (bs_read_only) {
> bdrv_reopen_set_read_only(bs, true, NULL);
> }
> - bdrv_unfreeze_backing_chain(bs, above_base);
> }
...
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [PULL 14/53] block: apply COR-filter to block-stream jobs
2021-01-28 18:38 ` Philippe Mathieu-Daudé
@ 2021-01-29 5:26 ` Vladimir Sementsov-Ogievskiy
2021-01-29 8:23 ` Max Reitz
0 siblings, 1 reply; 59+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-29 5:26 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, Max Reitz, qemu-block,
Andrey Shinkevich
Cc: Kevin Wolf, Peter Maydell, qemu-devel
28.01.2021 21:38, Philippe Mathieu-Daudé wrote:
> Hi Andrey,
>
> On 1/26/21 3:19 PM, Max Reitz wrote:
>> From: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
>>
>> This patch completes the series with the COR-filter applied to
>> block-stream operations.
>>
>> Adding the filter makes it possible in future implement discarding
>> copied regions in backing files during the block-stream job, to reduce
>> the disk overuse (we need control on permissions).
>>
>> Also, the filter now is smart enough to do copy-on-read with specified
>> base, so we have benefit on guest reads even when doing block-stream of
>> the part of the backing chain.
>>
>> Several iotests are slightly modified due to filter insertion.
>>
>> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> Message-Id: <20201216061703.70908-14-vsementsov@virtuozzo.com>
>> Reviewed-by: Max Reitz <mreitz@redhat.com>
>> Signed-off-by: Max Reitz <mreitz@redhat.com>
>> ---
>> block/stream.c | 105 ++++++++++++++++++++++---------------
>> tests/qemu-iotests/030 | 8 +--
>> tests/qemu-iotests/141.out | 2 +-
>> tests/qemu-iotests/245 | 20 ++++---
>> 4 files changed, 80 insertions(+), 55 deletions(-)
>>
>> diff --git a/block/stream.c b/block/stream.c
> ...
>> @@ -228,7 +211,9 @@ void stream_start(const char *job_id, BlockDriverState *bs,
>> bool bs_read_only;
>> int basic_flags = BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE_UNCHANGED;
>> BlockDriverState *base_overlay;
>> + BlockDriverState *cor_filter_bs = NULL;
>> BlockDriverState *above_base;
>> + QDict *opts;
>>
>> assert(!(base && bottom));
>> assert(!(backing_file_str && bottom));
>> @@ -266,30 +251,62 @@ void stream_start(const char *job_id, BlockDriverState *bs,
>> }
>> }
>>
>> - if (bdrv_freeze_backing_chain(bs, above_base, errp) < 0) {
>> - return;
>> - }
>> -
>> /* Make sure that the image is opened in read-write mode */
>> bs_read_only = bdrv_is_read_only(bs);
>> if (bs_read_only) {
>> - if (bdrv_reopen_set_read_only(bs, false, errp) != 0) {
>> - bs_read_only = false;
>> - goto fail;
>> + int ret;
>> + /* Hold the chain during reopen */
>> + if (bdrv_freeze_backing_chain(bs, above_base, errp) < 0) {
>> + return;
>> + }
>> +
>> + ret = bdrv_reopen_set_read_only(bs, false, errp);
>> +
>> + /* failure, or cor-filter will hold the chain */
>> + bdrv_unfreeze_backing_chain(bs, above_base);
>> +
>> + if (ret < 0) {
>> + return;
>> }
>> }
>>
>> - /* Prevent concurrent jobs trying to modify the graph structure here, we
>> - * already have our own plans. Also don't allow resize as the image size is
>> - * queried only at the job start and then cached. */
>> - s = block_job_create(job_id, &stream_job_driver, NULL, bs,
>> - basic_flags | BLK_PERM_GRAPH_MOD,
>> + opts = qdict_new();
>
> Coverity reported (CID 1445793) that this resource could be leaked...
>
>> +
>> + qdict_put_str(opts, "driver", "copy-on-read");
>> + qdict_put_str(opts, "file", bdrv_get_node_name(bs));
>> + /* Pass the base_overlay node name as 'bottom' to COR driver */
>> + qdict_put_str(opts, "bottom", base_overlay->node_name);
>> + if (filter_node_name) {
>> + qdict_put_str(opts, "node-name", filter_node_name);
>> + }
>> +
>> + cor_filter_bs = bdrv_insert_node(bs, opts, BDRV_O_RDWR, errp);
>> + if (!cor_filter_bs) {
>
> ... probably here.
>
> Should we call g_free(opts) here?
Actually, not..
bdrv_insert_node() calls bdrv_open() which eats options even on failure.
I see, CID already marked false-positive?
>
>> + goto fail;
>> + }
>> +
>> + if (!filter_node_name) {
>> + cor_filter_bs->implicit = true;
>> + }
>> +
>> + s = block_job_create(job_id, &stream_job_driver, NULL, cor_filter_bs,
>> + BLK_PERM_CONSISTENT_READ,
>> basic_flags | BLK_PERM_WRITE,
>> speed, creation_flags, NULL, NULL, errp);
>> if (!s) {
>> goto fail;
>> }
>>
>> + /*
>> + * Prevent concurrent jobs trying to modify the graph structure here, we
>> + * already have our own plans. Also don't allow resize as the image size is
>> + * queried only at the job start and then cached.
>> + */
>> + if (block_job_add_bdrv(&s->common, "active node", bs, 0,
>> + basic_flags | BLK_PERM_WRITE, &error_abort)) {
>> + goto fail;
>> + }
>> +
>> /* Block all intermediate nodes between bs and base, because they will
>> * disappear from the chain after this operation. The streaming job reads
>> * every block only once, assuming that it doesn't change, so forbid writes
>> @@ -310,9 +327,9 @@ void stream_start(const char *job_id, BlockDriverState *bs,
>> s->base_overlay = base_overlay;
>> s->above_base = above_base;
>> s->backing_file_str = g_strdup(backing_file_str);
>> + s->cor_filter_bs = cor_filter_bs;
>> s->target_bs = bs;
>> s->bs_read_only = bs_read_only;
>> - s->chain_frozen = true;
>>
>> s->on_error = on_error;
>> trace_stream_start(bs, base, s);
>> @@ -320,8 +337,10 @@ void stream_start(const char *job_id, BlockDriverState *bs,
>> return;
>>
>> fail:
>> + if (cor_filter_bs) {
>> + bdrv_cor_filter_drop(cor_filter_bs);
>> + }
>> if (bs_read_only) {
>> bdrv_reopen_set_read_only(bs, true, NULL);
>> }
>> - bdrv_unfreeze_backing_chain(bs, above_base);
>> }
> ...
>
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [PULL 14/53] block: apply COR-filter to block-stream jobs
2021-01-29 5:26 ` Vladimir Sementsov-Ogievskiy
@ 2021-01-29 8:23 ` Max Reitz
2021-01-29 9:23 ` Philippe Mathieu-Daudé
0 siblings, 1 reply; 59+ messages in thread
From: Max Reitz @ 2021-01-29 8:23 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, Philippe Mathieu-Daudé,
qemu-block, Andrey Shinkevich
Cc: Kevin Wolf, Peter Maydell, qemu-devel
On 29.01.21 06:26, Vladimir Sementsov-Ogievskiy wrote:
> 28.01.2021 21:38, Philippe Mathieu-Daudé wrote:
>> Hi Andrey,
>>
>> On 1/26/21 3:19 PM, Max Reitz wrote:
>>> From: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
>>>
>>> This patch completes the series with the COR-filter applied to
>>> block-stream operations.
>>>
>>> Adding the filter makes it possible in future implement discarding
>>> copied regions in backing files during the block-stream job, to reduce
>>> the disk overuse (we need control on permissions).
>>>
>>> Also, the filter now is smart enough to do copy-on-read with specified
>>> base, so we have benefit on guest reads even when doing block-stream of
>>> the part of the backing chain.
>>>
>>> Several iotests are slightly modified due to filter insertion.
>>>
>>> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>> Message-Id: <20201216061703.70908-14-vsementsov@virtuozzo.com>
>>> Reviewed-by: Max Reitz <mreitz@redhat.com>
>>> Signed-off-by: Max Reitz <mreitz@redhat.com>
>>> ---
>>> block/stream.c | 105 ++++++++++++++++++++++---------------
>>> tests/qemu-iotests/030 | 8 +--
>>> tests/qemu-iotests/141.out | 2 +-
>>> tests/qemu-iotests/245 | 20 ++++---
>>> 4 files changed, 80 insertions(+), 55 deletions(-)
>>>
>>> diff --git a/block/stream.c b/block/stream.c
>> ...
>>> @@ -228,7 +211,9 @@ void stream_start(const char *job_id,
>>> BlockDriverState *bs,
>>> bool bs_read_only;
>>> int basic_flags = BLK_PERM_CONSISTENT_READ |
>>> BLK_PERM_WRITE_UNCHANGED;
>>> BlockDriverState *base_overlay;
>>> + BlockDriverState *cor_filter_bs = NULL;
>>> BlockDriverState *above_base;
>>> + QDict *opts;
>>> assert(!(base && bottom));
>>> assert(!(backing_file_str && bottom));
>>> @@ -266,30 +251,62 @@ void stream_start(const char *job_id,
>>> BlockDriverState *bs,
>>> }
>>> }
>>> - if (bdrv_freeze_backing_chain(bs, above_base, errp) < 0) {
>>> - return;
>>> - }
>>> -
>>> /* Make sure that the image is opened in read-write mode */
>>> bs_read_only = bdrv_is_read_only(bs);
>>> if (bs_read_only) {
>>> - if (bdrv_reopen_set_read_only(bs, false, errp) != 0) {
>>> - bs_read_only = false;
>>> - goto fail;
>>> + int ret;
>>> + /* Hold the chain during reopen */
>>> + if (bdrv_freeze_backing_chain(bs, above_base, errp) < 0) {
>>> + return;
>>> + }
>>> +
>>> + ret = bdrv_reopen_set_read_only(bs, false, errp);
>>> +
>>> + /* failure, or cor-filter will hold the chain */
>>> + bdrv_unfreeze_backing_chain(bs, above_base);
>>> +
>>> + if (ret < 0) {
>>> + return;
>>> }
>>> }
>>> - /* Prevent concurrent jobs trying to modify the graph structure
>>> here, we
>>> - * already have our own plans. Also don't allow resize as the
>>> image size is
>>> - * queried only at the job start and then cached. */
>>> - s = block_job_create(job_id, &stream_job_driver, NULL, bs,
>>> - basic_flags | BLK_PERM_GRAPH_MOD,
>>> + opts = qdict_new();
>>
>> Coverity reported (CID 1445793) that this resource could be leaked...
>>
>>> +
>>> + qdict_put_str(opts, "driver", "copy-on-read");
>>> + qdict_put_str(opts, "file", bdrv_get_node_name(bs));
>>> + /* Pass the base_overlay node name as 'bottom' to COR driver */
>>> + qdict_put_str(opts, "bottom", base_overlay->node_name);
>>> + if (filter_node_name) {
>>> + qdict_put_str(opts, "node-name", filter_node_name);
>>> + }
>>> +
>>> + cor_filter_bs = bdrv_insert_node(bs, opts, BDRV_O_RDWR, errp);
>>> + if (!cor_filter_bs) {
>>
>> ... probably here.
>>
>> Should we call g_free(opts) here?
>
>
> Actually, not..
>
> bdrv_insert_node() calls bdrv_open() which eats options even on failure.
>
> I see, CID already marked false-positive?
Yes, I did that.
This isn’t the first time Coverity has reported a failed bdrv_open()
call would leak the options QDict. Perhaps someone™ should look into
why it likes to thinks that, but so far I haven’t been sufficiently
bothered by it.
Max
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [PULL 14/53] block: apply COR-filter to block-stream jobs
2021-01-29 8:23 ` Max Reitz
@ 2021-01-29 9:23 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 59+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-01-29 9:23 UTC (permalink / raw)
To: Max Reitz, Vladimir Sementsov-Ogievskiy, qemu-block,
Andrey Shinkevich
Cc: Kevin Wolf, Peter Maydell, qemu-devel
On 1/29/21 9:23 AM, Max Reitz wrote:
> On 29.01.21 06:26, Vladimir Sementsov-Ogievskiy wrote:
>> 28.01.2021 21:38, Philippe Mathieu-Daudé wrote:
>>> Hi Andrey,
>>>
>>> On 1/26/21 3:19 PM, Max Reitz wrote:
>>>> From: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
>>>>
>>>> This patch completes the series with the COR-filter applied to
>>>> block-stream operations.
>>>>
>>>> Adding the filter makes it possible in future implement discarding
>>>> copied regions in backing files during the block-stream job, to reduce
>>>> the disk overuse (we need control on permissions).
>>>>
>>>> Also, the filter now is smart enough to do copy-on-read with specified
>>>> base, so we have benefit on guest reads even when doing block-stream of
>>>> the part of the backing chain.
>>>>
>>>> Several iotests are slightly modified due to filter insertion.
>>>>
>>>> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>> Message-Id: <20201216061703.70908-14-vsementsov@virtuozzo.com>
>>>> Reviewed-by: Max Reitz <mreitz@redhat.com>
>>>> Signed-off-by: Max Reitz <mreitz@redhat.com>
>>>> ---
>>>> block/stream.c | 105
>>>> ++++++++++++++++++++++---------------
>>>> tests/qemu-iotests/030 | 8 +--
>>>> tests/qemu-iotests/141.out | 2 +-
>>>> tests/qemu-iotests/245 | 20 ++++---
>>>> 4 files changed, 80 insertions(+), 55 deletions(-)
>>>>
>>>> diff --git a/block/stream.c b/block/stream.c
>>> ...
>>>> @@ -228,7 +211,9 @@ void stream_start(const char *job_id,
>>>> BlockDriverState *bs,
>>>> bool bs_read_only;
>>>> int basic_flags = BLK_PERM_CONSISTENT_READ |
>>>> BLK_PERM_WRITE_UNCHANGED;
>>>> BlockDriverState *base_overlay;
>>>> + BlockDriverState *cor_filter_bs = NULL;
>>>> BlockDriverState *above_base;
>>>> + QDict *opts;
>>>> assert(!(base && bottom));
>>>> assert(!(backing_file_str && bottom));
>>>> @@ -266,30 +251,62 @@ void stream_start(const char *job_id,
>>>> BlockDriverState *bs,
>>>> }
>>>> }
>>>> - if (bdrv_freeze_backing_chain(bs, above_base, errp) < 0) {
>>>> - return;
>>>> - }
>>>> -
>>>> /* Make sure that the image is opened in read-write mode */
>>>> bs_read_only = bdrv_is_read_only(bs);
>>>> if (bs_read_only) {
>>>> - if (bdrv_reopen_set_read_only(bs, false, errp) != 0) {
>>>> - bs_read_only = false;
>>>> - goto fail;
>>>> + int ret;
>>>> + /* Hold the chain during reopen */
>>>> + if (bdrv_freeze_backing_chain(bs, above_base, errp) < 0) {
>>>> + return;
>>>> + }
>>>> +
>>>> + ret = bdrv_reopen_set_read_only(bs, false, errp);
>>>> +
>>>> + /* failure, or cor-filter will hold the chain */
>>>> + bdrv_unfreeze_backing_chain(bs, above_base);
>>>> +
>>>> + if (ret < 0) {
>>>> + return;
>>>> }
>>>> }
>>>> - /* Prevent concurrent jobs trying to modify the graph structure
>>>> here, we
>>>> - * already have our own plans. Also don't allow resize as the
>>>> image size is
>>>> - * queried only at the job start and then cached. */
>>>> - s = block_job_create(job_id, &stream_job_driver, NULL, bs,
>>>> - basic_flags | BLK_PERM_GRAPH_MOD,
>>>> + opts = qdict_new();
>>>
>>> Coverity reported (CID 1445793) that this resource could be leaked...
>>>
>>>> +
>>>> + qdict_put_str(opts, "driver", "copy-on-read");
>>>> + qdict_put_str(opts, "file", bdrv_get_node_name(bs));
>>>> + /* Pass the base_overlay node name as 'bottom' to COR driver */
>>>> + qdict_put_str(opts, "bottom", base_overlay->node_name);
>>>> + if (filter_node_name) {
>>>> + qdict_put_str(opts, "node-name", filter_node_name);
>>>> + }
>>>> +
>>>> + cor_filter_bs = bdrv_insert_node(bs, opts, BDRV_O_RDWR, errp);
>>>> + if (!cor_filter_bs) {
>>>
>>> ... probably here.
>>>
>>> Should we call g_free(opts) here?
>>
>>
>> Actually, not..
>>
>> bdrv_insert_node() calls bdrv_open() which eats options even on failure.
Ah OK.
>>
>> I see, CID already marked false-positive?
>
> Yes, I did that.
Thanks Max!
>
> This isn’t the first time Coverity has reported a failed bdrv_open()
> call would leak the options QDict. Perhaps someone™ should look into
> why it likes to thinks that, but so far I haven’t been sufficiently
> bothered by it.
>
> Max
>
^ permalink raw reply [flat|nested] 59+ messages in thread
* [PULL 15/53] iotests.py: Assume a couple of variables as given
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (13 preceding siblings ...)
2021-01-26 14:19 ` [PULL 14/53] block: apply COR-filter to block-stream jobs Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 16/53] iotests/297: Rewrite in Python and extend reach Max Reitz
` (38 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
There are a couple of environment variables that we fetch with
os.environ.get() without supplying a default. Clearly they are required
and expected to be set by the ./check script (as evidenced by
execute_setup_common(), which checks for test_dir and
qemu_default_machine to be set, and aborts if they are not).
Using .get() this way has the disadvantage of returning an Optional[str]
type, which mypy will complain about when tests just assume these values
to be str.
Use [] instead, which raises a KeyError for environment variables that
are not set. When this exception is raised, catch it and move the abort
code from execute_setup_common() there.
Drop the 'assert iotests.sock_dir is not None' from iotest 300, because
that sort of thing is precisely what this patch wants to prevent.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210118105720.14824-2-mreitz@redhat.com>
---
tests/qemu-iotests/300 | 1 -
tests/qemu-iotests/iotests.py | 26 +++++++++++++-------------
2 files changed, 13 insertions(+), 14 deletions(-)
diff --git a/tests/qemu-iotests/300 b/tests/qemu-iotests/300
index 23aca59d9c..38ef5945b7 100755
--- a/tests/qemu-iotests/300
+++ b/tests/qemu-iotests/300
@@ -28,7 +28,6 @@ import qemu
BlockBitmapMapping = List[Dict[str, Union[str, List[Dict[str, str]]]]]
-assert iotests.sock_dir is not None
mig_sock = os.path.join(iotests.sock_dir, 'mig_sock')
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 2e89c0ab1a..45cb9bd288 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -75,12 +75,20 @@ qemu_opts = os.environ.get('QEMU_OPTIONS', '').strip().split(' ')
imgfmt = os.environ.get('IMGFMT', 'raw')
imgproto = os.environ.get('IMGPROTO', 'file')
-test_dir = os.environ.get('TEST_DIR')
-sock_dir = os.environ.get('SOCK_DIR')
output_dir = os.environ.get('OUTPUT_DIR', '.')
-cachemode = os.environ.get('CACHEMODE')
-aiomode = os.environ.get('AIOMODE')
-qemu_default_machine = os.environ.get('QEMU_DEFAULT_MACHINE')
+
+try:
+ test_dir = os.environ['TEST_DIR']
+ sock_dir = os.environ['SOCK_DIR']
+ cachemode = os.environ['CACHEMODE']
+ aiomode = os.environ['AIOMODE']
+ qemu_default_machine = os.environ['QEMU_DEFAULT_MACHINE']
+except KeyError:
+ # We are using these variables as proxies to indicate that we're
+ # not being run via "check". There may be other things set up by
+ # "check" that individual test cases rely on.
+ sys.stderr.write('Please run this test via the "check" script\n')
+ sys.exit(os.EX_USAGE)
socket_scm_helper = os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_helper')
@@ -1286,14 +1294,6 @@ def execute_setup_common(supported_fmts: Sequence[str] = (),
"""
# Note: Python 3.6 and pylint do not like 'Collection' so use 'Sequence'.
- # We are using TEST_DIR and QEMU_DEFAULT_MACHINE as proxies to
- # indicate that we're not being run via "check". There may be
- # other things set up by "check" that individual test cases rely
- # on.
- if test_dir is None or qemu_default_machine is None:
- sys.stderr.write('Please run this test via the "check" script\n')
- sys.exit(os.EX_USAGE)
-
debug = '-d' in sys.argv
if debug:
sys.argv.remove('-d')
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 16/53] iotests/297: Rewrite in Python and extend reach
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (14 preceding siblings ...)
2021-01-26 14:19 ` [PULL 15/53] iotests.py: Assume a couple of variables as given Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 17/53] iotests: Move try_remove to iotests.py Max Reitz
` (37 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
Instead of checking iotests.py only, check all Python files in the
qemu-iotests/ directory. Of course, most of them do not pass, so there
is an extensive skip list for now. (The only files that do pass are
209, 254, 283, and iotests.py.)
(Alternatively, we could have the opposite, i.e. an explicit list of
files that we do want to check, but I think it is better to check files
by default.)
Unless started in debug mode (./check -d), the output has no information
on which files are tested, so we will not have a problem e.g. with
backports, where some files may be missing when compared to upstream.
Besides the technical rewrite, some more things are changed:
- For the pylint invocation, PYTHONPATH is adjusted. This mirrors
setting MYPYPATH for mypy.
- Also, MYPYPATH is now derived from PYTHONPATH, so that we include
paths set by the environment. Maybe at some point we want to let the
check script add '../../python/' to PYTHONPATH so that iotests.py does
not need to do that.
- Passing --notes=FIXME,XXX to pylint suppresses warnings for TODO
comments. TODO is fine, we do not need 297 to complain about such
comments.
- The "Success" line from mypy's output is suppressed, because (A) it
does not add useful information, and (B) it would leak information
about the files having been tested to the reference output, which we
decidedly do not want.
Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210118105720.14824-3-mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
tests/qemu-iotests/297 | 112 +++++++++++++++++++++++++++++--------
tests/qemu-iotests/297.out | 5 +-
2 files changed, 92 insertions(+), 25 deletions(-)
diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297
index 85bc1c0c85..b138b0539c 100755
--- a/tests/qemu-iotests/297
+++ b/tests/qemu-iotests/297
@@ -1,4 +1,4 @@
-#!/usr/bin/env bash
+#!/usr/bin/env python3
# group: meta
#
# Copyright (C) 2020 Red Hat, Inc.
@@ -16,30 +16,98 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-seq=$(basename $0)
-echo "QA output created by $seq"
+import os
+import re
+import shutil
+import subprocess
+import sys
-status=1 # failure is the default!
+import iotests
-# get standard environment
-. ./common.rc
-if ! type -p "pylint-3" > /dev/null; then
- _notrun "pylint-3 not found"
-fi
-if ! type -p "mypy" > /dev/null; then
- _notrun "mypy not found"
-fi
+# TODO: Empty this list!
+SKIP_FILES = (
+ '030', '040', '041', '044', '045', '055', '056', '057', '065', '093',
+ '096', '118', '124', '129', '132', '136', '139', '147', '148', '149',
+ '151', '152', '155', '163', '165', '169', '194', '196', '199', '202',
+ '203', '205', '206', '207', '208', '210', '211', '212', '213', '216',
+ '218', '219', '222', '224', '228', '234', '235', '236', '237', '238',
+ '240', '242', '245', '246', '248', '255', '256', '257', '258', '260',
+ '262', '264', '266', '274', '277', '280', '281', '295', '296', '298',
+ '299', '300', '302', '303', '304', '307',
+ 'nbd-fault-injector.py', 'qcow2.py', 'qcow2_format.py', 'qed.py'
+)
-pylint-3 --score=n iotests.py
-MYPYPATH=../../python/ mypy --warn-unused-configs --disallow-subclassing-any \
- --disallow-any-generics --disallow-incomplete-defs \
- --disallow-untyped-decorators --no-implicit-optional \
- --warn-redundant-casts --warn-unused-ignores \
- --no-implicit-reexport iotests.py
+def is_python_file(filename):
+ if not os.path.isfile(filename):
+ return False
-# success, all done
-echo "*** done"
-rm -f $seq.full
-status=0
+ if filename.endswith('.py'):
+ return True
+
+ with open(filename) as f:
+ try:
+ first_line = f.readline()
+ return re.match('^#!.*python', first_line) is not None
+ except UnicodeDecodeError: # Ignore binary files
+ return False
+
+
+def run_linters():
+ files = [filename for filename in (set(os.listdir('.')) - set(SKIP_FILES))
+ if is_python_file(filename)]
+
+ iotests.logger.debug('Files to be checked:')
+ iotests.logger.debug(', '.join(sorted(files)))
+
+ print('=== pylint ===')
+ sys.stdout.flush()
+
+ # Todo notes are fine, but fixme's or xxx's should probably just be
+ # fixed (in tests, at least)
+ env = os.environ.copy()
+ qemu_module_path = os.path.join(os.path.dirname(__file__),
+ '..', '..', 'python')
+ try:
+ env['PYTHONPATH'] += os.pathsep + qemu_module_path
+ except KeyError:
+ env['PYTHONPATH'] = qemu_module_path
+ subprocess.run(('pylint-3', '--score=n', '--notes=FIXME,XXX', *files),
+ env=env, check=False)
+
+ print('=== mypy ===')
+ sys.stdout.flush()
+
+ # We have to call mypy separately for each file. Otherwise, it
+ # will interpret all given files as belonging together (i.e., they
+ # may not both define the same classes, etc.; most notably, they
+ # must not both define the __main__ module).
+ env['MYPYPATH'] = env['PYTHONPATH']
+ for filename in files:
+ p = subprocess.run(('mypy',
+ '--warn-unused-configs',
+ '--disallow-subclassing-any',
+ '--disallow-any-generics',
+ '--disallow-incomplete-defs',
+ '--disallow-untyped-decorators',
+ '--no-implicit-optional',
+ '--warn-redundant-casts',
+ '--warn-unused-ignores',
+ '--no-implicit-reexport',
+ filename),
+ env=env,
+ check=False,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.STDOUT,
+ universal_newlines=True)
+
+ if p.returncode != 0:
+ print(p.stdout)
+
+
+for linter in ('pylint-3', 'mypy'):
+ if shutil.which(linter) is None:
+ iotests.notrun(f'{linter} not found')
+
+iotests.script_main(run_linters)
diff --git a/tests/qemu-iotests/297.out b/tests/qemu-iotests/297.out
index 6acc843649..f2e1314d10 100644
--- a/tests/qemu-iotests/297.out
+++ b/tests/qemu-iotests/297.out
@@ -1,3 +1,2 @@
-QA output created by 297
-Success: no issues found in 1 source file
-*** done
+=== pylint ===
+=== mypy ===
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 17/53] iotests: Move try_remove to iotests.py
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (15 preceding siblings ...)
2021-01-26 14:19 ` [PULL 16/53] iotests/297: Rewrite in Python and extend reach Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 18/53] iotests/129: Remove test images in tearDown() Max Reitz
` (36 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210118105720.14824-4-mreitz@redhat.com>
---
tests/qemu-iotests/124 | 8 +-------
tests/qemu-iotests/iotests.py | 11 +++++++----
2 files changed, 8 insertions(+), 11 deletions(-)
diff --git a/tests/qemu-iotests/124 b/tests/qemu-iotests/124
index 3b21bc497f..90cdbd8e24 100755
--- a/tests/qemu-iotests/124
+++ b/tests/qemu-iotests/124
@@ -23,6 +23,7 @@
import os
import iotests
+from iotests import try_remove
def io_write_patterns(img, patterns):
@@ -30,13 +31,6 @@ def io_write_patterns(img, patterns):
iotests.qemu_io('-c', 'write -P%s %s %s' % pattern, img)
-def try_remove(img):
- try:
- os.remove(img)
- except OSError:
- pass
-
-
def transaction_action(action, **kwargs):
return {
'type': action,
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 45cb9bd288..335e6feb70 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -515,12 +515,15 @@ class FilePath:
return False
+def try_remove(img):
+ try:
+ os.remove(img)
+ except OSError:
+ pass
+
def file_path_remover():
for path in reversed(file_path_remover.paths):
- try:
- os.remove(path)
- except OSError:
- pass
+ try_remove(path)
def file_path(*names, base_dir=test_dir):
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 18/53] iotests/129: Remove test images in tearDown()
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (16 preceding siblings ...)
2021-01-26 14:19 ` [PULL 17/53] iotests: Move try_remove to iotests.py Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 19/53] iotests/129: Do not check @busy Max Reitz
` (35 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210118105720.14824-5-mreitz@redhat.com>
---
tests/qemu-iotests/129 | 2 ++
1 file changed, 2 insertions(+)
diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129
index f57a2e19f6..bd29c54af8 100755
--- a/tests/qemu-iotests/129
+++ b/tests/qemu-iotests/129
@@ -48,6 +48,8 @@ class TestStopWithBlockJob(iotests.QMPTestCase):
result = self.vm.qmp("block_set_io_throttle", conv_keys=False,
**params)
self.vm.shutdown()
+ for img in (self.test_img, self.target_img, self.base_img):
+ iotests.try_remove(img)
def do_test_stop(self, cmd, **args):
"""Test 'stop' while block job is running on a throttled drive.
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 19/53] iotests/129: Do not check @busy
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (17 preceding siblings ...)
2021-01-26 14:19 ` [PULL 18/53] iotests/129: Remove test images in tearDown() Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 20/53] iotests/129: Use throttle node Max Reitz
` (34 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
@busy is false when the job is paused, which happens all the time
because that is how jobs yield (e.g. for mirror at least since commit
565ac01f8d3).
Back when 129 was added (2015), perhaps there was no better way of
checking whether the job was still actually running. Now we have the
@status field (as of 58b295ba52c, i.e. 2018), which can give us exactly
that information.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210118105720.14824-6-mreitz@redhat.com>
---
tests/qemu-iotests/129 | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129
index bd29c54af8..0f2e5418ef 100755
--- a/tests/qemu-iotests/129
+++ b/tests/qemu-iotests/129
@@ -70,7 +70,7 @@ class TestStopWithBlockJob(iotests.QMPTestCase):
result = self.vm.qmp("stop")
self.assert_qmp(result, 'return', {})
result = self.vm.qmp("query-block-jobs")
- self.assert_qmp(result, 'return[0]/busy', True)
+ self.assert_qmp(result, 'return[0]/status', 'running')
self.assert_qmp(result, 'return[0]/ready', False)
def test_drive_mirror(self):
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 20/53] iotests/129: Use throttle node
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (18 preceding siblings ...)
2021-01-26 14:19 ` [PULL 19/53] iotests/129: Do not check @busy Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 21/53] iotests/129: Actually test a commit job Max Reitz
` (33 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
Throttling on the BB has not affected block jobs in a while, so it is
possible that one of the jobs in 129 finishes before the VM is stopped.
We can fix that by running the job from a throttle node.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210118105720.14824-7-mreitz@redhat.com>
---
tests/qemu-iotests/129 | 37 +++++++++++++------------------------
1 file changed, 13 insertions(+), 24 deletions(-)
diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129
index 0f2e5418ef..14c057f8ad 100755
--- a/tests/qemu-iotests/129
+++ b/tests/qemu-iotests/129
@@ -33,20 +33,18 @@ class TestStopWithBlockJob(iotests.QMPTestCase):
iotests.qemu_img('create', '-f', iotests.imgfmt, self.test_img,
"-b", self.base_img, '-F', iotests.imgfmt)
iotests.qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x5d 1M 128M', self.test_img)
- self.vm = iotests.VM().add_drive(self.test_img)
+ self.vm = iotests.VM()
+ self.vm.add_object('throttle-group,id=tg0,x-bps-total=1024')
+
+ source_drive = 'driver=throttle,' \
+ 'throttle-group=tg0,' \
+ f'file.driver={iotests.imgfmt},' \
+ f'file.file.filename={self.test_img}'
+
+ self.vm.add_drive(None, source_drive)
self.vm.launch()
def tearDown(self):
- params = {"device": "drive0",
- "bps": 0,
- "bps_rd": 0,
- "bps_wr": 0,
- "iops": 0,
- "iops_rd": 0,
- "iops_wr": 0,
- }
- result = self.vm.qmp("block_set_io_throttle", conv_keys=False,
- **params)
self.vm.shutdown()
for img in (self.test_img, self.target_img, self.base_img):
iotests.try_remove(img)
@@ -54,33 +52,24 @@ class TestStopWithBlockJob(iotests.QMPTestCase):
def do_test_stop(self, cmd, **args):
"""Test 'stop' while block job is running on a throttled drive.
The 'stop' command shouldn't drain the job"""
- params = {"device": "drive0",
- "bps": 1024,
- "bps_rd": 0,
- "bps_wr": 0,
- "iops": 0,
- "iops_rd": 0,
- "iops_wr": 0,
- }
- result = self.vm.qmp("block_set_io_throttle", conv_keys=False,
- **params)
- self.assert_qmp(result, 'return', {})
result = self.vm.qmp(cmd, **args)
self.assert_qmp(result, 'return', {})
+
result = self.vm.qmp("stop")
self.assert_qmp(result, 'return', {})
result = self.vm.qmp("query-block-jobs")
+
self.assert_qmp(result, 'return[0]/status', 'running')
self.assert_qmp(result, 'return[0]/ready', False)
def test_drive_mirror(self):
self.do_test_stop("drive-mirror", device="drive0",
- target=self.target_img,
+ target=self.target_img, format=iotests.imgfmt,
sync="full")
def test_drive_backup(self):
self.do_test_stop("drive-backup", device="drive0",
- target=self.target_img,
+ target=self.target_img, format=iotests.imgfmt,
sync="full")
def test_block_commit(self):
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 21/53] iotests/129: Actually test a commit job
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (19 preceding siblings ...)
2021-01-26 14:19 ` [PULL 20/53] iotests/129: Use throttle node Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 22/53] iotests/129: Limit mirror job's buffer size Max Reitz
` (32 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
Before this patch, test_block_commit() performs an active commit, which
under the hood is a mirror job. If we want to test various different
block jobs, we should perhaps run an actual commit job instead.
Doing so requires adding an overlay above the source node before the
commit is done (and then specifying the source node as the top node for
the commit job).
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210118105720.14824-8-mreitz@redhat.com>
---
tests/qemu-iotests/129 | 27 +++++++++++++++++++++++++--
1 file changed, 25 insertions(+), 2 deletions(-)
diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129
index 14c057f8ad..f4b447b04a 100755
--- a/tests/qemu-iotests/129
+++ b/tests/qemu-iotests/129
@@ -27,6 +27,7 @@ class TestStopWithBlockJob(iotests.QMPTestCase):
test_img = os.path.join(iotests.test_dir, 'test.img')
target_img = os.path.join(iotests.test_dir, 'target.img')
base_img = os.path.join(iotests.test_dir, 'base.img')
+ overlay_img = os.path.join(iotests.test_dir, 'overlay.img')
def setUp(self):
iotests.qemu_img('create', '-f', iotests.imgfmt, self.base_img, "1G")
@@ -37,6 +38,7 @@ class TestStopWithBlockJob(iotests.QMPTestCase):
self.vm.add_object('throttle-group,id=tg0,x-bps-total=1024')
source_drive = 'driver=throttle,' \
+ 'node-name=source,' \
'throttle-group=tg0,' \
f'file.driver={iotests.imgfmt},' \
f'file.file.filename={self.test_img}'
@@ -46,7 +48,8 @@ class TestStopWithBlockJob(iotests.QMPTestCase):
def tearDown(self):
self.vm.shutdown()
- for img in (self.test_img, self.target_img, self.base_img):
+ for img in (self.test_img, self.target_img, self.base_img,
+ self.overlay_img):
iotests.try_remove(img)
def do_test_stop(self, cmd, **args):
@@ -73,7 +76,27 @@ class TestStopWithBlockJob(iotests.QMPTestCase):
sync="full")
def test_block_commit(self):
- self.do_test_stop("block-commit", device="drive0")
+ # Add overlay above the source node so that we actually use a
+ # commit job instead of a mirror job
+
+ iotests.qemu_img('create', '-f', iotests.imgfmt, self.overlay_img,
+ '1G')
+
+ result = self.vm.qmp('blockdev-add', **{
+ 'node-name': 'overlay',
+ 'driver': iotests.imgfmt,
+ 'file': {
+ 'driver': 'file',
+ 'filename': self.overlay_img
+ }
+ })
+ self.assert_qmp(result, 'return', {})
+
+ result = self.vm.qmp('blockdev-snapshot',
+ node='source', overlay='overlay')
+ self.assert_qmp(result, 'return', {})
+
+ self.do_test_stop('block-commit', device='drive0', top_node='source')
if __name__ == '__main__':
iotests.main(supported_fmts=["qcow2"],
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 22/53] iotests/129: Limit mirror job's buffer size
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (20 preceding siblings ...)
2021-01-26 14:19 ` [PULL 21/53] iotests/129: Actually test a commit job Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 23/53] iotests/129: Clean up pylint and mypy complaints Max Reitz
` (31 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
Issuing 'stop' on the VM drains all nodes. If the mirror job has many
large requests in flight, this may lead to significant I/O that looks a
bit like 'stop' would make the job try to complete (which is what 129
should verify not to happen).
We can limit the I/O in flight by limiting the buffer size, so mirror
will make very little progress during the 'stop' drain.
(We do not need to do anything about commit, which has a buffer size of
512 kB by default; or backup, which goes cluster by cluster. Once we
have asynchronous requests for backup, that will change, but then we can
fine-tune the backup job to only perform a single request on a very
small chunk, too.)
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210118105720.14824-9-mreitz@redhat.com>
---
tests/qemu-iotests/129 | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129
index f4b447b04a..03843fc255 100755
--- a/tests/qemu-iotests/129
+++ b/tests/qemu-iotests/129
@@ -68,7 +68,7 @@ class TestStopWithBlockJob(iotests.QMPTestCase):
def test_drive_mirror(self):
self.do_test_stop("drive-mirror", device="drive0",
target=self.target_img, format=iotests.imgfmt,
- sync="full")
+ sync="full", buf_size=65536)
def test_drive_backup(self):
self.do_test_stop("drive-backup", device="drive0",
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 23/53] iotests/129: Clean up pylint and mypy complaints
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (21 preceding siblings ...)
2021-01-26 14:19 ` [PULL 22/53] iotests/129: Limit mirror job's buffer size Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 24/53] iotests/300: " Max Reitz
` (30 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
And consequentially drop it from 297's skip list.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210118105720.14824-10-mreitz@redhat.com>
---
tests/qemu-iotests/129 | 4 ++--
tests/qemu-iotests/297 | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129
index 03843fc255..2f7b28d4a0 100755
--- a/tests/qemu-iotests/129
+++ b/tests/qemu-iotests/129
@@ -21,7 +21,6 @@
import os
import iotests
-import time
class TestStopWithBlockJob(iotests.QMPTestCase):
test_img = os.path.join(iotests.test_dir, 'test.img')
@@ -33,7 +32,8 @@ class TestStopWithBlockJob(iotests.QMPTestCase):
iotests.qemu_img('create', '-f', iotests.imgfmt, self.base_img, "1G")
iotests.qemu_img('create', '-f', iotests.imgfmt, self.test_img,
"-b", self.base_img, '-F', iotests.imgfmt)
- iotests.qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x5d 1M 128M', self.test_img)
+ iotests.qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x5d 1M 128M',
+ self.test_img)
self.vm = iotests.VM()
self.vm.add_object('throttle-group,id=tg0,x-bps-total=1024')
diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297
index b138b0539c..79e63f8625 100755
--- a/tests/qemu-iotests/297
+++ b/tests/qemu-iotests/297
@@ -28,7 +28,7 @@ import iotests
# TODO: Empty this list!
SKIP_FILES = (
'030', '040', '041', '044', '045', '055', '056', '057', '065', '093',
- '096', '118', '124', '129', '132', '136', '139', '147', '148', '149',
+ '096', '118', '124', '132', '136', '139', '147', '148', '149',
'151', '152', '155', '163', '165', '169', '194', '196', '199', '202',
'203', '205', '206', '207', '208', '210', '211', '212', '213', '216',
'218', '219', '222', '224', '228', '234', '235', '236', '237', '238',
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 24/53] iotests/300: Clean up pylint and mypy complaints
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (22 preceding siblings ...)
2021-01-26 14:19 ` [PULL 23/53] iotests/129: Clean up pylint and mypy complaints Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 25/53] coroutine-sigaltstack: Add SIGUSR2 mutex Max Reitz
` (29 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
And consequentially drop it from 297's skip list.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210118105720.14824-11-mreitz@redhat.com>
---
tests/qemu-iotests/297 | 2 +-
tests/qemu-iotests/300 | 18 +++++++++++++++---
2 files changed, 16 insertions(+), 4 deletions(-)
diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297
index 79e63f8625..a37910b42d 100755
--- a/tests/qemu-iotests/297
+++ b/tests/qemu-iotests/297
@@ -34,7 +34,7 @@ SKIP_FILES = (
'218', '219', '222', '224', '228', '234', '235', '236', '237', '238',
'240', '242', '245', '246', '248', '255', '256', '257', '258', '260',
'262', '264', '266', '274', '277', '280', '281', '295', '296', '298',
- '299', '300', '302', '303', '304', '307',
+ '299', '302', '303', '304', '307',
'nbd-fault-injector.py', 'qcow2.py', 'qcow2_format.py', 'qed.py'
)
diff --git a/tests/qemu-iotests/300 b/tests/qemu-iotests/300
index 38ef5945b7..43264d883d 100755
--- a/tests/qemu-iotests/300
+++ b/tests/qemu-iotests/300
@@ -23,7 +23,11 @@ import os
import random
import re
from typing import Dict, List, Optional, Union
+
import iotests
+
+# Import qemu after iotests.py has amended sys.path
+# pylint: disable=wrong-import-order
import qemu
BlockBitmapMapping = List[Dict[str, Union[str, List[Dict[str, str]]]]]
@@ -111,10 +115,14 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase):
If @msg is None, check that there has not been any error.
"""
self.vm_b.shutdown()
+
+ log = self.vm_b.get_log()
+ assert log is not None # Loaded after shutdown
+
if msg is None:
- self.assertNotIn('qemu-system-', self.vm_b.get_log())
+ self.assertNotIn('qemu-system-', log)
else:
- self.assertIn(msg, self.vm_b.get_log())
+ self.assertIn(msg, log)
@staticmethod
def mapping(node_name: str, node_alias: str,
@@ -446,9 +454,13 @@ class TestBlockBitmapMappingErrors(TestDirtyBitmapMigration):
# Check for the error in the source's log
self.vm_a.shutdown()
+
+ log = self.vm_a.get_log()
+ assert log is not None # Loaded after shutdown
+
self.assertIn(f"Cannot migrate bitmap '{name}' on node "
f"'{self.src_node_name}': Name is longer than 255 bytes",
- self.vm_a.get_log())
+ log)
# Expect abnormal shutdown of the destination VM because of
# the failed migration
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 25/53] coroutine-sigaltstack: Add SIGUSR2 mutex
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (23 preceding siblings ...)
2021-01-26 14:19 ` [PULL 24/53] iotests/300: " Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 26/53] qapi: backup: add perf.use-copy-range parameter Max Reitz
` (28 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
Disposition (action) for any given signal is global for the process.
When two threads run coroutine-sigaltstack's qemu_coroutine_new()
concurrently, they may interfere with each other: One of them may revert
the SIGUSR2 handler to SIG_DFL, between the other thread (a) setting up
coroutine_trampoline() as the handler and (b) raising SIGUSR2. That
SIGUSR2 will then terminate the QEMU process abnormally.
We have to ensure that only one thread at a time can modify the
process-global SIGUSR2 handler. To do so, wrap the whole section where
that is done in a mutex.
Alternatively, we could for example have the SIGUSR2 handler always be
coroutine_trampoline(), so there would be no need to invoke sigaction()
in qemu_coroutine_new(). Laszlo has posted a patch to do so here:
https://lists.nongnu.org/archive/html/qemu-devel/2021-01/msg05962.html
However, given that coroutine-sigaltstack is more of a fallback
implementation for platforms that do not support ucontext, that change
may be a bit too invasive to be comfortable with it. The mutex proposed
here may negatively impact performance, but the change is much simpler.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210125120305.19520-1-mreitz@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
util/coroutine-sigaltstack.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/util/coroutine-sigaltstack.c b/util/coroutine-sigaltstack.c
index aade82afb8..e99b8a4f9c 100644
--- a/util/coroutine-sigaltstack.c
+++ b/util/coroutine-sigaltstack.c
@@ -157,6 +157,7 @@ Coroutine *qemu_coroutine_new(void)
sigset_t sigs;
sigset_t osigs;
sigjmp_buf old_env;
+ static pthread_mutex_t sigusr2_mutex = PTHREAD_MUTEX_INITIALIZER;
/* The way to manipulate stack is with the sigaltstack function. We
* prepare a stack, with it delivering a signal to ourselves and then
@@ -186,6 +187,12 @@ Coroutine *qemu_coroutine_new(void)
sa.sa_handler = coroutine_trampoline;
sigfillset(&sa.sa_mask);
sa.sa_flags = SA_ONSTACK;
+
+ /*
+ * sigaction() is a process-global operation. We must not run
+ * this code in multiple threads at once.
+ */
+ pthread_mutex_lock(&sigusr2_mutex);
if (sigaction(SIGUSR2, &sa, &osa) != 0) {
abort();
}
@@ -234,6 +241,8 @@ Coroutine *qemu_coroutine_new(void)
* Restore the old SIGUSR2 signal handler and mask
*/
sigaction(SIGUSR2, &osa, NULL);
+ pthread_mutex_unlock(&sigusr2_mutex);
+
pthread_sigmask(SIG_SETMASK, &osigs, NULL);
/*
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 26/53] qapi: backup: add perf.use-copy-range parameter
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (24 preceding siblings ...)
2021-01-26 14:19 ` [PULL 25/53] coroutine-sigaltstack: Add SIGUSR2 mutex Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 27/53] block/block-copy: More explicit call_state Max Reitz
` (27 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Experiments show, that copy_range is not always making things faster.
So, to make experimentation simpler, let's add a parameter. Some more
perf parameters will be added soon, so here is a new struct.
For now, add new backup qmp parameter with x- prefix for the following
reasons:
- We are going to add more performance parameters, some will be
related to the whole block-copy process, some only to background
copying in backup (ignored for copy-before-write operations).
- On the other hand, we are going to use block-copy interface in other
block jobs, which will need performance options as well.. And it
should be the same structure or at least somehow related.
So, there are too much unclean things about how the interface and now
we need the new options mostly for testing. Let's keep them
experimental for a while.
In do_backup_common() new x-perf parameter handled in a way to
make further options addition simpler.
We add use-copy-range with default=true, and we'll change the default
in further patch, after moving backup to use block-copy.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-2-vsementsov@virtuozzo.com>
[mreitz: s/5\.2/6.0/]
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
qapi/block-core.json | 17 ++++++++++++++++-
block/backup-top.h | 1 +
include/block/block-copy.h | 2 +-
include/block/block_int.h | 3 +++
block/backup-top.c | 4 +++-
block/backup.c | 6 +++++-
block/block-copy.c | 4 ++--
block/replication.c | 2 ++
blockdev.c | 8 ++++++++
9 files changed, 41 insertions(+), 6 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 1d9dcd7d30..83f661d7f6 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1371,6 +1371,19 @@
{ 'struct': 'BlockdevSnapshot',
'data': { 'node': 'str', 'overlay': 'str' } }
+##
+# @BackupPerf:
+#
+# Optional parameters for backup. These parameters don't affect
+# functionality, but may significantly affect performance.
+#
+# @use-copy-range: Use copy offloading. Default true.
+#
+# Since: 6.0
+##
+{ 'struct': 'BackupPerf',
+ 'data': { '*use-copy-range': 'bool' }}
+
##
# @BackupCommon:
#
@@ -1426,6 +1439,8 @@
# above node specified by @drive. If this option is not given,
# a node name is autogenerated. (Since: 4.2)
#
+# @x-perf: Performance options. (Since 6.0)
+#
# Note: @on-source-error and @on-target-error only affect background
# I/O. If an error occurs during a guest write request, the device's
# rerror/werror actions will be used.
@@ -1440,7 +1455,7 @@
'*on-source-error': 'BlockdevOnError',
'*on-target-error': 'BlockdevOnError',
'*auto-finalize': 'bool', '*auto-dismiss': 'bool',
- '*filter-node-name': 'str' } }
+ '*filter-node-name': 'str', '*x-perf': 'BackupPerf' } }
##
# @DriveBackup:
diff --git a/block/backup-top.h b/block/backup-top.h
index e5cabfa197..b28b0031c4 100644
--- a/block/backup-top.h
+++ b/block/backup-top.h
@@ -33,6 +33,7 @@ BlockDriverState *bdrv_backup_top_append(BlockDriverState *source,
BlockDriverState *target,
const char *filter_node_name,
uint64_t cluster_size,
+ BackupPerf *perf,
BdrvRequestFlags write_flags,
BlockCopyState **bcs,
Error **errp);
diff --git a/include/block/block-copy.h b/include/block/block-copy.h
index aac85e1488..6397505f30 100644
--- a/include/block/block-copy.h
+++ b/include/block/block-copy.h
@@ -22,7 +22,7 @@ typedef void (*ProgressBytesCallbackFunc)(int64_t bytes, void *opaque);
typedef struct BlockCopyState BlockCopyState;
BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
- int64_t cluster_size,
+ int64_t cluster_size, bool use_copy_range,
BdrvRequestFlags write_flags,
Error **errp);
diff --git a/include/block/block_int.h b/include/block/block_int.h
index f4b844f310..d01fc23720 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -1266,6 +1266,8 @@ void mirror_start(const char *job_id, BlockDriverState *bs,
* @sync_mode: What parts of the disk image should be copied to the destination.
* @sync_bitmap: The dirty bitmap if sync_mode is 'bitmap' or 'incremental'
* @bitmap_mode: The bitmap synchronization policy to use.
+ * @perf: Performance options. All actual fields assumed to be present,
+ * all ".has_*" fields are ignored.
* @on_source_error: The action to take upon error reading from the source.
* @on_target_error: The action to take upon error writing to the target.
* @creation_flags: Flags that control the behavior of the Job lifetime.
@@ -1284,6 +1286,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
BitmapSyncMode bitmap_mode,
bool compress,
const char *filter_node_name,
+ BackupPerf *perf,
BlockdevOnError on_source_error,
BlockdevOnError on_target_error,
int creation_flags,
diff --git a/block/backup-top.c b/block/backup-top.c
index fe6883cc97..789acf6965 100644
--- a/block/backup-top.c
+++ b/block/backup-top.c
@@ -186,6 +186,7 @@ BlockDriverState *bdrv_backup_top_append(BlockDriverState *source,
BlockDriverState *target,
const char *filter_node_name,
uint64_t cluster_size,
+ BackupPerf *perf,
BdrvRequestFlags write_flags,
BlockCopyState **bcs,
Error **errp)
@@ -244,7 +245,8 @@ BlockDriverState *bdrv_backup_top_append(BlockDriverState *source,
state->cluster_size = cluster_size;
state->bcs = block_copy_state_new(top->backing, state->target,
- cluster_size, write_flags, &local_err);
+ cluster_size, perf->use_copy_range,
+ write_flags, &local_err);
if (local_err) {
error_prepend(&local_err, "Cannot create block-copy-state: ");
goto fail;
diff --git a/block/backup.c b/block/backup.c
index 9afa0bf3b4..4b07e9115d 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -46,6 +46,7 @@ typedef struct BackupBlockJob {
uint64_t len;
uint64_t bytes_read;
int64_t cluster_size;
+ BackupPerf perf;
BlockCopyState *bcs;
} BackupBlockJob;
@@ -335,6 +336,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
BitmapSyncMode bitmap_mode,
bool compress,
const char *filter_node_name,
+ BackupPerf *perf,
BlockdevOnError on_source_error,
BlockdevOnError on_target_error,
int creation_flags,
@@ -441,7 +443,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
(compress ? BDRV_REQ_WRITE_COMPRESSED : 0),
backup_top = bdrv_backup_top_append(bs, target, filter_node_name,
- cluster_size, write_flags, &bcs, errp);
+ cluster_size, perf,
+ write_flags, &bcs, errp);
if (!backup_top) {
goto error;
}
@@ -464,6 +467,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
job->bcs = bcs;
job->cluster_size = cluster_size;
job->len = len;
+ job->perf = *perf;
block_copy_set_progress_callback(bcs, backup_progress_bytes_callback, job);
block_copy_set_progress_meter(bcs, &job->common.job.progress);
diff --git a/block/block-copy.c b/block/block-copy.c
index cd9bc47c8f..63398a171c 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -218,7 +218,7 @@ static uint32_t block_copy_max_transfer(BdrvChild *source, BdrvChild *target)
}
BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
- int64_t cluster_size,
+ int64_t cluster_size, bool use_copy_range,
BdrvRequestFlags write_flags, Error **errp)
{
BlockCopyState *s;
@@ -260,7 +260,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
* We enable copy-range, but keep small copy_size, until first
* successful copy_range (look at block_copy_do_copy).
*/
- s->use_copy_range = true;
+ s->use_copy_range = use_copy_range;
s->copy_size = MAX(s->cluster_size, BLOCK_COPY_MAX_BUFFER);
}
diff --git a/block/replication.c b/block/replication.c
index 0c70215784..22ffc811ee 100644
--- a/block/replication.c
+++ b/block/replication.c
@@ -454,6 +454,7 @@ static void replication_start(ReplicationState *rs, ReplicationMode mode,
int64_t active_length, hidden_length, disk_length;
AioContext *aio_context;
Error *local_err = NULL;
+ BackupPerf perf = { .use_copy_range = true };
aio_context = bdrv_get_aio_context(bs);
aio_context_acquire(aio_context);
@@ -558,6 +559,7 @@ static void replication_start(ReplicationState *rs, ReplicationMode mode,
s->backup_job = backup_job_create(
NULL, s->secondary_disk->bs, s->hidden_disk->bs,
0, MIRROR_SYNC_MODE_NONE, NULL, 0, false, NULL,
+ &perf,
BLOCKDEV_ON_ERROR_REPORT,
BLOCKDEV_ON_ERROR_REPORT, JOB_INTERNAL,
backup_job_completed, bs, NULL, &local_err);
diff --git a/blockdev.c b/blockdev.c
index 0540c621da..fc88dc03e1 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2829,6 +2829,7 @@ static BlockJob *do_backup_common(BackupCommon *backup,
{
BlockJob *job = NULL;
BdrvDirtyBitmap *bmap = NULL;
+ BackupPerf perf = { .use_copy_range = true };
int job_flags = JOB_DEFAULT;
if (!backup->has_speed) {
@@ -2853,6 +2854,12 @@ static BlockJob *do_backup_common(BackupCommon *backup,
backup->compress = false;
}
+ if (backup->x_perf) {
+ if (backup->x_perf->has_use_copy_range) {
+ perf.use_copy_range = backup->x_perf->use_copy_range;
+ }
+ }
+
if ((backup->sync == MIRROR_SYNC_MODE_BITMAP) ||
(backup->sync == MIRROR_SYNC_MODE_INCREMENTAL)) {
/* done before desugaring 'incremental' to print the right message */
@@ -2926,6 +2933,7 @@ static BlockJob *do_backup_common(BackupCommon *backup,
backup->sync, bmap, backup->bitmap_mode,
backup->compress,
backup->filter_node_name,
+ &perf,
backup->on_source_error,
backup->on_target_error,
job_flags, NULL, NULL, txn, errp);
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 27/53] block/block-copy: More explicit call_state
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (25 preceding siblings ...)
2021-01-26 14:19 ` [PULL 26/53] qapi: backup: add perf.use-copy-range parameter Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 28/53] block/block-copy: implement block_copy_async Max Reitz
` (26 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Refactor common path to use BlockCopyCallState pointer as parameter, to
prepare it for use in asynchronous block-copy (at least, we'll need to
run block-copy in a coroutine, passing the whole parameters as one
pointer).
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-3-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/block-copy.c | 51 ++++++++++++++++++++++++++++++++++------------
1 file changed, 38 insertions(+), 13 deletions(-)
diff --git a/block/block-copy.c b/block/block-copy.c
index 63398a171c..6ea55f1f9a 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -30,7 +30,15 @@
static coroutine_fn int block_copy_task_entry(AioTask *task);
typedef struct BlockCopyCallState {
+ /* IN parameters */
+ BlockCopyState *s;
+ int64_t offset;
+ int64_t bytes;
+
+ /* State */
bool failed;
+
+ /* OUT parameters */
bool error_is_read;
} BlockCopyCallState;
@@ -544,15 +552,17 @@ int64_t block_copy_reset_unallocated(BlockCopyState *s,
* Returns 1 if dirty clusters found and successfully copied, 0 if no dirty
* clusters found and -errno on failure.
*/
-static int coroutine_fn block_copy_dirty_clusters(BlockCopyState *s,
- int64_t offset, int64_t bytes,
- bool *error_is_read)
+static int coroutine_fn
+block_copy_dirty_clusters(BlockCopyCallState *call_state)
{
+ BlockCopyState *s = call_state->s;
+ int64_t offset = call_state->offset;
+ int64_t bytes = call_state->bytes;
+
int ret = 0;
bool found_dirty = false;
int64_t end = offset + bytes;
AioTaskPool *aio = NULL;
- BlockCopyCallState call_state = {false, false};
/*
* block_copy() user is responsible for keeping source and target in same
@@ -568,7 +578,7 @@ static int coroutine_fn block_copy_dirty_clusters(BlockCopyState *s,
BlockCopyTask *task;
int64_t status_bytes;
- task = block_copy_task_create(s, &call_state, offset, bytes);
+ task = block_copy_task_create(s, call_state, offset, bytes);
if (!task) {
/* No more dirty bits in the bitmap */
trace_block_copy_skip_range(s, offset, bytes);
@@ -633,15 +643,12 @@ out:
aio_task_pool_free(aio);
}
- if (error_is_read && ret < 0) {
- *error_is_read = call_state.error_is_read;
- }
return ret < 0 ? ret : found_dirty;
}
/*
- * block_copy
+ * block_copy_common
*
* Copy requested region, accordingly to dirty bitmap.
* Collaborate with parallel block_copy requests: if they succeed it will help
@@ -649,16 +656,16 @@ out:
* it means that some I/O operation failed in context of _this_ block_copy call,
* not some parallel operation.
*/
-int coroutine_fn block_copy(BlockCopyState *s, int64_t offset, int64_t bytes,
- bool *error_is_read)
+static int coroutine_fn block_copy_common(BlockCopyCallState *call_state)
{
int ret;
do {
- ret = block_copy_dirty_clusters(s, offset, bytes, error_is_read);
+ ret = block_copy_dirty_clusters(call_state);
if (ret == 0) {
- ret = block_copy_wait_one(s, offset, bytes);
+ ret = block_copy_wait_one(call_state->s, call_state->offset,
+ call_state->bytes);
}
/*
@@ -675,6 +682,24 @@ int coroutine_fn block_copy(BlockCopyState *s, int64_t offset, int64_t bytes,
return ret;
}
+int coroutine_fn block_copy(BlockCopyState *s, int64_t start, int64_t bytes,
+ bool *error_is_read)
+{
+ BlockCopyCallState call_state = {
+ .s = s,
+ .offset = start,
+ .bytes = bytes,
+ };
+
+ int ret = block_copy_common(&call_state);
+
+ if (error_is_read && ret < 0) {
+ *error_is_read = call_state.error_is_read;
+ }
+
+ return ret;
+}
+
BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s)
{
return s->copy_bitmap;
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 28/53] block/block-copy: implement block_copy_async
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (26 preceding siblings ...)
2021-01-26 14:19 ` [PULL 27/53] block/block-copy: More explicit call_state Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 29/53] block/block-copy: add max_chunk and max_workers parameters Max Reitz
` (25 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
We'll need async block-copy invocation to use in backup directly.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-4-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
include/block/block-copy.h | 29 ++++++++++++++
block/block-copy.c | 81 ++++++++++++++++++++++++++++++++++++--
2 files changed, 106 insertions(+), 4 deletions(-)
diff --git a/include/block/block-copy.h b/include/block/block-copy.h
index 6397505f30..8c225ebf81 100644
--- a/include/block/block-copy.h
+++ b/include/block/block-copy.h
@@ -19,7 +19,9 @@
#include "qemu/co-shared-resource.h"
typedef void (*ProgressBytesCallbackFunc)(int64_t bytes, void *opaque);
+typedef void (*BlockCopyAsyncCallbackFunc)(void *opaque);
typedef struct BlockCopyState BlockCopyState;
+typedef struct BlockCopyCallState BlockCopyCallState;
BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
int64_t cluster_size, bool use_copy_range,
@@ -41,6 +43,33 @@ int64_t block_copy_reset_unallocated(BlockCopyState *s,
int coroutine_fn block_copy(BlockCopyState *s, int64_t offset, int64_t bytes,
bool *error_is_read);
+/*
+ * Run block-copy in a coroutine, create corresponding BlockCopyCallState
+ * object and return pointer to it. Never returns NULL.
+ *
+ * Caller is responsible to call block_copy_call_free() to free
+ * BlockCopyCallState object.
+ */
+BlockCopyCallState *block_copy_async(BlockCopyState *s,
+ int64_t offset, int64_t bytes,
+ BlockCopyAsyncCallbackFunc cb,
+ void *cb_opaque);
+
+/*
+ * Free finished BlockCopyCallState. Trying to free running
+ * block-copy will crash.
+ */
+void block_copy_call_free(BlockCopyCallState *call_state);
+
+/*
+ * Note, that block-copy call is marked finished prior to calling
+ * the callback.
+ */
+bool block_copy_call_finished(BlockCopyCallState *call_state);
+bool block_copy_call_succeeded(BlockCopyCallState *call_state);
+bool block_copy_call_failed(BlockCopyCallState *call_state);
+int block_copy_call_status(BlockCopyCallState *call_state, bool *error_is_read);
+
BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s);
void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip);
diff --git a/block/block-copy.c b/block/block-copy.c
index 6ea55f1f9a..74655b86f8 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -30,13 +30,19 @@
static coroutine_fn int block_copy_task_entry(AioTask *task);
typedef struct BlockCopyCallState {
- /* IN parameters */
+ /* IN parameters. Initialized in block_copy_async() and never changed. */
BlockCopyState *s;
int64_t offset;
int64_t bytes;
+ BlockCopyAsyncCallbackFunc cb;
+ void *cb_opaque;
+
+ /* Coroutine where async block-copy is running */
+ Coroutine *co;
/* State */
- bool failed;
+ int ret;
+ bool finished;
/* OUT parameters */
bool error_is_read;
@@ -428,8 +434,8 @@ static coroutine_fn int block_copy_task_entry(AioTask *task)
ret = block_copy_do_copy(t->s, t->offset, t->bytes, t->zeroes,
&error_is_read);
- if (ret < 0 && !t->call_state->failed) {
- t->call_state->failed = true;
+ if (ret < 0 && !t->call_state->ret) {
+ t->call_state->ret = ret;
t->call_state->error_is_read = error_is_read;
} else {
progress_work_done(t->s->progress, t->bytes);
@@ -679,6 +685,12 @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state)
*/
} while (ret > 0);
+ call_state->finished = true;
+
+ if (call_state->cb) {
+ call_state->cb(call_state->cb_opaque);
+ }
+
return ret;
}
@@ -700,6 +712,67 @@ int coroutine_fn block_copy(BlockCopyState *s, int64_t start, int64_t bytes,
return ret;
}
+static void coroutine_fn block_copy_async_co_entry(void *opaque)
+{
+ block_copy_common(opaque);
+}
+
+BlockCopyCallState *block_copy_async(BlockCopyState *s,
+ int64_t offset, int64_t bytes,
+ BlockCopyAsyncCallbackFunc cb,
+ void *cb_opaque)
+{
+ BlockCopyCallState *call_state = g_new(BlockCopyCallState, 1);
+
+ *call_state = (BlockCopyCallState) {
+ .s = s,
+ .offset = offset,
+ .bytes = bytes,
+ .cb = cb,
+ .cb_opaque = cb_opaque,
+
+ .co = qemu_coroutine_create(block_copy_async_co_entry, call_state),
+ };
+
+ qemu_coroutine_enter(call_state->co);
+
+ return call_state;
+}
+
+void block_copy_call_free(BlockCopyCallState *call_state)
+{
+ if (!call_state) {
+ return;
+ }
+
+ assert(call_state->finished);
+ g_free(call_state);
+}
+
+bool block_copy_call_finished(BlockCopyCallState *call_state)
+{
+ return call_state->finished;
+}
+
+bool block_copy_call_succeeded(BlockCopyCallState *call_state)
+{
+ return call_state->finished && call_state->ret == 0;
+}
+
+bool block_copy_call_failed(BlockCopyCallState *call_state)
+{
+ return call_state->finished && call_state->ret < 0;
+}
+
+int block_copy_call_status(BlockCopyCallState *call_state, bool *error_is_read)
+{
+ assert(call_state->finished);
+ if (error_is_read) {
+ *error_is_read = call_state->error_is_read;
+ }
+ return call_state->ret;
+}
+
BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s)
{
return s->copy_bitmap;
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 29/53] block/block-copy: add max_chunk and max_workers parameters
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (27 preceding siblings ...)
2021-01-26 14:19 ` [PULL 28/53] block/block-copy: implement block_copy_async Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 30/53] block/block-copy: add list of all call-states Max Reitz
` (24 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
They will be used for backup.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-5-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
include/block/block-copy.h | 6 ++++++
block/block-copy.c | 11 +++++++++--
2 files changed, 15 insertions(+), 2 deletions(-)
diff --git a/include/block/block-copy.h b/include/block/block-copy.h
index 8c225ebf81..22372aa375 100644
--- a/include/block/block-copy.h
+++ b/include/block/block-copy.h
@@ -49,9 +49,15 @@ int coroutine_fn block_copy(BlockCopyState *s, int64_t offset, int64_t bytes,
*
* Caller is responsible to call block_copy_call_free() to free
* BlockCopyCallState object.
+ *
+ * @max_workers means maximum of parallel coroutines to execute sub-requests,
+ * must be > 0.
+ *
+ * @max_chunk means maximum length for one IO operation. Zero means unlimited.
*/
BlockCopyCallState *block_copy_async(BlockCopyState *s,
int64_t offset, int64_t bytes,
+ int max_workers, int64_t max_chunk,
BlockCopyAsyncCallbackFunc cb,
void *cb_opaque);
diff --git a/block/block-copy.c b/block/block-copy.c
index 74655b86f8..35213bd832 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -34,6 +34,8 @@ typedef struct BlockCopyCallState {
BlockCopyState *s;
int64_t offset;
int64_t bytes;
+ int max_workers;
+ int64_t max_chunk;
BlockCopyAsyncCallbackFunc cb;
void *cb_opaque;
@@ -148,10 +150,11 @@ static BlockCopyTask *block_copy_task_create(BlockCopyState *s,
int64_t offset, int64_t bytes)
{
BlockCopyTask *task;
+ int64_t max_chunk = MIN_NON_ZERO(s->copy_size, call_state->max_chunk);
if (!bdrv_dirty_bitmap_next_dirty_area(s->copy_bitmap,
offset, offset + bytes,
- s->copy_size, &offset, &bytes))
+ max_chunk, &offset, &bytes))
{
return NULL;
}
@@ -623,7 +626,7 @@ block_copy_dirty_clusters(BlockCopyCallState *call_state)
bytes = end - offset;
if (!aio && bytes) {
- aio = aio_task_pool_new(BLOCK_COPY_MAX_WORKERS);
+ aio = aio_task_pool_new(call_state->max_workers);
}
ret = block_copy_task_run(aio, task);
@@ -701,6 +704,7 @@ int coroutine_fn block_copy(BlockCopyState *s, int64_t start, int64_t bytes,
.s = s,
.offset = start,
.bytes = bytes,
+ .max_workers = BLOCK_COPY_MAX_WORKERS,
};
int ret = block_copy_common(&call_state);
@@ -719,6 +723,7 @@ static void coroutine_fn block_copy_async_co_entry(void *opaque)
BlockCopyCallState *block_copy_async(BlockCopyState *s,
int64_t offset, int64_t bytes,
+ int max_workers, int64_t max_chunk,
BlockCopyAsyncCallbackFunc cb,
void *cb_opaque)
{
@@ -728,6 +733,8 @@ BlockCopyCallState *block_copy_async(BlockCopyState *s,
.s = s,
.offset = offset,
.bytes = bytes,
+ .max_workers = max_workers,
+ .max_chunk = max_chunk,
.cb = cb,
.cb_opaque = cb_opaque,
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 30/53] block/block-copy: add list of all call-states
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (28 preceding siblings ...)
2021-01-26 14:19 ` [PULL 29/53] block/block-copy: add max_chunk and max_workers parameters Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 31/53] block/block-copy: add ratelimit to block-copy Max Reitz
` (23 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
It simplifies debugging.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-6-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/block-copy.c | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/block/block-copy.c b/block/block-copy.c
index 35213bd832..6bf1735b93 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -42,6 +42,9 @@ typedef struct BlockCopyCallState {
/* Coroutine where async block-copy is running */
Coroutine *co;
+ /* To reference all call states from BlockCopyState */
+ QLIST_ENTRY(BlockCopyCallState) list;
+
/* State */
int ret;
bool finished;
@@ -81,7 +84,8 @@ typedef struct BlockCopyState {
bool use_copy_range;
int64_t copy_size;
uint64_t len;
- QLIST_HEAD(, BlockCopyTask) tasks;
+ QLIST_HEAD(, BlockCopyTask) tasks; /* All tasks from all block-copy calls */
+ QLIST_HEAD(, BlockCopyCallState) calls;
BdrvRequestFlags write_flags;
@@ -282,6 +286,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
}
QLIST_INIT(&s->tasks);
+ QLIST_INIT(&s->calls);
return s;
}
@@ -669,6 +674,8 @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state)
{
int ret;
+ QLIST_INSERT_HEAD(&call_state->s->calls, call_state, list);
+
do {
ret = block_copy_dirty_clusters(call_state);
@@ -694,6 +701,8 @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state)
call_state->cb(call_state->cb_opaque);
}
+ QLIST_REMOVE(call_state, list);
+
return ret;
}
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 31/53] block/block-copy: add ratelimit to block-copy
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (29 preceding siblings ...)
2021-01-26 14:19 ` [PULL 30/53] block/block-copy: add list of all call-states Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 32/53] block/block-copy: add block_copy_cancel Max Reitz
` (22 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
We are going to directly use one async block-copy operation for backup
job, so we need rate limiter.
We want to maintain current backup behavior: only background copying is
limited and copy-before-write operations only participate in limit
calculation. Therefore we need one rate limiter for block-copy state
and boolean flag for block-copy call state for actual limitation.
Note, that we can't just calculate each chunk in limiter after
successful copying: it will not save us from starting a lot of async
sub-requests which will exceed limit too much. Instead let's use the
following scheme on sub-request creation:
1. If at the moment limit is not exceeded, create the request and
account it immediately.
2. If at the moment limit is already exceeded, drop create sub-request
and handle limit instead (by sleep).
With this approach we'll never exceed the limit more than by one
sub-request (which pretty much matches current backup behavior).
Note also, that if there is in-flight block-copy async call,
block_copy_kick() should be used after set-speed to apply new setup
faster. For that block_copy_kick() published in this patch.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-7-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
include/block/block-copy.h | 5 ++++-
block/backup-top.c | 2 +-
block/backup.c | 2 +-
block/block-copy.c | 46 +++++++++++++++++++++++++++++++++++++-
4 files changed, 51 insertions(+), 4 deletions(-)
diff --git a/include/block/block-copy.h b/include/block/block-copy.h
index 22372aa375..b5a53ad59e 100644
--- a/include/block/block-copy.h
+++ b/include/block/block-copy.h
@@ -41,7 +41,7 @@ int64_t block_copy_reset_unallocated(BlockCopyState *s,
int64_t offset, int64_t *count);
int coroutine_fn block_copy(BlockCopyState *s, int64_t offset, int64_t bytes,
- bool *error_is_read);
+ bool ignore_ratelimit, bool *error_is_read);
/*
* Run block-copy in a coroutine, create corresponding BlockCopyCallState
@@ -76,6 +76,9 @@ bool block_copy_call_succeeded(BlockCopyCallState *call_state);
bool block_copy_call_failed(BlockCopyCallState *call_state);
int block_copy_call_status(BlockCopyCallState *call_state, bool *error_is_read);
+void block_copy_set_speed(BlockCopyState *s, uint64_t speed);
+void block_copy_kick(BlockCopyCallState *call_state);
+
BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s);
void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip);
diff --git a/block/backup-top.c b/block/backup-top.c
index 789acf6965..779956ddc2 100644
--- a/block/backup-top.c
+++ b/block/backup-top.c
@@ -61,7 +61,7 @@ static coroutine_fn int backup_top_cbw(BlockDriverState *bs, uint64_t offset,
off = QEMU_ALIGN_DOWN(offset, s->cluster_size);
end = QEMU_ALIGN_UP(offset + bytes, s->cluster_size);
- return block_copy(s->bcs, off, end - off, NULL);
+ return block_copy(s->bcs, off, end - off, true, NULL);
}
static int coroutine_fn backup_top_co_pdiscard(BlockDriverState *bs,
diff --git a/block/backup.c b/block/backup.c
index 4b07e9115d..09ff5a92ef 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -72,7 +72,7 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job,
trace_backup_do_cow_enter(job, start, offset, bytes);
- ret = block_copy(job->bcs, start, end - start, error_is_read);
+ ret = block_copy(job->bcs, start, end - start, true, error_is_read);
trace_backup_do_cow_return(job, offset, bytes, ret);
diff --git a/block/block-copy.c b/block/block-copy.c
index 6bf1735b93..fa27450b14 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -26,6 +26,7 @@
#define BLOCK_COPY_MAX_BUFFER (1 * MiB)
#define BLOCK_COPY_MAX_MEM (128 * MiB)
#define BLOCK_COPY_MAX_WORKERS 64
+#define BLOCK_COPY_SLICE_TIME 100000000ULL /* ns */
static coroutine_fn int block_copy_task_entry(AioTask *task);
@@ -36,6 +37,7 @@ typedef struct BlockCopyCallState {
int64_t bytes;
int max_workers;
int64_t max_chunk;
+ bool ignore_ratelimit;
BlockCopyAsyncCallbackFunc cb;
void *cb_opaque;
@@ -48,6 +50,7 @@ typedef struct BlockCopyCallState {
/* State */
int ret;
bool finished;
+ QemuCoSleepState *sleep_state;
/* OUT parameters */
bool error_is_read;
@@ -111,6 +114,9 @@ typedef struct BlockCopyState {
void *progress_opaque;
SharedResource *mem;
+
+ uint64_t speed;
+ RateLimit rate_limit;
} BlockCopyState;
static BlockCopyTask *find_conflicting_task(BlockCopyState *s,
@@ -623,6 +629,21 @@ block_copy_dirty_clusters(BlockCopyCallState *call_state)
}
task->zeroes = ret & BDRV_BLOCK_ZERO;
+ if (s->speed) {
+ if (!call_state->ignore_ratelimit) {
+ uint64_t ns = ratelimit_calculate_delay(&s->rate_limit, 0);
+ if (ns > 0) {
+ block_copy_task_end(task, -EAGAIN);
+ g_free(task);
+ qemu_co_sleep_ns_wakeable(QEMU_CLOCK_REALTIME, ns,
+ &call_state->sleep_state);
+ continue;
+ }
+ }
+
+ ratelimit_calculate_delay(&s->rate_limit, task->bytes);
+ }
+
trace_block_copy_process(s, task->offset);
co_get_from_shres(s->mem, task->bytes);
@@ -661,6 +682,13 @@ out:
return ret < 0 ? ret : found_dirty;
}
+void block_copy_kick(BlockCopyCallState *call_state)
+{
+ if (call_state->sleep_state) {
+ qemu_co_sleep_wake(call_state->sleep_state);
+ }
+}
+
/*
* block_copy_common
*
@@ -707,12 +735,13 @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state)
}
int coroutine_fn block_copy(BlockCopyState *s, int64_t start, int64_t bytes,
- bool *error_is_read)
+ bool ignore_ratelimit, bool *error_is_read)
{
BlockCopyCallState call_state = {
.s = s,
.offset = start,
.bytes = bytes,
+ .ignore_ratelimit = ignore_ratelimit,
.max_workers = BLOCK_COPY_MAX_WORKERS,
};
@@ -798,3 +827,18 @@ void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip)
{
s->skip_unallocated = skip;
}
+
+void block_copy_set_speed(BlockCopyState *s, uint64_t speed)
+{
+ s->speed = speed;
+ if (speed > 0) {
+ ratelimit_set_speed(&s->rate_limit, speed, BLOCK_COPY_SLICE_TIME);
+ }
+
+ /*
+ * Note: it's good to kick all call states from here, but it should be done
+ * only from a coroutine, to not crash if s->calls list changed while
+ * entering one call. So for now, the only user of this function kicks its
+ * only one call_state by hand.
+ */
+}
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 32/53] block/block-copy: add block_copy_cancel
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (30 preceding siblings ...)
2021-01-26 14:19 ` [PULL 31/53] block/block-copy: add ratelimit to block-copy Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 33/53] blockjob: add set_speed to BlockJobDriver Max Reitz
` (21 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Add function to cancel running async block-copy call. It will be used
in backup.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-8-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
include/block/block-copy.h | 13 +++++++++++++
block/block-copy.c | 24 +++++++++++++++++++-----
2 files changed, 32 insertions(+), 5 deletions(-)
diff --git a/include/block/block-copy.h b/include/block/block-copy.h
index b5a53ad59e..7821850f88 100644
--- a/include/block/block-copy.h
+++ b/include/block/block-copy.h
@@ -74,11 +74,24 @@ void block_copy_call_free(BlockCopyCallState *call_state);
bool block_copy_call_finished(BlockCopyCallState *call_state);
bool block_copy_call_succeeded(BlockCopyCallState *call_state);
bool block_copy_call_failed(BlockCopyCallState *call_state);
+bool block_copy_call_cancelled(BlockCopyCallState *call_state);
int block_copy_call_status(BlockCopyCallState *call_state, bool *error_is_read);
void block_copy_set_speed(BlockCopyState *s, uint64_t speed);
void block_copy_kick(BlockCopyCallState *call_state);
+/*
+ * Cancel running block-copy call.
+ *
+ * Cancel leaves block-copy state valid: dirty bits are correct and you may use
+ * cancel + <run block_copy with same parameters> to emulate pause/resume.
+ *
+ * Note also, that the cancel is async: it only marks block-copy call to be
+ * cancelled. So, the call may be cancelled (block_copy_call_cancelled() reports
+ * true) but not yet finished (block_copy_call_finished() reports false).
+ */
+void block_copy_call_cancel(BlockCopyCallState *call_state);
+
BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s);
void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip);
diff --git a/block/block-copy.c b/block/block-copy.c
index fa27450b14..82cf945693 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -51,6 +51,7 @@ typedef struct BlockCopyCallState {
int ret;
bool finished;
QemuCoSleepState *sleep_state;
+ bool cancelled;
/* OUT parameters */
bool error_is_read;
@@ -594,7 +595,7 @@ block_copy_dirty_clusters(BlockCopyCallState *call_state)
assert(QEMU_IS_ALIGNED(offset, s->cluster_size));
assert(QEMU_IS_ALIGNED(bytes, s->cluster_size));
- while (bytes && aio_task_pool_status(aio) == 0) {
+ while (bytes && aio_task_pool_status(aio) == 0 && !call_state->cancelled) {
BlockCopyTask *task;
int64_t status_bytes;
@@ -707,7 +708,7 @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state)
do {
ret = block_copy_dirty_clusters(call_state);
- if (ret == 0) {
+ if (ret == 0 && !call_state->cancelled) {
ret = block_copy_wait_one(call_state->s, call_state->offset,
call_state->bytes);
}
@@ -721,7 +722,7 @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state)
* 2. We have waited for some intersecting block-copy request
* It may have failed and produced new dirty bits.
*/
- } while (ret > 0);
+ } while (ret > 0 && !call_state->cancelled);
call_state->finished = true;
@@ -801,12 +802,19 @@ bool block_copy_call_finished(BlockCopyCallState *call_state)
bool block_copy_call_succeeded(BlockCopyCallState *call_state)
{
- return call_state->finished && call_state->ret == 0;
+ return call_state->finished && !call_state->cancelled &&
+ call_state->ret == 0;
}
bool block_copy_call_failed(BlockCopyCallState *call_state)
{
- return call_state->finished && call_state->ret < 0;
+ return call_state->finished && !call_state->cancelled &&
+ call_state->ret < 0;
+}
+
+bool block_copy_call_cancelled(BlockCopyCallState *call_state)
+{
+ return call_state->cancelled;
}
int block_copy_call_status(BlockCopyCallState *call_state, bool *error_is_read)
@@ -818,6 +826,12 @@ int block_copy_call_status(BlockCopyCallState *call_state, bool *error_is_read)
return call_state->ret;
}
+void block_copy_call_cancel(BlockCopyCallState *call_state)
+{
+ call_state->cancelled = true;
+ block_copy_kick(call_state);
+}
+
BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s)
{
return s->copy_bitmap;
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 33/53] blockjob: add set_speed to BlockJobDriver
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (31 preceding siblings ...)
2021-01-26 14:19 ` [PULL 32/53] block/block-copy: add block_copy_cancel Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 34/53] job: call job_enter from job_pause Max Reitz
` (20 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
We are going to use async block-copy call in backup, so we'll need to
passthrough setting backup speed to block-copy call.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-9-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
include/block/blockjob_int.h | 2 ++
blockjob.c | 6 ++++++
2 files changed, 8 insertions(+)
diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h
index e2824a36a8..6633d83da2 100644
--- a/include/block/blockjob_int.h
+++ b/include/block/blockjob_int.h
@@ -52,6 +52,8 @@ struct BlockJobDriver {
* besides job->blk to the new AioContext.
*/
void (*attached_aio_context)(BlockJob *job, AioContext *new_context);
+
+ void (*set_speed)(BlockJob *job, int64_t speed);
};
/**
diff --git a/blockjob.c b/blockjob.c
index 98ac8af982..db3a21699c 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -256,6 +256,7 @@ static bool job_timer_pending(Job *job)
void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp)
{
+ const BlockJobDriver *drv = block_job_driver(job);
int64_t old_speed = job->speed;
if (job_apply_verb(&job->job, JOB_VERB_SET_SPEED, errp)) {
@@ -270,6 +271,11 @@ void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp)
ratelimit_set_speed(&job->limit, speed, BLOCK_JOB_SLICE_TIME);
job->speed = speed;
+
+ if (drv->set_speed) {
+ drv->set_speed(job, speed);
+ }
+
if (speed && speed <= old_speed) {
return;
}
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 34/53] job: call job_enter from job_pause
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (32 preceding siblings ...)
2021-01-26 14:19 ` [PULL 33/53] blockjob: add set_speed to BlockJobDriver Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 35/53] qapi: backup: add max-chunk and max-workers to x-perf struct Max Reitz
` (19 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
If main job coroutine called job_yield (while some background process
is in progress), we should give it a chance to call job_pause_point().
It will be used in backup, when moved on async block-copy.
Note, that job_user_pause is not enough: we want to handle
child_job_drained_begin() as well, which call job_pause().
Still, if job is already in job_do_yield() in job_pause_point() we
should not enter it.
iotest 109 output is modified: on stop we do bdrv_drain_all() which now
triggers job pause immediately (and pause after ready is standby).
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210116214705.822267-10-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
job.c | 3 +++
tests/qemu-iotests/109.out | 24 ++++++++++++++++++++++++
2 files changed, 27 insertions(+)
diff --git a/job.c b/job.c
index 8fecf38960..3aaaebafe2 100644
--- a/job.c
+++ b/job.c
@@ -553,6 +553,9 @@ static bool job_timer_not_pending(Job *job)
void job_pause(Job *job)
{
job->pause_count++;
+ if (!job->paused) {
+ job_enter(job);
+ }
}
void job_resume(Job *job)
diff --git a/tests/qemu-iotests/109.out b/tests/qemu-iotests/109.out
index 6e73406cdb..8f839b4b7f 100644
--- a/tests/qemu-iotests/109.out
+++ b/tests/qemu-iotests/109.out
@@ -42,6 +42,8 @@ read 512/512 bytes at offset 0
{"execute":"quit"}
{"return": {}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}}
+{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "src"}}
+{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 1024, "offset": 1024, "speed": 0, "type": "mirror"}}
@@ -91,6 +93,8 @@ read 512/512 bytes at offset 0
{"execute":"quit"}
{"return": {}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}}
+{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "src"}}
+{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 197120, "offset": 197120, "speed": 0, "type": "mirror"}}
@@ -140,6 +144,8 @@ read 512/512 bytes at offset 0
{"execute":"quit"}
{"return": {}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}}
+{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "src"}}
+{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 327680, "offset": 327680, "speed": 0, "type": "mirror"}}
@@ -189,6 +195,8 @@ read 512/512 bytes at offset 0
{"execute":"quit"}
{"return": {}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}}
+{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "src"}}
+{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 1024, "offset": 1024, "speed": 0, "type": "mirror"}}
@@ -238,6 +246,8 @@ read 512/512 bytes at offset 0
{"execute":"quit"}
{"return": {}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}}
+{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "src"}}
+{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 65536, "offset": 65536, "speed": 0, "type": "mirror"}}
@@ -287,6 +297,8 @@ read 512/512 bytes at offset 0
{"execute":"quit"}
{"return": {}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}}
+{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "src"}}
+{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 2560, "offset": 2560, "speed": 0, "type": "mirror"}}
@@ -335,6 +347,8 @@ read 512/512 bytes at offset 0
{"execute":"quit"}
{"return": {}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}}
+{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "src"}}
+{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 2560, "offset": 2560, "speed": 0, "type": "mirror"}}
@@ -383,6 +397,8 @@ read 512/512 bytes at offset 0
{"execute":"quit"}
{"return": {}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}}
+{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "src"}}
+{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 31457280, "offset": 31457280, "speed": 0, "type": "mirror"}}
@@ -431,6 +447,8 @@ read 512/512 bytes at offset 0
{"execute":"quit"}
{"return": {}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}}
+{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "src"}}
+{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 327680, "offset": 327680, "speed": 0, "type": "mirror"}}
@@ -479,6 +497,8 @@ read 512/512 bytes at offset 0
{"execute":"quit"}
{"return": {}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}}
+{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "src"}}
+{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 2048, "offset": 2048, "speed": 0, "type": "mirror"}}
@@ -507,6 +527,8 @@ WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed
{"execute":"quit"}
{"return": {}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}}
+{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "src"}}
+{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 512, "offset": 512, "speed": 0, "type": "mirror"}}
@@ -528,6 +550,8 @@ Images are identical.
{"execute":"quit"}
{"return": {}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}}
+{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "src"}}
+{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 512, "offset": 512, "speed": 0, "type": "mirror"}}
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 35/53] qapi: backup: add max-chunk and max-workers to x-perf struct
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (33 preceding siblings ...)
2021-01-26 14:19 ` [PULL 34/53] job: call job_enter from job_pause Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:19 ` [PULL 36/53] iotests: 56: prepare for backup over block-copy Max Reitz
` (18 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Add new parameters to configure future backup features. The patch
doesn't introduce aio backup requests (so we actually have only one
worker) neither requests larger than one cluster. Still, formally we
satisfy these maximums anyway, so add the parameters now, to facilitate
further patch which will really change backup job behavior.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-11-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
qapi/block-core.json | 13 ++++++++++++-
block/backup.c | 28 +++++++++++++++++++++++-----
block/replication.c | 2 +-
blockdev.c | 8 +++++++-
4 files changed, 43 insertions(+), 8 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 83f661d7f6..abcd41ed63 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1379,10 +1379,21 @@
#
# @use-copy-range: Use copy offloading. Default true.
#
+# @max-workers: Maximum number of parallel requests for the sustained background
+# copying process. Doesn't influence copy-before-write operations.
+# Default 64.
+#
+# @max-chunk: Maximum request length for the sustained background copying
+# process. Doesn't influence copy-before-write operations.
+# 0 means unlimited. If max-chunk is non-zero then it should not be
+# less than job cluster size which is calculated as maximum of
+# target image cluster size and 64k. Default 0.
+#
# Since: 6.0
##
{ 'struct': 'BackupPerf',
- 'data': { '*use-copy-range': 'bool' }}
+ 'data': { '*use-copy-range': 'bool',
+ '*max-workers': 'int', '*max-chunk': 'int64' } }
##
# @BackupCommon:
diff --git a/block/backup.c b/block/backup.c
index 09ff5a92ef..5522c0f3fe 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -388,6 +388,29 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
return NULL;
}
+ cluster_size = backup_calculate_cluster_size(target, errp);
+ if (cluster_size < 0) {
+ goto error;
+ }
+
+ if (perf->max_workers < 1) {
+ error_setg(errp, "max-workers must be greater than zero");
+ return NULL;
+ }
+
+ if (perf->max_chunk < 0) {
+ error_setg(errp, "max-chunk must be zero (which means no limit) or "
+ "positive");
+ return NULL;
+ }
+
+ if (perf->max_chunk && perf->max_chunk < cluster_size) {
+ error_setg(errp, "Required max-chunk (%" PRIi64 ") is less than backup "
+ "cluster size (%" PRIi64 ")", perf->max_chunk, cluster_size);
+ return NULL;
+ }
+
+
if (sync_bitmap) {
/* If we need to write to this bitmap, check that we can: */
if (bitmap_mode != BITMAP_SYNC_MODE_NEVER &&
@@ -420,11 +443,6 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
goto error;
}
- cluster_size = backup_calculate_cluster_size(target, errp);
- if (cluster_size < 0) {
- goto error;
- }
-
/*
* If source is in backing chain of target assume that target is going to be
* used for "image fleecing", i.e. it should represent a kind of snapshot of
diff --git a/block/replication.c b/block/replication.c
index 22ffc811ee..97be7ef4de 100644
--- a/block/replication.c
+++ b/block/replication.c
@@ -454,7 +454,7 @@ static void replication_start(ReplicationState *rs, ReplicationMode mode,
int64_t active_length, hidden_length, disk_length;
AioContext *aio_context;
Error *local_err = NULL;
- BackupPerf perf = { .use_copy_range = true };
+ BackupPerf perf = { .use_copy_range = true, .max_workers = 1 };
aio_context = bdrv_get_aio_context(bs);
aio_context_acquire(aio_context);
diff --git a/blockdev.c b/blockdev.c
index fc88dc03e1..25aaacf253 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2829,7 +2829,7 @@ static BlockJob *do_backup_common(BackupCommon *backup,
{
BlockJob *job = NULL;
BdrvDirtyBitmap *bmap = NULL;
- BackupPerf perf = { .use_copy_range = true };
+ BackupPerf perf = { .use_copy_range = true, .max_workers = 64 };
int job_flags = JOB_DEFAULT;
if (!backup->has_speed) {
@@ -2858,6 +2858,12 @@ static BlockJob *do_backup_common(BackupCommon *backup,
if (backup->x_perf->has_use_copy_range) {
perf.use_copy_range = backup->x_perf->use_copy_range;
}
+ if (backup->x_perf->has_max_workers) {
+ perf.max_workers = backup->x_perf->max_workers;
+ }
+ if (backup->x_perf->has_max_chunk) {
+ perf.max_chunk = backup->x_perf->max_chunk;
+ }
}
if ((backup->sync == MIRROR_SYNC_MODE_BITMAP) ||
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 36/53] iotests: 56: prepare for backup over block-copy
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (34 preceding siblings ...)
2021-01-26 14:19 ` [PULL 35/53] qapi: backup: add max-chunk and max-workers to x-perf struct Max Reitz
@ 2021-01-26 14:19 ` Max Reitz
2021-01-26 14:20 ` [PULL 37/53] iotests/129: Limit backup's max-chunk/max-workers Max Reitz
` (17 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:19 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
After introducing parallel async copy requests instead of plain
cluster-by-cluster copying loop, we'll have to wait for paused status,
as we need to wait for several parallel request. So, let's gently wait
instead of just asserting that job already paused.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-12-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
tests/qemu-iotests/056 | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/tests/qemu-iotests/056 b/tests/qemu-iotests/056
index 0e6b8591e7..b459a3f1e8 100755
--- a/tests/qemu-iotests/056
+++ b/tests/qemu-iotests/056
@@ -308,8 +308,13 @@ class BackupTest(iotests.QMPTestCase):
event = self.vm.event_wait(name="BLOCK_JOB_ERROR",
match={'data': {'device': 'drive0'}})
self.assertNotEqual(event, None)
- # OK, job should be wedged
- res = self.vm.qmp('query-block-jobs')
+ # OK, job should pause, but it can't do it immediately, as it can't
+ # cancel other parallel requests (which didn't fail)
+ with iotests.Timeout(60, "Timeout waiting for backup actually paused"):
+ while True:
+ res = self.vm.qmp('query-block-jobs')
+ if res['return'][0]['status'] == 'paused':
+ break
self.assert_qmp(res, 'return[0]/status', 'paused')
res = self.vm.qmp('block-job-dismiss', id='drive0')
self.assert_qmp(res, 'error/desc',
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 37/53] iotests/129: Limit backup's max-chunk/max-workers
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (35 preceding siblings ...)
2021-01-26 14:19 ` [PULL 36/53] iotests: 56: prepare for backup over block-copy Max Reitz
@ 2021-01-26 14:20 ` Max Reitz
2021-01-26 14:20 ` [PULL 38/53] iotests: 185: prepare for backup over block-copy Max Reitz
` (16 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
Right now, this does not change anything, because backup ignores
max-chunk and max-workers. However, as soon as backup is switched over
to block-copy for the background copying process, we will need it to
keep 129 passing.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210120102043.28346-1-mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
tests/qemu-iotests/129 | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129
index 2f7b28d4a0..5251e2669e 100755
--- a/tests/qemu-iotests/129
+++ b/tests/qemu-iotests/129
@@ -71,9 +71,14 @@ class TestStopWithBlockJob(iotests.QMPTestCase):
sync="full", buf_size=65536)
def test_drive_backup(self):
+ # Limit max-chunk and max-workers so that block-copy will not
+ # launch so many workers working on so much data each that
+ # stop's bdrv_drain_all() would finish the job
self.do_test_stop("drive-backup", device="drive0",
target=self.target_img, format=iotests.imgfmt,
- sync="full")
+ sync="full",
+ x_perf={ 'max-chunk': 65536,
+ 'max-workers': 8 })
def test_block_commit(self):
# Add overlay above the source node so that we actually use a
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 38/53] iotests: 185: prepare for backup over block-copy
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (36 preceding siblings ...)
2021-01-26 14:20 ` [PULL 37/53] iotests/129: Limit backup's max-chunk/max-workers Max Reitz
@ 2021-01-26 14:20 ` Max Reitz
2021-01-26 14:20 ` [PULL 39/53] iotests: 219: " Max Reitz
` (15 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
The further change of moving backup to be a one block-copy call will
make copying chunk-size and cluster-size two separate things. So, even
with 64k cluster sized qcow2 image, default chunk would be 1M.
185 test however assumes, that with speed limited to 64K, one iteration
would result in offset=64K. It will change, as first iteration would
result in offset=1M independently of speed.
So, let's explicitly specify, what test wants: set max-chunk to 64K, so
that one iteration is 64K. Note, that we don't need to limit
max-workers, as block-copy rate limiter will handle the situation and
wouldn't start new workers when speed limit is obviously reached.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-13-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
tests/qemu-iotests/185 | 3 ++-
tests/qemu-iotests/185.out | 3 ++-
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/tests/qemu-iotests/185 b/tests/qemu-iotests/185
index 7bc8fe5767..f2ec5c5ceb 100755
--- a/tests/qemu-iotests/185
+++ b/tests/qemu-iotests/185
@@ -183,7 +183,8 @@ _send_qemu_cmd $h \
'target': '$TEST_IMG.copy',
'format': '$IMGFMT',
'sync': 'full',
- 'speed': 65536 } }" \
+ 'speed': 65536,
+ 'x-perf': {'max-chunk': 65536} } }" \
"return"
# If we don't sleep here 'quit' command races with disk I/O
diff --git a/tests/qemu-iotests/185.out b/tests/qemu-iotests/185.out
index eab55d22bf..9dedc8eacb 100644
--- a/tests/qemu-iotests/185.out
+++ b/tests/qemu-iotests/185.out
@@ -88,7 +88,8 @@ Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 extended_l2=off
'target': 'TEST_DIR/t.IMGFMT.copy',
'format': 'IMGFMT',
'sync': 'full',
- 'speed': 65536 } }
+ 'speed': 65536,
+ 'x-perf': { 'max-chunk': 65536 } } }
Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "disk"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}}
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 39/53] iotests: 219: prepare for backup over block-copy
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (37 preceding siblings ...)
2021-01-26 14:20 ` [PULL 38/53] iotests: 185: prepare for backup over block-copy Max Reitz
@ 2021-01-26 14:20 ` Max Reitz
2021-01-26 14:20 ` [PULL 40/53] iotests: 257: " Max Reitz
` (14 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
The further change of moving backup to be a one block-copy call will
make copying chunk-size and cluster-size two separate things. So, even
with 64k cluster sized qcow2 image, default chunk would be 1M.
Test 219 depends on specified chunk-size. Update it for explicit
chunk-size for backup as for mirror.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-14-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
tests/qemu-iotests/219 | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/tests/qemu-iotests/219 b/tests/qemu-iotests/219
index 16c3ca7fff..d1757e9e6f 100755
--- a/tests/qemu-iotests/219
+++ b/tests/qemu-iotests/219
@@ -204,13 +204,13 @@ with iotests.FilePath('disk.img') as disk_path, \
# but related to this also automatic state transitions like job
# completion), but still get pause points often enough to avoid making this
# test very slow, it's important to have the right ratio between speed and
- # buf_size.
+ # copy-chunk-size.
#
- # For backup, buf_size is hard-coded to the source image cluster size (64k),
- # so we'll pick the same for mirror. The slice time, i.e. the granularity
- # of the rate limiting is 100ms. With a speed of 256k per second, we can
- # get four pause points per second. This gives us 250ms per iteration,
- # which should be enough to stay deterministic.
+ # Chose 64k copy-chunk-size both for mirror (by buf_size) and backup (by
+ # x-max-chunk). The slice time, i.e. the granularity of the rate limiting
+ # is 100ms. With a speed of 256k per second, we can get four pause points
+ # per second. This gives us 250ms per iteration, which should be enough to
+ # stay deterministic.
test_job_lifecycle(vm, 'drive-mirror', has_ready=True, job_args={
'device': 'drive0-node',
@@ -227,6 +227,7 @@ with iotests.FilePath('disk.img') as disk_path, \
'target': copy_path,
'sync': 'full',
'speed': 262144,
+ 'x-perf': {'max-chunk': 65536},
'auto-finalize': auto_finalize,
'auto-dismiss': auto_dismiss,
})
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 40/53] iotests: 257: prepare for backup over block-copy
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (38 preceding siblings ...)
2021-01-26 14:20 ` [PULL 39/53] iotests: 219: " Max Reitz
@ 2021-01-26 14:20 ` Max Reitz
2021-01-26 14:20 ` [PULL 41/53] block/block-copy: make progress_bytes_callback optional Max Reitz
` (13 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Iotest 257 dumps a lot of in-progress information of backup job, such
as offset and bitmap dirtiness. Further commit will move backup to be
one block-copy call, which will introduce async parallel requests
instead of plain cluster-by-cluster copying. To keep things
deterministic, allow only one worker (only one copy request at a time)
for this test.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-15-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
tests/qemu-iotests/257 | 1 +
tests/qemu-iotests/257.out | 306 ++++++++++++++++++-------------------
2 files changed, 154 insertions(+), 153 deletions(-)
diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
index a2f4b5afe6..7cd2520829 100755
--- a/tests/qemu-iotests/257
+++ b/tests/qemu-iotests/257
@@ -192,6 +192,7 @@ def blockdev_backup(vm, device, target, sync, **kwargs):
target=target,
sync=sync,
filter_node_name='backup-top',
+ x_perf={'max-workers': 1},
**kwargs)
return result
diff --git a/tests/qemu-iotests/257.out b/tests/qemu-iotests/257.out
index 64dd460055..a7ba512f4c 100644
--- a/tests/qemu-iotests/257.out
+++ b/tests/qemu-iotests/257.out
@@ -30,7 +30,7 @@ write -P0x76 0x3ff0000 0x10000
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -78,7 +78,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -92,7 +92,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
--- Write #2 ---
@@ -205,7 +205,7 @@ expecting 15 dirty sectors; have 15. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -219,7 +219,7 @@ expecting 15 dirty sectors; have 15. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
@@ -290,7 +290,7 @@ write -P0x76 0x3ff0000 0x10000
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -338,7 +338,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -354,7 +354,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"action": "report", "device": "backup_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{"data": {"device": "backup_1", "error": "Input/output error", "len": 393216, "offset": 65536, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -416,7 +416,7 @@ expecting 14 dirty sectors; have 14. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -430,7 +430,7 @@ expecting 14 dirty sectors; have 14. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
@@ -501,7 +501,7 @@ write -P0x76 0x3ff0000 0x10000
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -549,7 +549,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -563,7 +563,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
--- Write #2 ---
@@ -676,7 +676,7 @@ expecting 15 dirty sectors; have 15. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -690,7 +690,7 @@ expecting 15 dirty sectors; have 15. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
@@ -761,7 +761,7 @@ write -P0x76 0x3ff0000 0x10000
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -809,7 +809,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -823,7 +823,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
--- Write #2 ---
@@ -936,7 +936,7 @@ expecting 15 dirty sectors; have 15. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -950,7 +950,7 @@ expecting 15 dirty sectors; have 15. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
@@ -1021,7 +1021,7 @@ write -P0x76 0x3ff0000 0x10000
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -1069,7 +1069,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -1085,7 +1085,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"action": "report", "device": "backup_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{"data": {"device": "backup_1", "error": "Input/output error", "len": 393216, "offset": 65536, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -1147,7 +1147,7 @@ expecting 14 dirty sectors; have 14. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -1161,7 +1161,7 @@ expecting 14 dirty sectors; have 14. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
@@ -1232,7 +1232,7 @@ write -P0x76 0x3ff0000 0x10000
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -1280,7 +1280,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -1294,7 +1294,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
--- Write #2 ---
@@ -1407,7 +1407,7 @@ expecting 12 dirty sectors; have 12. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -1421,7 +1421,7 @@ expecting 12 dirty sectors; have 12. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
@@ -1492,7 +1492,7 @@ write -P0x76 0x3ff0000 0x10000
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -1540,7 +1540,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -1554,7 +1554,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
--- Write #2 ---
@@ -1667,7 +1667,7 @@ expecting 12 dirty sectors; have 12. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -1681,7 +1681,7 @@ expecting 12 dirty sectors; have 12. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
@@ -1752,7 +1752,7 @@ write -P0x76 0x3ff0000 0x10000
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -1800,7 +1800,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -1816,7 +1816,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"action": "report", "device": "backup_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{"data": {"device": "backup_1", "error": "Input/output error", "len": 393216, "offset": 65536, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -1878,7 +1878,7 @@ expecting 13 dirty sectors; have 13. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -1892,7 +1892,7 @@ expecting 13 dirty sectors; have 13. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
@@ -1963,7 +1963,7 @@ write -P0x76 0x3ff0000 0x10000
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -2011,7 +2011,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -2025,7 +2025,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
--- Write #2 ---
@@ -2138,7 +2138,7 @@ expecting 12 dirty sectors; have 12. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -2152,7 +2152,7 @@ expecting 12 dirty sectors; have 12. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
@@ -2223,7 +2223,7 @@ write -P0x76 0x3ff0000 0x10000
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -2271,7 +2271,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -2285,7 +2285,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "full", "target": "backup_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "full", "target": "backup_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
--- Write #2 ---
@@ -2398,7 +2398,7 @@ expecting 15 dirty sectors; have 15. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -2412,7 +2412,7 @@ expecting 15 dirty sectors; have 15. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
@@ -2483,7 +2483,7 @@ write -P0x76 0x3ff0000 0x10000
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -2531,7 +2531,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -2547,7 +2547,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "full", "target": "backup_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "full", "target": "backup_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"action": "report", "device": "backup_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{"data": {"device": "backup_1", "error": "Input/output error", "len": 67108864, "offset": 983040, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -2609,7 +2609,7 @@ expecting 14 dirty sectors; have 14. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -2623,7 +2623,7 @@ expecting 14 dirty sectors; have 14. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
@@ -2694,7 +2694,7 @@ write -P0x76 0x3ff0000 0x10000
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -2742,7 +2742,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -2756,7 +2756,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "full", "target": "backup_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "full", "target": "backup_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
--- Write #2 ---
@@ -2869,7 +2869,7 @@ expecting 12 dirty sectors; have 12. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -2883,7 +2883,7 @@ expecting 12 dirty sectors; have 12. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
@@ -2954,7 +2954,7 @@ write -P0x76 0x3ff0000 0x10000
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -3002,7 +3002,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -3016,7 +3016,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "full", "target": "backup_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "full", "target": "backup_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
--- Write #2 ---
@@ -3129,7 +3129,7 @@ expecting 12 dirty sectors; have 12. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -3143,7 +3143,7 @@ expecting 12 dirty sectors; have 12. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
@@ -3214,7 +3214,7 @@ write -P0x76 0x3ff0000 0x10000
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -3262,7 +3262,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -3278,7 +3278,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "full", "target": "backup_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "full", "target": "backup_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"action": "report", "device": "backup_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{"data": {"device": "backup_1", "error": "Input/output error", "len": 67108864, "offset": 983040, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -3340,7 +3340,7 @@ expecting 1014 dirty sectors; have 1014. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -3354,7 +3354,7 @@ expecting 1014 dirty sectors; have 1014. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
@@ -3425,7 +3425,7 @@ write -P0x76 0x3ff0000 0x10000
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -3473,7 +3473,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -3487,7 +3487,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "full", "target": "backup_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "full", "target": "backup_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
--- Write #2 ---
@@ -3600,7 +3600,7 @@ expecting 12 dirty sectors; have 12. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -3614,7 +3614,7 @@ expecting 12 dirty sectors; have 12. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
@@ -3685,7 +3685,7 @@ write -P0x76 0x3ff0000 0x10000
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -3733,7 +3733,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -3747,7 +3747,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "top", "target": "backup_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "top", "target": "backup_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
--- Write #2 ---
@@ -3860,7 +3860,7 @@ expecting 15 dirty sectors; have 15. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -3874,7 +3874,7 @@ expecting 15 dirty sectors; have 15. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
@@ -3945,7 +3945,7 @@ write -P0x76 0x3ff0000 0x10000
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -3993,7 +3993,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -4009,7 +4009,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "top", "target": "backup_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "top", "target": "backup_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"action": "report", "device": "backup_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{"data": {"device": "backup_1", "error": "Input/output error", "len": 458752, "offset": 65536, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -4071,7 +4071,7 @@ expecting 14 dirty sectors; have 14. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -4085,7 +4085,7 @@ expecting 14 dirty sectors; have 14. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
@@ -4156,7 +4156,7 @@ write -P0x76 0x3ff0000 0x10000
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -4204,7 +4204,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -4218,7 +4218,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "top", "target": "backup_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "top", "target": "backup_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
--- Write #2 ---
@@ -4331,7 +4331,7 @@ expecting 12 dirty sectors; have 12. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -4345,7 +4345,7 @@ expecting 12 dirty sectors; have 12. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
@@ -4416,7 +4416,7 @@ write -P0x76 0x3ff0000 0x10000
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -4464,7 +4464,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -4478,7 +4478,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "top", "target": "backup_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "top", "target": "backup_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
--- Write #2 ---
@@ -4591,7 +4591,7 @@ expecting 12 dirty sectors; have 12. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -4605,7 +4605,7 @@ expecting 12 dirty sectors; have 12. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
@@ -4676,7 +4676,7 @@ write -P0x76 0x3ff0000 0x10000
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -4724,7 +4724,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -4740,7 +4740,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "top", "target": "backup_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "top", "target": "backup_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"action": "report", "device": "backup_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{"data": {"device": "backup_1", "error": "Input/output error", "len": 458752, "offset": 65536, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -4802,7 +4802,7 @@ expecting 14 dirty sectors; have 14. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -4816,7 +4816,7 @@ expecting 14 dirty sectors; have 14. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
@@ -4887,7 +4887,7 @@ write -P0x76 0x3ff0000 0x10000
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -4935,7 +4935,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -4949,7 +4949,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "top", "target": "backup_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_1", "sync": "top", "target": "backup_target_1", "x-perf": {"max-workers": 1}}}
{"return": {}}
--- Write #2 ---
@@ -5062,7 +5062,7 @@ expecting 12 dirty sectors; have 12. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
@@ -5076,7 +5076,7 @@ expecting 12 dirty sectors; have 12. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2", "x-perf": {"max-workers": 1}}}
{"return": {}}
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
@@ -5139,155 +5139,155 @@ qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
-- Sync mode incremental tests --
-{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "incremental", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "incremental", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "must provide a valid bitmap name for 'incremental' sync mode"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "incremental", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "incremental", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "must provide a valid bitmap name for 'incremental' sync mode"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "incremental", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "incremental", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "must provide a valid bitmap name for 'incremental' sync mode"}}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "incremental", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "incremental", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "must provide a valid bitmap name for 'incremental' sync mode"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "incremental", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "incremental", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "incremental", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "incremental", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap sync mode must be 'on-success' when using sync mode 'incremental'"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "incremental", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "incremental", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap sync mode must be 'on-success' when using sync mode 'incremental'"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "incremental", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "incremental", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "incremental", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "incremental", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap sync mode must be 'on-success' when using sync mode 'incremental'"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "incremental", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "incremental", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap sync mode must be 'on-success' when using sync mode 'incremental'"}}
-- Sync mode bitmap tests --
-{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "bitmap", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "bitmap", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "must provide a valid bitmap name for 'bitmap' sync mode"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "bitmap", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "bitmap", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "must provide a valid bitmap name for 'bitmap' sync mode"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "bitmap", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "bitmap", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "must provide a valid bitmap name for 'bitmap' sync mode"}}
-{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "bitmap", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "bitmap", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "must provide a valid bitmap name for 'bitmap' sync mode"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "bitmap", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "bitmap", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "bitmap", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "bitmap", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "bitmap", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "bitmap", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "bitmap", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "bitmap", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "bitmap", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "bitmap", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap sync mode must be given when providing a bitmap"}}
-- Sync mode full tests --
-{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "full", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "full", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mode without a bitmap"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "full", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "full", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mode without a bitmap"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "full", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "full", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mode without a bitmap"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "full", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "full", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "full", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "full", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "full", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "full", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "full", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "full", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "full", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "full", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap sync mode 'never' has no meaningful effect when combined with sync mode 'full'"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "full", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "full", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap sync mode must be given when providing a bitmap"}}
-- Sync mode top tests --
-{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "top", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "top", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mode without a bitmap"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "top", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "top", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mode without a bitmap"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "top", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "top", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mode without a bitmap"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "top", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "top", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "top", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "top", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "top", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "top", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "top", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "top", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "top", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "top", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap sync mode 'never' has no meaningful effect when combined with sync mode 'top'"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "top", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "top", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap sync mode must be given when providing a bitmap"}}
-- Sync mode none tests --
-{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "none", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "none", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mode without a bitmap"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "none", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "none", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mode without a bitmap"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "none", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "none", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mode without a bitmap"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "none", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "none", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "none", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "none", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "none", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "none", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "none", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "none", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "none", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "none", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "sync mode 'none' does not produce meaningful bitmap outputs"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "none", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "none", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "sync mode 'none' does not produce meaningful bitmap outputs"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "none", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "none", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "sync mode 'none' does not produce meaningful bitmap outputs"}}
-{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "none", "target": "backup_target"}}
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": "none", "target": "backup_target", "x-perf": {"max-workers": 1}}}
{"error": {"class": "GenericError", "desc": "Bitmap sync mode must be given when providing a bitmap"}}
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 41/53] block/block-copy: make progress_bytes_callback optional
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (39 preceding siblings ...)
2021-01-26 14:20 ` [PULL 40/53] iotests: 257: " Max Reitz
@ 2021-01-26 14:20 ` Max Reitz
2021-01-26 14:20 ` [PULL 42/53] block/backup: drop extra gotos from backup_run() Max Reitz
` (12 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
We are going to stop use of this callback in the following commit.
Still the callback handling code will be dropped in a separate commit.
So, for now let's make it optional.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-16-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/block-copy.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/block/block-copy.c b/block/block-copy.c
index 82cf945693..61d82d9a1c 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -454,7 +454,9 @@ static coroutine_fn int block_copy_task_entry(AioTask *task)
t->call_state->error_is_read = error_is_read;
} else {
progress_work_done(t->s->progress, t->bytes);
- t->s->progress_bytes_callback(t->bytes, t->s->progress_opaque);
+ if (t->s->progress_bytes_callback) {
+ t->s->progress_bytes_callback(t->bytes, t->s->progress_opaque);
+ }
}
co_put_to_shres(t->s->mem, t->bytes);
block_copy_task_end(t, ret);
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 42/53] block/backup: drop extra gotos from backup_run()
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (40 preceding siblings ...)
2021-01-26 14:20 ` [PULL 41/53] block/block-copy: make progress_bytes_callback optional Max Reitz
@ 2021-01-26 14:20 ` Max Reitz
2021-01-26 14:20 ` [PULL 43/53] backup: move to block-copy Max Reitz
` (11 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-17-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/backup.c | 12 +++++-------
1 file changed, 5 insertions(+), 7 deletions(-)
diff --git a/block/backup.c b/block/backup.c
index 5522c0f3fe..466608ee55 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -236,7 +236,7 @@ static void backup_init_bcs_bitmap(BackupBlockJob *job)
static int coroutine_fn backup_run(Job *job, Error **errp)
{
BackupBlockJob *s = container_of(job, BackupBlockJob, common.job);
- int ret = 0;
+ int ret;
backup_init_bcs_bitmap(s);
@@ -246,13 +246,12 @@ static int coroutine_fn backup_run(Job *job, Error **errp)
for (offset = 0; offset < s->len; ) {
if (yield_and_check(s)) {
- ret = -ECANCELED;
- goto out;
+ return -ECANCELED;
}
ret = block_copy_reset_unallocated(s->bcs, offset, &count);
if (ret < 0) {
- goto out;
+ return ret;
}
offset += count;
@@ -273,11 +272,10 @@ static int coroutine_fn backup_run(Job *job, Error **errp)
job_yield(job);
}
} else {
- ret = backup_loop(s);
+ return backup_loop(s);
}
- out:
- return ret;
+ return 0;
}
static const BlockJobDriver backup_job_driver = {
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 43/53] backup: move to block-copy
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (41 preceding siblings ...)
2021-01-26 14:20 ` [PULL 42/53] block/backup: drop extra gotos from backup_run() Max Reitz
@ 2021-01-26 14:20 ` Max Reitz
2021-01-26 14:20 ` [PULL 44/53] qapi: backup: disable copy_range by default Max Reitz
` (10 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
This brings async request handling and block-status driven chunk sizes
to backup out of the box, which improves backup performance.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-18-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/backup.c | 187 +++++++++++++++++++++++++++++++------------------
1 file changed, 120 insertions(+), 67 deletions(-)
diff --git a/block/backup.c b/block/backup.c
index 466608ee55..cc525d5544 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -22,7 +22,6 @@
#include "block/block-copy.h"
#include "qapi/error.h"
#include "qapi/qmp/qerror.h"
-#include "qemu/ratelimit.h"
#include "qemu/cutils.h"
#include "sysemu/block-backend.h"
#include "qemu/bitmap.h"
@@ -44,41 +43,17 @@ typedef struct BackupBlockJob {
BlockdevOnError on_source_error;
BlockdevOnError on_target_error;
uint64_t len;
- uint64_t bytes_read;
int64_t cluster_size;
BackupPerf perf;
BlockCopyState *bcs;
+
+ bool wait;
+ BlockCopyCallState *bg_bcs_call;
} BackupBlockJob;
static const BlockJobDriver backup_job_driver;
-static void backup_progress_bytes_callback(int64_t bytes, void *opaque)
-{
- BackupBlockJob *s = opaque;
-
- s->bytes_read += bytes;
-}
-
-static int coroutine_fn backup_do_cow(BackupBlockJob *job,
- int64_t offset, uint64_t bytes,
- bool *error_is_read)
-{
- int ret = 0;
- int64_t start, end; /* bytes */
-
- start = QEMU_ALIGN_DOWN(offset, job->cluster_size);
- end = QEMU_ALIGN_UP(bytes + offset, job->cluster_size);
-
- trace_backup_do_cow_enter(job, start, offset, bytes);
-
- ret = block_copy(job->bcs, start, end - start, true, error_is_read);
-
- trace_backup_do_cow_return(job, offset, bytes, ret);
-
- return ret;
-}
-
static void backup_cleanup_sync_bitmap(BackupBlockJob *job, int ret)
{
BdrvDirtyBitmap *bm;
@@ -158,53 +133,96 @@ static BlockErrorAction backup_error_action(BackupBlockJob *job,
}
}
-static bool coroutine_fn yield_and_check(BackupBlockJob *job)
+static void coroutine_fn backup_block_copy_callback(void *opaque)
{
- uint64_t delay_ns;
-
- if (job_is_cancelled(&job->common.job)) {
- return true;
- }
-
- /*
- * We need to yield even for delay_ns = 0 so that bdrv_drain_all() can
- * return. Without a yield, the VM would not reboot.
- */
- delay_ns = block_job_ratelimit_get_delay(&job->common, job->bytes_read);
- job->bytes_read = 0;
- job_sleep_ns(&job->common.job, delay_ns);
+ BackupBlockJob *s = opaque;
- if (job_is_cancelled(&job->common.job)) {
- return true;
+ if (s->wait) {
+ s->wait = false;
+ aio_co_wake(s->common.job.co);
+ } else {
+ job_enter(&s->common.job);
}
-
- return false;
}
static int coroutine_fn backup_loop(BackupBlockJob *job)
{
- bool error_is_read;
- int64_t offset;
- BdrvDirtyBitmapIter *bdbi;
+ BlockCopyCallState *s = NULL;
int ret = 0;
+ bool error_is_read;
+ BlockErrorAction act;
+
+ while (true) { /* retry loop */
+ job->bg_bcs_call = s = block_copy_async(job->bcs, 0,
+ QEMU_ALIGN_UP(job->len, job->cluster_size),
+ job->perf.max_workers, job->perf.max_chunk,
+ backup_block_copy_callback, job);
+
+ while (!block_copy_call_finished(s) &&
+ !job_is_cancelled(&job->common.job))
+ {
+ job_yield(&job->common.job);
+ }
- bdbi = bdrv_dirty_iter_new(block_copy_dirty_bitmap(job->bcs));
- while ((offset = bdrv_dirty_iter_next(bdbi)) != -1) {
- do {
- if (yield_and_check(job)) {
- goto out;
- }
- ret = backup_do_cow(job, offset, job->cluster_size, &error_is_read);
- if (ret < 0 && backup_error_action(job, error_is_read, -ret) ==
- BLOCK_ERROR_ACTION_REPORT)
- {
- goto out;
- }
- } while (ret < 0);
+ if (!block_copy_call_finished(s)) {
+ assert(job_is_cancelled(&job->common.job));
+ /*
+ * Note that we can't use job_yield() here, as it doesn't work for
+ * cancelled job.
+ */
+ block_copy_call_cancel(s);
+ job->wait = true;
+ qemu_coroutine_yield();
+ assert(block_copy_call_finished(s));
+ ret = 0;
+ goto out;
+ }
+
+ if (job_is_cancelled(&job->common.job) ||
+ block_copy_call_succeeded(s))
+ {
+ ret = 0;
+ goto out;
+ }
+
+ if (block_copy_call_cancelled(s)) {
+ /*
+ * Job is not cancelled but only block-copy call. This is possible
+ * after job pause. Now the pause is finished, start new block-copy
+ * iteration.
+ */
+ block_copy_call_free(s);
+ continue;
+ }
+
+ /* The only remaining case is failed block-copy call. */
+ assert(block_copy_call_failed(s));
+
+ ret = block_copy_call_status(s, &error_is_read);
+ act = backup_error_action(job, error_is_read, -ret);
+ switch (act) {
+ case BLOCK_ERROR_ACTION_REPORT:
+ goto out;
+ case BLOCK_ERROR_ACTION_STOP:
+ /*
+ * Go to pause prior to starting new block-copy call on the next
+ * iteration.
+ */
+ job_pause_point(&job->common.job);
+ break;
+ case BLOCK_ERROR_ACTION_IGNORE:
+ /* Proceed to new block-copy call to retry. */
+ break;
+ default:
+ abort();
+ }
+
+ block_copy_call_free(s);
}
- out:
- bdrv_dirty_iter_free(bdbi);
+out:
+ block_copy_call_free(s);
+ job->bg_bcs_call = NULL;
return ret;
}
@@ -245,7 +263,13 @@ static int coroutine_fn backup_run(Job *job, Error **errp)
int64_t count;
for (offset = 0; offset < s->len; ) {
- if (yield_and_check(s)) {
+ if (job_is_cancelled(job)) {
+ return -ECANCELED;
+ }
+
+ job_pause_point(job);
+
+ if (job_is_cancelled(job)) {
return -ECANCELED;
}
@@ -278,6 +302,33 @@ static int coroutine_fn backup_run(Job *job, Error **errp)
return 0;
}
+static void coroutine_fn backup_pause(Job *job)
+{
+ BackupBlockJob *s = container_of(job, BackupBlockJob, common.job);
+
+ if (s->bg_bcs_call && !block_copy_call_finished(s->bg_bcs_call)) {
+ block_copy_call_cancel(s->bg_bcs_call);
+ s->wait = true;
+ qemu_coroutine_yield();
+ }
+}
+
+static void coroutine_fn backup_set_speed(BlockJob *job, int64_t speed)
+{
+ BackupBlockJob *s = container_of(job, BackupBlockJob, common);
+
+ /*
+ * block_job_set_speed() is called first from block_job_create(), when we
+ * don't yet have s->bcs.
+ */
+ if (s->bcs) {
+ block_copy_set_speed(s->bcs, speed);
+ if (s->bg_bcs_call) {
+ block_copy_kick(s->bg_bcs_call);
+ }
+ }
+}
+
static const BlockJobDriver backup_job_driver = {
.job_driver = {
.instance_size = sizeof(BackupBlockJob),
@@ -288,7 +339,9 @@ static const BlockJobDriver backup_job_driver = {
.commit = backup_commit,
.abort = backup_abort,
.clean = backup_clean,
- }
+ .pause = backup_pause,
+ },
+ .set_speed = backup_set_speed,
};
static int64_t backup_calculate_cluster_size(BlockDriverState *target,
@@ -485,8 +538,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
job->len = len;
job->perf = *perf;
- block_copy_set_progress_callback(bcs, backup_progress_bytes_callback, job);
block_copy_set_progress_meter(bcs, &job->common.job.progress);
+ block_copy_set_speed(bcs, speed);
/* Required permissions are already taken by backup-top target */
block_job_add_bdrv(&job->common, "target", target, 0, BLK_PERM_ALL,
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 44/53] qapi: backup: disable copy_range by default
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (42 preceding siblings ...)
2021-01-26 14:20 ` [PULL 43/53] backup: move to block-copy Max Reitz
@ 2021-01-26 14:20 ` Max Reitz
2021-01-26 14:20 ` [PULL 45/53] block/block-copy: drop unused block_copy_set_progress_callback() Max Reitz
` (9 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Further commit will add a benchmark
(scripts/simplebench/bench-backup.py), which will show that backup
works better with async parallel requests (previous commit) and
disabled copy_range. So, let's disable copy_range by default.
Note: the option was added several commits ago with default to true,
to follow old behavior (the feature was enabled unconditionally), and
only now we are going to change the default behavior.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-19-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
qapi/block-core.json | 2 +-
blockdev.c | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index abcd41ed63..9f555d5c1d 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1377,7 +1377,7 @@
# Optional parameters for backup. These parameters don't affect
# functionality, but may significantly affect performance.
#
-# @use-copy-range: Use copy offloading. Default true.
+# @use-copy-range: Use copy offloading. Default false.
#
# @max-workers: Maximum number of parallel requests for the sustained background
# copying process. Doesn't influence copy-before-write operations.
diff --git a/blockdev.c b/blockdev.c
index 25aaacf253..93417f6302 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2829,7 +2829,7 @@ static BlockJob *do_backup_common(BackupCommon *backup,
{
BlockJob *job = NULL;
BdrvDirtyBitmap *bmap = NULL;
- BackupPerf perf = { .use_copy_range = true, .max_workers = 64 };
+ BackupPerf perf = { .max_workers = 64 };
int job_flags = JOB_DEFAULT;
if (!backup->has_speed) {
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 45/53] block/block-copy: drop unused block_copy_set_progress_callback()
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (43 preceding siblings ...)
2021-01-26 14:20 ` [PULL 44/53] qapi: backup: disable copy_range by default Max Reitz
@ 2021-01-26 14:20 ` Max Reitz
2021-01-26 14:20 ` [PULL 46/53] block/block-copy: drop unused argument of block_copy() Max Reitz
` (8 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Drop unused code.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-20-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
include/block/block-copy.h | 6 ------
block/block-copy.c | 15 ---------------
2 files changed, 21 deletions(-)
diff --git a/include/block/block-copy.h b/include/block/block-copy.h
index 7821850f88..1cbea0b79b 100644
--- a/include/block/block-copy.h
+++ b/include/block/block-copy.h
@@ -18,7 +18,6 @@
#include "block/block.h"
#include "qemu/co-shared-resource.h"
-typedef void (*ProgressBytesCallbackFunc)(int64_t bytes, void *opaque);
typedef void (*BlockCopyAsyncCallbackFunc)(void *opaque);
typedef struct BlockCopyState BlockCopyState;
typedef struct BlockCopyCallState BlockCopyCallState;
@@ -28,11 +27,6 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
BdrvRequestFlags write_flags,
Error **errp);
-void block_copy_set_progress_callback(
- BlockCopyState *s,
- ProgressBytesCallbackFunc progress_bytes_callback,
- void *progress_opaque);
-
void block_copy_set_progress_meter(BlockCopyState *s, ProgressMeter *pm);
void block_copy_state_free(BlockCopyState *s);
diff --git a/block/block-copy.c b/block/block-copy.c
index 61d82d9a1c..2ea8b28684 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -110,9 +110,6 @@ typedef struct BlockCopyState {
bool skip_unallocated;
ProgressMeter *progress;
- /* progress_bytes_callback: called when some copying progress is done. */
- ProgressBytesCallbackFunc progress_bytes_callback;
- void *progress_opaque;
SharedResource *mem;
@@ -298,15 +295,6 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
return s;
}
-void block_copy_set_progress_callback(
- BlockCopyState *s,
- ProgressBytesCallbackFunc progress_bytes_callback,
- void *progress_opaque)
-{
- s->progress_bytes_callback = progress_bytes_callback;
- s->progress_opaque = progress_opaque;
-}
-
void block_copy_set_progress_meter(BlockCopyState *s, ProgressMeter *pm)
{
s->progress = pm;
@@ -454,9 +442,6 @@ static coroutine_fn int block_copy_task_entry(AioTask *task)
t->call_state->error_is_read = error_is_read;
} else {
progress_work_done(t->s->progress, t->bytes);
- if (t->s->progress_bytes_callback) {
- t->s->progress_bytes_callback(t->bytes, t->s->progress_opaque);
- }
}
co_put_to_shres(t->s->mem, t->bytes);
block_copy_task_end(t, ret);
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 46/53] block/block-copy: drop unused argument of block_copy()
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (44 preceding siblings ...)
2021-01-26 14:20 ` [PULL 45/53] block/block-copy: drop unused block_copy_set_progress_callback() Max Reitz
@ 2021-01-26 14:20 ` Max Reitz
2021-01-26 14:20 ` [PULL 47/53] simplebench/bench_block_job: use correct shebang line with python3 Max Reitz
` (7 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-21-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
include/block/block-copy.h | 2 +-
block/backup-top.c | 2 +-
block/block-copy.c | 10 ++--------
3 files changed, 4 insertions(+), 10 deletions(-)
diff --git a/include/block/block-copy.h b/include/block/block-copy.h
index 1cbea0b79b..338f2ea7fd 100644
--- a/include/block/block-copy.h
+++ b/include/block/block-copy.h
@@ -35,7 +35,7 @@ int64_t block_copy_reset_unallocated(BlockCopyState *s,
int64_t offset, int64_t *count);
int coroutine_fn block_copy(BlockCopyState *s, int64_t offset, int64_t bytes,
- bool ignore_ratelimit, bool *error_is_read);
+ bool ignore_ratelimit);
/*
* Run block-copy in a coroutine, create corresponding BlockCopyCallState
diff --git a/block/backup-top.c b/block/backup-top.c
index 779956ddc2..6e7e7bf340 100644
--- a/block/backup-top.c
+++ b/block/backup-top.c
@@ -61,7 +61,7 @@ static coroutine_fn int backup_top_cbw(BlockDriverState *bs, uint64_t offset,
off = QEMU_ALIGN_DOWN(offset, s->cluster_size);
end = QEMU_ALIGN_UP(offset + bytes, s->cluster_size);
- return block_copy(s->bcs, off, end - off, true, NULL);
+ return block_copy(s->bcs, off, end - off, true);
}
static int coroutine_fn backup_top_co_pdiscard(BlockDriverState *bs,
diff --git a/block/block-copy.c b/block/block-copy.c
index 2ea8b28684..39ae481c8b 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -723,7 +723,7 @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state)
}
int coroutine_fn block_copy(BlockCopyState *s, int64_t start, int64_t bytes,
- bool ignore_ratelimit, bool *error_is_read)
+ bool ignore_ratelimit)
{
BlockCopyCallState call_state = {
.s = s,
@@ -733,13 +733,7 @@ int coroutine_fn block_copy(BlockCopyState *s, int64_t start, int64_t bytes,
.max_workers = BLOCK_COPY_MAX_WORKERS,
};
- int ret = block_copy_common(&call_state);
-
- if (error_is_read && ret < 0) {
- *error_is_read = call_state.error_is_read;
- }
-
- return ret;
+ return block_copy_common(&call_state);
}
static void coroutine_fn block_copy_async_co_entry(void *opaque)
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 47/53] simplebench/bench_block_job: use correct shebang line with python3
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (45 preceding siblings ...)
2021-01-26 14:20 ` [PULL 46/53] block/block-copy: drop unused argument of block_copy() Max Reitz
@ 2021-01-26 14:20 ` Max Reitz
2021-01-26 14:20 ` [PULL 48/53] simplebench: bench_block_job: add cmd_options argument Max Reitz
` (6 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-22-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
scripts/simplebench/bench_block_job.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/simplebench/bench_block_job.py b/scripts/simplebench/bench_block_job.py
index 9808d696cf..a0dda1dc4e 100755
--- a/scripts/simplebench/bench_block_job.py
+++ b/scripts/simplebench/bench_block_job.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
#
# Benchmark block jobs
#
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 48/53] simplebench: bench_block_job: add cmd_options argument
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (46 preceding siblings ...)
2021-01-26 14:20 ` [PULL 47/53] simplebench/bench_block_job: use correct shebang line with python3 Max Reitz
@ 2021-01-26 14:20 ` Max Reitz
2021-01-26 14:20 ` [PULL 49/53] simplebench: add bench-backup.py Max Reitz
` (5 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Add argument to allow additional block-job options.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-23-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
scripts/simplebench/bench-example.py | 2 +-
scripts/simplebench/bench_block_job.py | 11 +++++++----
2 files changed, 8 insertions(+), 5 deletions(-)
diff --git a/scripts/simplebench/bench-example.py b/scripts/simplebench/bench-example.py
index d9c7f7bc17..4864435f39 100644
--- a/scripts/simplebench/bench-example.py
+++ b/scripts/simplebench/bench-example.py
@@ -25,7 +25,7 @@ from bench_block_job import bench_block_copy, drv_file, drv_nbd
def bench_func(env, case):
""" Handle one "cell" of benchmarking table. """
- return bench_block_copy(env['qemu_binary'], env['cmd'],
+ return bench_block_copy(env['qemu_binary'], env['cmd'], {}
case['source'], case['target'])
diff --git a/scripts/simplebench/bench_block_job.py b/scripts/simplebench/bench_block_job.py
index a0dda1dc4e..7332845c1c 100755
--- a/scripts/simplebench/bench_block_job.py
+++ b/scripts/simplebench/bench_block_job.py
@@ -78,16 +78,19 @@ def bench_block_job(cmd, cmd_args, qemu_args):
# Bench backup or mirror
-def bench_block_copy(qemu_binary, cmd, source, target):
+def bench_block_copy(qemu_binary, cmd, cmd_options, source, target):
"""Helper to run bench_block_job() for mirror or backup"""
assert cmd in ('blockdev-backup', 'blockdev-mirror')
source['node-name'] = 'source'
target['node-name'] = 'target'
- return bench_block_job(cmd,
- {'job-id': 'job0', 'device': 'source',
- 'target': 'target', 'sync': 'full'},
+ cmd_options['job-id'] = 'job0'
+ cmd_options['device'] = 'source'
+ cmd_options['target'] = 'target'
+ cmd_options['sync'] = 'full'
+
+ return bench_block_job(cmd, cmd_options,
[qemu_binary,
'-blockdev', json.dumps(source),
'-blockdev', json.dumps(target)])
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 49/53] simplebench: add bench-backup.py
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (47 preceding siblings ...)
2021-01-26 14:20 ` [PULL 48/53] simplebench: bench_block_job: add cmd_options argument Max Reitz
@ 2021-01-26 14:20 ` Max Reitz
2021-01-26 14:20 ` [PULL 50/53] block: report errno when flock fcntl fails Max Reitz
` (4 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Add script to benchmark new backup architecture.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210116214705.822267-24-vsementsov@virtuozzo.com>
[mreitz: s/not unsupported/not supported/]
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
scripts/simplebench/bench-backup.py | 167 ++++++++++++++++++++++++++++
1 file changed, 167 insertions(+)
create mode 100755 scripts/simplebench/bench-backup.py
diff --git a/scripts/simplebench/bench-backup.py b/scripts/simplebench/bench-backup.py
new file mode 100755
index 0000000000..33a1ecfefa
--- /dev/null
+++ b/scripts/simplebench/bench-backup.py
@@ -0,0 +1,167 @@
+#!/usr/bin/env python3
+#
+# Bench backup block-job
+#
+# Copyright (c) 2020 Virtuozzo International GmbH.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+import argparse
+import json
+
+import simplebench
+from results_to_text import results_to_text
+from bench_block_job import bench_block_copy, drv_file, drv_nbd
+
+
+def bench_func(env, case):
+ """ Handle one "cell" of benchmarking table. """
+ cmd_options = env['cmd-options'] if 'cmd-options' in env else {}
+ return bench_block_copy(env['qemu-binary'], env['cmd'],
+ cmd_options,
+ case['source'], case['target'])
+
+
+def bench(args):
+ test_cases = []
+
+ sources = {}
+ targets = {}
+ for d in args.dir:
+ label, path = d.split(':') # paths with colon not supported
+ sources[label] = drv_file(path + '/test-source')
+ targets[label] = drv_file(path + '/test-target')
+
+ if args.nbd:
+ nbd = args.nbd.split(':')
+ host = nbd[0]
+ port = '10809' if len(nbd) == 1 else nbd[1]
+ drv = drv_nbd(host, port)
+ sources['nbd'] = drv
+ targets['nbd'] = drv
+
+ for t in args.test:
+ src, dst = t.split(':')
+
+ test_cases.append({
+ 'id': t,
+ 'source': sources[src],
+ 'target': targets[dst]
+ })
+
+ binaries = [] # list of (<label>, <path>, [<options>])
+ for i, q in enumerate(args.env):
+ name_path = q.split(':')
+ if len(name_path) == 1:
+ label = f'q{i}'
+ path_opts = name_path[0].split(',')
+ else:
+ assert len(name_path) == 2 # paths with colon not supported
+ label = name_path[0]
+ path_opts = name_path[1].split(',')
+
+ binaries.append((label, path_opts[0], path_opts[1:]))
+
+ test_envs = []
+
+ bin_paths = {}
+ for i, q in enumerate(args.env):
+ opts = q.split(',')
+ label_path = opts[0]
+ opts = opts[1:]
+
+ if ':' in label_path:
+ # path with colon inside is not supported
+ label, path = label_path.split(':')
+ bin_paths[label] = path
+ elif label_path in bin_paths:
+ label = label_path
+ path = bin_paths[label]
+ else:
+ path = label_path
+ label = f'q{i}'
+ bin_paths[label] = path
+
+ x_perf = {}
+ is_mirror = False
+ for opt in opts:
+ if opt == 'mirror':
+ is_mirror = True
+ elif opt == 'copy-range=on':
+ x_perf['use-copy-range'] = True
+ elif opt == 'copy-range=off':
+ x_perf['use-copy-range'] = False
+ elif opt.startswith('max-workers='):
+ x_perf['max-workers'] = int(opt.split('=')[1])
+
+ if is_mirror:
+ assert not x_perf
+ test_envs.append({
+ 'id': f'mirror({label})',
+ 'cmd': 'blockdev-mirror',
+ 'qemu-binary': path
+ })
+ else:
+ test_envs.append({
+ 'id': f'backup({label})\n' + '\n'.join(opts),
+ 'cmd': 'blockdev-backup',
+ 'cmd-options': {'x-perf': x_perf} if x_perf else {},
+ 'qemu-binary': path
+ })
+
+ result = simplebench.bench(bench_func, test_envs, test_cases, count=3)
+ with open('results.json', 'w') as f:
+ json.dump(result, f, indent=4)
+ print(results_to_text(result))
+
+
+class ExtendAction(argparse.Action):
+ def __call__(self, parser, namespace, values, option_string=None):
+ items = getattr(namespace, self.dest) or []
+ items.extend(values)
+ setattr(namespace, self.dest, items)
+
+
+if __name__ == '__main__':
+ p = argparse.ArgumentParser('Backup benchmark', epilog='''
+ENV format
+
+ (LABEL:PATH|LABEL|PATH)[,max-workers=N][,use-copy-range=(on|off)][,mirror]
+
+ LABEL short name for the binary
+ PATH path to the binary
+ max-workers set x-perf.max-workers of backup job
+ use-copy-range set x-perf.use-copy-range of backup job
+ mirror use mirror job instead of backup''',
+ formatter_class=argparse.RawTextHelpFormatter)
+ p.add_argument('--env', nargs='+', help='''\
+Qemu binaries with labels and options, see below
+"ENV format" section''',
+ action=ExtendAction)
+ p.add_argument('--dir', nargs='+', help='''\
+Directories, each containing "test-source" and/or
+"test-target" files, raw images to used in
+benchmarking. File path with label, like
+label:/path/to/directory''',
+ action=ExtendAction)
+ p.add_argument('--nbd', help='''\
+host:port for remote NBD image, (or just host, for
+default port 10809). Use it in tests, label is "nbd"
+(but you cannot create test nbd:nbd).''')
+ p.add_argument('--test', nargs='+', help='''\
+Tests, in form source-dir-label:target-dir-label''',
+ action=ExtendAction)
+
+ bench(p.parse_args())
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 50/53] block: report errno when flock fcntl fails
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (48 preceding siblings ...)
2021-01-26 14:20 ` [PULL 49/53] simplebench: add bench-backup.py Max Reitz
@ 2021-01-26 14:20 ` Max Reitz
2021-01-26 14:20 ` [PULL 51/53] iotests: Add test for the regression fixed in c8bf9a9169 Max Reitz
` (3 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: David Edmondson <david.edmondson@oracle.com>
When a call to fcntl(2) for the purpose of adding file locks fails
with an error other than EAGAIN or EACCES, report the error returned
by fcntl.
EAGAIN or EACCES are elided as they are considered to be common
failures, indicating that a conflicting lock is held by another
process.
No errors are elided when removing file locks.
Signed-off-by: David Edmondson <david.edmondson@oracle.com>
Message-Id: <20210113164447.2545785-1-david.edmondson@oracle.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/file-posix.c | 38 ++++++++++++++++++++++++++++----------
1 file changed, 28 insertions(+), 10 deletions(-)
diff --git a/block/file-posix.c b/block/file-posix.c
index 00cdaaa2d4..11aafa9d82 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -216,6 +216,20 @@ typedef struct RawPosixAIOData {
static int cdrom_reopen(BlockDriverState *bs);
#endif
+/*
+ * Elide EAGAIN and EACCES details when failing to lock, as this
+ * indicates that the specified file region is already locked by
+ * another process, which is considered a common scenario.
+ */
+#define raw_lock_error_setg_errno(errp, err, fmt, ...) \
+ do { \
+ if ((err) == EAGAIN || (err) == EACCES) { \
+ error_setg((errp), (fmt), ## __VA_ARGS__); \
+ } else { \
+ error_setg_errno((errp), (err), (fmt), ## __VA_ARGS__); \
+ } \
+ } while (0)
+
#if defined(__NetBSD__)
static int raw_normalize_devicepath(const char **filename, Error **errp)
{
@@ -836,7 +850,8 @@ static int raw_apply_lock_bytes(BDRVRawState *s, int fd,
if ((perm_lock_bits & bit) && !(locked_perm & bit)) {
ret = qemu_lock_fd(fd, off, 1, false);
if (ret) {
- error_setg(errp, "Failed to lock byte %d", off);
+ raw_lock_error_setg_errno(errp, -ret, "Failed to lock byte %d",
+ off);
return ret;
} else if (s) {
s->locked_perm |= bit;
@@ -844,7 +859,7 @@ static int raw_apply_lock_bytes(BDRVRawState *s, int fd,
} else if (unlock && (locked_perm & bit) && !(perm_lock_bits & bit)) {
ret = qemu_unlock_fd(fd, off, 1);
if (ret) {
- error_setg(errp, "Failed to unlock byte %d", off);
+ error_setg_errno(errp, -ret, "Failed to unlock byte %d", off);
return ret;
} else if (s) {
s->locked_perm &= ~bit;
@@ -857,7 +872,8 @@ static int raw_apply_lock_bytes(BDRVRawState *s, int fd,
if ((shared_perm_lock_bits & bit) && !(locked_shared_perm & bit)) {
ret = qemu_lock_fd(fd, off, 1, false);
if (ret) {
- error_setg(errp, "Failed to lock byte %d", off);
+ raw_lock_error_setg_errno(errp, -ret, "Failed to lock byte %d",
+ off);
return ret;
} else if (s) {
s->locked_shared_perm |= bit;
@@ -866,7 +882,7 @@ static int raw_apply_lock_bytes(BDRVRawState *s, int fd,
!(shared_perm_lock_bits & bit)) {
ret = qemu_unlock_fd(fd, off, 1);
if (ret) {
- error_setg(errp, "Failed to unlock byte %d", off);
+ error_setg_errno(errp, -ret, "Failed to unlock byte %d", off);
return ret;
} else if (s) {
s->locked_shared_perm &= ~bit;
@@ -890,9 +906,10 @@ static int raw_check_lock_bytes(int fd, uint64_t perm, uint64_t shared_perm,
ret = qemu_lock_fd_test(fd, off, 1, true);
if (ret) {
char *perm_name = bdrv_perm_names(p);
- error_setg(errp,
- "Failed to get \"%s\" lock",
- perm_name);
+
+ raw_lock_error_setg_errno(errp, -ret,
+ "Failed to get \"%s\" lock",
+ perm_name);
g_free(perm_name);
return ret;
}
@@ -905,9 +922,10 @@ static int raw_check_lock_bytes(int fd, uint64_t perm, uint64_t shared_perm,
ret = qemu_lock_fd_test(fd, off, 1, true);
if (ret) {
char *perm_name = bdrv_perm_names(p);
- error_setg(errp,
- "Failed to get shared \"%s\" lock",
- perm_name);
+
+ raw_lock_error_setg_errno(errp, -ret,
+ "Failed to get shared \"%s\" lock",
+ perm_name);
g_free(perm_name);
return ret;
}
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 51/53] iotests: Add test for the regression fixed in c8bf9a9169
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (49 preceding siblings ...)
2021-01-26 14:20 ` [PULL 50/53] block: report errno when flock fcntl fails Max Reitz
@ 2021-01-26 14:20 ` Max Reitz
2021-01-26 14:20 ` [PULL 52/53] iotests/118: Drop 'change' test Max Reitz
` (2 subsequent siblings)
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Alberto Garcia <berto@igalia.com>
Signed-off-by: Alberto Garcia <berto@igalia.com>
Suggested-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-Id: <20210112170540.2912-1-berto@igalia.com>
[mreitz: Add "# group:" line]
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
tests/qemu-iotests/313 | 104 +++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/313.out | 29 +++++++++++
tests/qemu-iotests/group | 1 +
3 files changed, 134 insertions(+)
create mode 100755 tests/qemu-iotests/313
create mode 100644 tests/qemu-iotests/313.out
diff --git a/tests/qemu-iotests/313 b/tests/qemu-iotests/313
new file mode 100755
index 0000000000..a75655b7ef
--- /dev/null
+++ b/tests/qemu-iotests/313
@@ -0,0 +1,104 @@
+#!/usr/bin/env bash
+# group: rw auto quick
+#
+# Test for the regression fixed in commit c8bf9a9169
+#
+# Copyright (C) 2020 Igalia, S.L.
+# Author: Alberto Garcia <berto@igalia.com>
+# Based on a test case by Maxim Levitsky <mlevitsk@redhat.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+# creator
+owner=berto@igalia.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+status=1 # failure is the default!
+
+_cleanup()
+{
+ _cleanup_test_img
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_supported_fmt qcow2
+_supported_proto file
+_supported_os Linux
+_unsupported_imgopts cluster_size refcount_bits extended_l2 compat=0.10 data_file
+
+# The cluster size must be at least the granularity of the mirror job (4KB)
+# Note that larger cluster sizes will produce very large images (several GBs)
+cluster_size=4096
+refcount_bits=64 # Make it equal to the L2 entry size for convenience
+options="cluster_size=${cluster_size},refcount_bits=${refcount_bits}"
+
+# Number of refcount entries per refcount blocks
+ref_entries=$(( ${cluster_size} * 8 / ${refcount_bits} ))
+
+# Number of data clusters needed to fill a refcount block
+# Equals ${ref_entries} minus two (one L2 table and one refcount block)
+data_clusters_per_refblock=$(( ${ref_entries} - 2 ))
+
+# Number of entries in the refcount cache
+ref_blocks=4
+
+# Write enough data clusters to fill the refcount cache and allocate
+# one more refcount block.
+# Subtract 3 clusters from the total: qcow2 header, refcount table, L1 table
+total_data_clusters=$(( ${data_clusters_per_refblock} * ${ref_blocks} + 1 - 3 ))
+
+# Total size to write in bytes
+total_size=$(( ${total_data_clusters} * ${cluster_size} ))
+
+echo
+echo '### Create the image'
+echo
+TEST_IMG_FILE=$TEST_IMG.base _make_test_img -o $options $total_size | _filter_img_create_size
+
+echo
+echo '### Write data to allocate more refcount blocks than the cache can hold'
+echo
+$QEMU_IO -c "write -P 1 0 $total_size" $TEST_IMG.base | _filter_qemu_io
+
+echo
+echo '### Create an overlay'
+echo
+_make_test_img -F $IMGFMT -b $TEST_IMG.base -o $options | _filter_img_create_size
+
+echo
+echo '### Fill the overlay with zeroes'
+echo
+$QEMU_IO -c "write -z 0 $total_size" $TEST_IMG | _filter_qemu_io
+
+echo
+echo '### Commit changes to the base image'
+echo
+$QEMU_IMG commit $TEST_IMG
+
+echo
+echo '### Check the base image'
+echo
+$QEMU_IMG check $TEST_IMG.base
+
+# success, all done
+echo "*** done"
+rm -f $seq.full
+status=0
diff --git a/tests/qemu-iotests/313.out b/tests/qemu-iotests/313.out
new file mode 100644
index 0000000000..adb9f7bd95
--- /dev/null
+++ b/tests/qemu-iotests/313.out
@@ -0,0 +1,29 @@
+QA output created by 313
+
+### Create the image
+
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=SIZE
+
+### Write data to allocate more refcount blocks than the cache can hold
+
+wrote 8347648/8347648 bytes at offset 0
+7.961 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+### Create an overlay
+
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=SIZE backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
+
+### Fill the overlay with zeroes
+
+wrote 8347648/8347648 bytes at offset 0
+7.961 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+### Commit changes to the base image
+
+Image committed.
+
+### Check the base image
+
+No errors were found on the image.
+Image end offset: 8396800
+*** done
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index d4a3e36a9a..a0d0bf1688 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -320,3 +320,4 @@
309 rw auto quick
310 rw quick
312 rw quick
+313 rw auto quick
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 52/53] iotests/118: Drop 'change' test
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (50 preceding siblings ...)
2021-01-26 14:20 ` [PULL 51/53] iotests: Add test for the regression fixed in c8bf9a9169 Max Reitz
@ 2021-01-26 14:20 ` Max Reitz
2021-01-26 14:20 ` [PULL 53/53] iotests/178: Pass value to invalid option Max Reitz
2021-01-27 17:40 ` [PULL 00/53] Block patches Peter Maydell
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
Commit 0afec75734331 removed the 'change' QMP command, so we can no
longer test it in 118.
Fixes: 0afec75734331a0b52fa3aa4235220eda8c7846f
('qmp: remove deprecated "change" command')
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210126104833.57026-1-mreitz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
tests/qemu-iotests/118 | 20 +-------------------
tests/qemu-iotests/118.out | 4 ++--
2 files changed, 3 insertions(+), 21 deletions(-)
diff --git a/tests/qemu-iotests/118 b/tests/qemu-iotests/118
index 1a2e219057..cae52ffa5e 100755
--- a/tests/qemu-iotests/118
+++ b/tests/qemu-iotests/118
@@ -1,8 +1,7 @@
#!/usr/bin/env python3
# group: rw
#
-# Test case for the QMP 'change' command and all other associated
-# commands
+# Test case for media change monitor commands
#
# Copyright (C) 2015 Red Hat, Inc.
#
@@ -74,23 +73,6 @@ class ChangeBaseClass(iotests.QMPTestCase):
class GeneralChangeTestsBaseClass(ChangeBaseClass):
- def test_change(self):
- # 'change' requires a drive name, so skip the test for blockdev
- if not self.use_drive:
- return
-
- result = self.vm.qmp('change', device='drive0', target=new_img,
- arg=iotests.imgfmt)
- self.assert_qmp(result, 'return', {})
-
- self.wait_for_open()
- self.wait_for_close()
-
- result = self.vm.qmp('query-block')
- if self.has_real_tray:
- self.assert_qmp(result, 'return[0]/tray_open', False)
- self.assert_qmp(result, 'return[0]/inserted/image/filename', new_img)
-
def test_blockdev_change_medium(self):
result = self.vm.qmp('blockdev-change-medium',
id=self.device_name, filename=new_img,
diff --git a/tests/qemu-iotests/118.out b/tests/qemu-iotests/118.out
index bf5bfd5aca..0a70391105 100644
--- a/tests/qemu-iotests/118.out
+++ b/tests/qemu-iotests/118.out
@@ -1,5 +1,5 @@
-.......................................................................................................................................................................
+...........................................................................................................................................................
----------------------------------------------------------------------
-Ran 167 tests
+Ran 155 tests
OK
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PULL 53/53] iotests/178: Pass value to invalid option
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (51 preceding siblings ...)
2021-01-26 14:20 ` [PULL 52/53] iotests/118: Drop 'change' test Max Reitz
@ 2021-01-26 14:20 ` Max Reitz
2021-01-27 17:40 ` [PULL 00/53] Block patches Peter Maydell
53 siblings, 0 replies; 59+ messages in thread
From: Max Reitz @ 2021-01-26 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
ccd3b3b8112 has deprecated short-hand boolean options (i.e., options
with values). All options without values are interpreted as boolean
options, so this includes the invalid option "snapshot.foo" used in
iotest 178.
So after ccd3b3b8112, 178 fails with:
+qemu-img: warning: short-form boolean option 'snapshot.foo' deprecated
+Please use snapshot.foo=on instead
Suppress that deprecation warning by passing some value to it (it does
not matter which, because the option is invalid anyway).
Fixes: ccd3b3b8112b670fdccf8a392b8419b173ffccb4
("qemu-option: warn for short-form boolean options")
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210126123834.115915-1-mreitz@redhat.com>
---
tests/qemu-iotests/178 | 2 +-
tests/qemu-iotests/178.out.qcow2 | 2 +-
tests/qemu-iotests/178.out.raw | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/tests/qemu-iotests/178 b/tests/qemu-iotests/178
index 3b1a7adce4..8df241ead8 100755
--- a/tests/qemu-iotests/178
+++ b/tests/qemu-iotests/178
@@ -57,7 +57,7 @@ $QEMU_IMG measure --image-opts # missing filename
$QEMU_IMG measure -f qcow2 # missing filename
$QEMU_IMG measure -l snap1 # missing filename
$QEMU_IMG measure -o , # invalid option list
-$QEMU_IMG measure -l snapshot.foo # invalid snapshot option
+$QEMU_IMG measure -l snapshot.foo=bar # invalid snapshot option
$QEMU_IMG measure --output foo # invalid output format
$QEMU_IMG measure --size -1 # invalid image size
$QEMU_IMG measure -O foo "$TEST_IMG" # unknown image file format
diff --git a/tests/qemu-iotests/178.out.qcow2 b/tests/qemu-iotests/178.out.qcow2
index c7997760fd..fe193fd5f4 100644
--- a/tests/qemu-iotests/178.out.qcow2
+++ b/tests/qemu-iotests/178.out.qcow2
@@ -11,7 +11,7 @@ qemu-img: --image-opts, -f, and -l require a filename argument.
qemu-img: --image-opts, -f, and -l require a filename argument.
qemu-img: Invalid option list: ,
qemu-img: Invalid parameter 'snapshot.foo'
-qemu-img: Failed in parsing snapshot param 'snapshot.foo'
+qemu-img: Failed in parsing snapshot param 'snapshot.foo=bar'
qemu-img: --output must be used with human or json as argument.
qemu-img: Invalid image size specified. Must be between 0 and 9223372036854775807.
qemu-img: Unknown file format 'foo'
diff --git a/tests/qemu-iotests/178.out.raw b/tests/qemu-iotests/178.out.raw
index 20e17da115..445e460fad 100644
--- a/tests/qemu-iotests/178.out.raw
+++ b/tests/qemu-iotests/178.out.raw
@@ -11,7 +11,7 @@ qemu-img: --image-opts, -f, and -l require a filename argument.
qemu-img: --image-opts, -f, and -l require a filename argument.
qemu-img: Invalid option list: ,
qemu-img: Invalid parameter 'snapshot.foo'
-qemu-img: Failed in parsing snapshot param 'snapshot.foo'
+qemu-img: Failed in parsing snapshot param 'snapshot.foo=bar'
qemu-img: --output must be used with human or json as argument.
qemu-img: Invalid image size specified. Must be between 0 and 9223372036854775807.
qemu-img: Unknown file format 'foo'
--
2.29.2
^ permalink raw reply related [flat|nested] 59+ messages in thread* Re: [PULL 00/53] Block patches
2021-01-26 14:19 [PULL 00/53] Block patches Max Reitz
` (52 preceding siblings ...)
2021-01-26 14:20 ` [PULL 53/53] iotests/178: Pass value to invalid option Max Reitz
@ 2021-01-27 17:40 ` Peter Maydell
53 siblings, 0 replies; 59+ messages in thread
From: Peter Maydell @ 2021-01-27 17:40 UTC (permalink / raw)
To: Max Reitz; +Cc: Kevin Wolf, QEMU Developers, Qemu-block
On Tue, 26 Jan 2021 at 14:20, Max Reitz <mreitz@redhat.com> wrote:
>
> The following changes since commit 31ee895047bdcf7387e3570cbd2a473c6f744b08:
>
> Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging (2021-01-25 15:56:13 +0000)
>
> are available in the Git repository at:
>
> https://github.com/XanClic/qemu.git tags/pull-block-2021-01-26
>
> for you to fetch changes up to bb24cdc5efee580e81f71c5ff0fd980f2cc179d0:
>
> iotests/178: Pass value to invalid option (2021-01-26 14:36:37 +0100)
>
> ----------------------------------------------------------------
> Block patches:
> - Make backup block jobs use asynchronous requests with the block-copy
> module
> - Use COR filter node for stream block jobs
> - Make coroutine-sigaltstack’s qemu_coroutine_new() function thread-safe
> - Report error string when file locking fails with an unexpected errno
> - iotest fixes, additions, and some refactoring
>
Applied, thanks.
Please update the changelog at https://wiki.qemu.org/ChangeLog/6.0
for any user-visible changes.
-- PMM
^ permalink raw reply [flat|nested] 59+ messages in thread