From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
To: qemu-block@nongnu.org
Cc: qemu-devel@nongnu.org, eblake@redhat.com, armbru@redhat.com,
crosa@redhat.com, ehabkost@redhat.com, berrange@redhat.com,
pbonzini@redhat.com, vsementsov@virtuozzo.com, jsnow@redhat.com,
hreitz@redhat.com, kwolf@redhat.com,
Max Reitz <mreitz@redhat.com>
Subject: [PATCH v8 09/34] block/backup: move cluster size calculation to block-copy
Date: Tue, 24 Aug 2021 11:38:31 +0300 [thread overview]
Message-ID: <20210824083856.17408-10-vsementsov@virtuozzo.com> (raw)
In-Reply-To: <20210824083856.17408-1-vsementsov@virtuozzo.com>
The main consumer of cluster-size is block-copy. Let's calculate it
here instead of passing through backup-top.
We are going to publish copy-before-write filter soon, so it will be
created through options. But we don't want for now to make explicit
option for cluster-size, let's continue to calculate it automatically.
So, now is the time to get rid of cluster_size argument for
bdrv_cbw_append().
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
---
block/copy-before-write.h | 1 -
include/block/block-copy.h | 5 +--
block/backup.c | 62 ++++++--------------------------------
block/block-copy.c | 51 ++++++++++++++++++++++++++++++-
block/copy-before-write.c | 10 +++---
5 files changed, 66 insertions(+), 63 deletions(-)
diff --git a/block/copy-before-write.h b/block/copy-before-write.h
index 538aab8bdb..b386fd8f01 100644
--- a/block/copy-before-write.h
+++ b/block/copy-before-write.h
@@ -32,7 +32,6 @@
BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
BlockDriverState *target,
const char *filter_node_name,
- uint64_t cluster_size,
bool compress,
BlockCopyState **bcs,
Error **errp);
diff --git a/include/block/block-copy.h b/include/block/block-copy.h
index dca6c4ce36..b8a2d63545 100644
--- a/include/block/block-copy.h
+++ b/include/block/block-copy.h
@@ -25,8 +25,8 @@ typedef struct BlockCopyState BlockCopyState;
typedef struct BlockCopyCallState BlockCopyCallState;
BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
- int64_t cluster_size, bool use_copy_range,
- bool compress, Error **errp);
+ bool use_copy_range, bool compress,
+ Error **errp);
/* Function should be called prior any actual copy request */
void block_copy_set_copy_opts(BlockCopyState *s, bool use_copy_range,
@@ -91,6 +91,7 @@ void block_copy_kick(BlockCopyCallState *call_state);
void block_copy_call_cancel(BlockCopyCallState *call_state);
BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s);
+int64_t block_copy_cluster_size(BlockCopyState *s);
void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip);
#endif /* BLOCK_COPY_H */
diff --git a/block/backup.c b/block/backup.c
index b31fd99ab3..83516297cb 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -29,8 +29,6 @@
#include "block/copy-before-write.h"
-#define BACKUP_CLUSTER_SIZE_DEFAULT (1 << 16)
-
typedef struct BackupBlockJob {
BlockJob common;
BlockDriverState *cbw;
@@ -354,43 +352,6 @@ static const BlockJobDriver backup_job_driver = {
.set_speed = backup_set_speed,
};
-static int64_t backup_calculate_cluster_size(BlockDriverState *target,
- Error **errp)
-{
- int ret;
- BlockDriverInfo bdi;
- bool target_does_cow = bdrv_backing_chain_next(target);
-
- /*
- * If there is no backing file on the target, we cannot rely on COW if our
- * backup cluster size is smaller than the target cluster size. Even for
- * targets with a backing file, try to avoid COW if possible.
- */
- ret = bdrv_get_info(target, &bdi);
- if (ret == -ENOTSUP && !target_does_cow) {
- /* Cluster size is not defined */
- warn_report("The target block device doesn't provide "
- "information about the block size and it doesn't have a "
- "backing file. The default block size of %u bytes is "
- "used. If the actual block size of the target exceeds "
- "this default, the backup may be unusable",
- BACKUP_CLUSTER_SIZE_DEFAULT);
- return BACKUP_CLUSTER_SIZE_DEFAULT;
- } else if (ret < 0 && !target_does_cow) {
- error_setg_errno(errp, -ret,
- "Couldn't determine the cluster size of the target image, "
- "which has no backing file");
- error_append_hint(errp,
- "Aborting, since this may create an unusable destination image\n");
- return ret;
- } else if (ret < 0 && target_does_cow) {
- /* Not fatal; just trudge on ahead. */
- return BACKUP_CLUSTER_SIZE_DEFAULT;
- }
-
- return MAX(BACKUP_CLUSTER_SIZE_DEFAULT, bdi.cluster_size);
-}
-
BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
BlockDriverState *target, int64_t speed,
MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap,
@@ -448,11 +409,6 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
return NULL;
}
- cluster_size = backup_calculate_cluster_size(target, errp);
- if (cluster_size < 0) {
- goto error;
- }
-
if (perf->max_workers < 1) {
error_setg(errp, "max-workers must be greater than zero");
return NULL;
@@ -464,13 +420,6 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
return NULL;
}
- if (perf->max_chunk && perf->max_chunk < cluster_size) {
- error_setg(errp, "Required max-chunk (%" PRIi64 ") is less than backup "
- "cluster size (%" PRIi64 ")", perf->max_chunk, cluster_size);
- return NULL;
- }
-
-
if (sync_bitmap) {
/* If we need to write to this bitmap, check that we can: */
if (bitmap_mode != BITMAP_SYNC_MODE_NEVER &&
@@ -503,12 +452,19 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
goto error;
}
- cbw = bdrv_cbw_append(bs, target, filter_node_name,
- cluster_size, false, &bcs, errp);
+ cbw = bdrv_cbw_append(bs, target, filter_node_name, false, &bcs, errp);
if (!cbw) {
goto error;
}
+ cluster_size = block_copy_cluster_size(bcs);
+
+ if (perf->max_chunk && perf->max_chunk < cluster_size) {
+ error_setg(errp, "Required max-chunk (%" PRIi64 ") is less than backup "
+ "cluster size (%" PRIi64 ")", perf->max_chunk, cluster_size);
+ goto error;
+ }
+
/* job->len is fixed, so we can't allow resize */
job = block_job_create(job_id, &backup_job_driver, txn, cbw,
0, BLK_PERM_ALL,
diff --git a/block/block-copy.c b/block/block-copy.c
index e83870ff81..b0e0a38330 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -27,6 +27,7 @@
#define BLOCK_COPY_MAX_MEM (128 * MiB)
#define BLOCK_COPY_MAX_WORKERS 64
#define BLOCK_COPY_SLICE_TIME 100000000ULL /* ns */
+#define BLOCK_COPY_CLUSTER_SIZE_DEFAULT (1 << 16)
typedef enum {
COPY_READ_WRITE_CLUSTER,
@@ -342,14 +343,57 @@ void block_copy_set_copy_opts(BlockCopyState *s, bool use_copy_range,
}
}
+static int64_t block_copy_calculate_cluster_size(BlockDriverState *target,
+ Error **errp)
+{
+ int ret;
+ BlockDriverInfo bdi;
+ bool target_does_cow = bdrv_backing_chain_next(target);
+
+ /*
+ * If there is no backing file on the target, we cannot rely on COW if our
+ * backup cluster size is smaller than the target cluster size. Even for
+ * targets with a backing file, try to avoid COW if possible.
+ */
+ ret = bdrv_get_info(target, &bdi);
+ if (ret == -ENOTSUP && !target_does_cow) {
+ /* Cluster size is not defined */
+ warn_report("The target block device doesn't provide "
+ "information about the block size and it doesn't have a "
+ "backing file. The default block size of %u bytes is "
+ "used. If the actual block size of the target exceeds "
+ "this default, the backup may be unusable",
+ BLOCK_COPY_CLUSTER_SIZE_DEFAULT);
+ return BLOCK_COPY_CLUSTER_SIZE_DEFAULT;
+ } else if (ret < 0 && !target_does_cow) {
+ error_setg_errno(errp, -ret,
+ "Couldn't determine the cluster size of the target image, "
+ "which has no backing file");
+ error_append_hint(errp,
+ "Aborting, since this may create an unusable destination image\n");
+ return ret;
+ } else if (ret < 0 && target_does_cow) {
+ /* Not fatal; just trudge on ahead. */
+ return BLOCK_COPY_CLUSTER_SIZE_DEFAULT;
+ }
+
+ return MAX(BLOCK_COPY_CLUSTER_SIZE_DEFAULT, bdi.cluster_size);
+}
+
BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
- int64_t cluster_size, bool use_copy_range,
+ bool use_copy_range,
bool compress, Error **errp)
{
BlockCopyState *s;
+ int64_t cluster_size;
BdrvDirtyBitmap *copy_bitmap;
bool is_fleecing;
+ cluster_size = block_copy_calculate_cluster_size(target->bs, errp);
+ if (cluster_size < 0) {
+ return NULL;
+ }
+
copy_bitmap = bdrv_create_dirty_bitmap(source->bs, cluster_size, NULL,
errp);
if (!copy_bitmap) {
@@ -960,6 +1004,11 @@ BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s)
return s->copy_bitmap;
}
+int64_t block_copy_cluster_size(BlockCopyState *s)
+{
+ return s->cluster_size;
+}
+
void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip)
{
qatomic_set(&s->skip_unallocated, skip);
diff --git a/block/copy-before-write.c b/block/copy-before-write.c
index 235251a640..a7996d54db 100644
--- a/block/copy-before-write.c
+++ b/block/copy-before-write.c
@@ -37,7 +37,6 @@
typedef struct BDRVCopyBeforeWriteState {
BlockCopyState *bcs;
BdrvChild *target;
- int64_t cluster_size;
} BDRVCopyBeforeWriteState;
static coroutine_fn int cbw_co_preadv(
@@ -52,13 +51,14 @@ static coroutine_fn int cbw_do_copy_before_write(BlockDriverState *bs,
{
BDRVCopyBeforeWriteState *s = bs->opaque;
uint64_t off, end;
+ int64_t cluster_size = block_copy_cluster_size(s->bcs);
if (flags & BDRV_REQ_WRITE_UNCHANGED) {
return 0;
}
- off = QEMU_ALIGN_DOWN(offset, s->cluster_size);
- end = QEMU_ALIGN_UP(offset + bytes, s->cluster_size);
+ off = QEMU_ALIGN_DOWN(offset, cluster_size);
+ end = QEMU_ALIGN_UP(offset + bytes, cluster_size);
return block_copy(s->bcs, off, end - off, true);
}
@@ -169,7 +169,6 @@ BlockDriver bdrv_cbw_filter = {
BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
BlockDriverState *target,
const char *filter_node_name,
- uint64_t cluster_size,
bool compress,
BlockCopyState **bcs,
Error **errp)
@@ -214,9 +213,8 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
}
appended = true;
- state->cluster_size = cluster_size;
state->bcs = block_copy_state_new(top->backing, state->target,
- cluster_size, false, compress, errp);
+ false, compress, errp);
if (!state->bcs) {
error_prepend(errp, "Cannot create block-copy-state: ");
goto fail;
--
2.29.2
next prev parent reply other threads:[~2021-08-24 8:49 UTC|newest]
Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-24 8:38 [PATCH v8 00/34] block: publish backup-top filter Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 01/34] block: introduce bdrv_replace_child_bs() Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 02/34] block: introduce blk_replace_bs Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 03/34] qdev-properties: PropertyInfo: add realized_set_allowed field Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 04/34] qdev: allow setting drive property for realized device Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 05/34] block: rename backup-top to copy-before-write Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 06/34] block-copy: move detecting fleecing scheme to block-copy Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 07/34] block/block-copy: introduce block_copy_set_copy_opts() Vladimir Sementsov-Ogievskiy
2021-08-24 13:32 ` Hanna Reitz
2021-08-24 8:38 ` [PATCH v8 08/34] block/backup: set copy_range and compress after filter insertion Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` Vladimir Sementsov-Ogievskiy [this message]
2021-09-01 11:57 ` [PATCH v8 09/34] block/backup: move cluster size calculation to block-copy Hanna Reitz
2021-09-01 12:04 ` Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 10/34] block/copy-before-write: relax permission requirements when no parents Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 11/34] block/copy-before-write: drop extra bdrv_unref on failure path Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 12/34] block/copy-before-write: use file child instead of backing Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 13/34] block/copy-before-write: bdrv_cbw_append(): replace child at last Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 14/34] block/copy-before-write: introduce cbw_init() Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 15/34] block/copy-before-write: cbw_init(): rename variables Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 16/34] block/copy-before-write: cbw_init(): use file child after attaching Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 17/34] block/copy-before-write: bdrv_cbw_append(): drop unused compress arg Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 18/34] block/copy-before-write: cbw_init(): use options Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 19/34] block/copy-before-write: initialize block-copy bitmap Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 20/34] block/block-copy: make setting progress optional Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 21/34] block/copy-before-write: make public block driver Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 22/34] qapi: publish copy-before-write filter Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 23/34] python/qemu/machine.py: refactor _qemu_args() Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 24/34] python/qemu/machine: QEMUMachine: improve qmp() method Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 25/34] python:QEMUMachine: template typing for self returning methods Vladimir Sementsov-Ogievskiy
2021-08-24 13:56 ` Hanna Reitz
2021-08-24 8:38 ` [PATCH v8 26/34] iotests/222: fix pylint and mypy complains Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 27/34] iotests/222: constantly use single quotes for strings Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 28/34] iotests: move 222 to tests/image-fleecing Vladimir Sementsov-Ogievskiy
2021-09-01 12:37 ` Hanna Reitz
2021-09-01 12:47 ` Vladimir Sementsov-Ogievskiy
2021-09-01 12:54 ` Hanna Reitz
2021-09-01 13:00 ` Daniel P. Berrangé
2021-08-24 8:38 ` [PATCH v8 29/34] iotests.py: hmp_qemu_io: support qdev Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 30/34] iotests/image-fleecing: proper source device Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 31/34] iotests/image-fleecing: rename tgt_node Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 32/34] iotests/image-fleecing: prepare for adding new test-case Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 33/34] iotests/image-fleecing: add test-case for copy-before-write filter Vladimir Sementsov-Ogievskiy
2021-08-24 8:38 ` [PATCH v8 34/34] block/block-copy: block_copy_state_new(): drop extra arguments Vladimir Sementsov-Ogievskiy
2021-08-24 14:05 ` Hanna Reitz
2021-08-24 15:51 ` [PATCH v8 00/34] block: publish backup-top filter Hanna Reitz
2021-08-24 16:24 ` Vladimir Sementsov-Ogievskiy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210824083856.17408-10-vsementsov@virtuozzo.com \
--to=vsementsov@virtuozzo.com \
--cc=armbru@redhat.com \
--cc=berrange@redhat.com \
--cc=crosa@redhat.com \
--cc=eblake@redhat.com \
--cc=ehabkost@redhat.com \
--cc=hreitz@redhat.com \
--cc=jsnow@redhat.com \
--cc=kwolf@redhat.com \
--cc=mreitz@redhat.com \
--cc=pbonzini@redhat.com \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).