qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
To: qemu-block@nongnu.org
Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com,
	wencongyang2@huawei.com, xiechanglong.d@gmail.com,
	armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com,
	crosa@redhat.com, den@openvz.org, mreitz@redhat.com
Subject: [RFC 16/24] block/block-copy: add max_chunk and max_workers paramters
Date: Fri, 15 Nov 2019 17:14:36 +0300	[thread overview]
Message-ID: <20191115141444.24155-17-vsementsov@virtuozzo.com> (raw)
In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com>

They will be used for backup.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 include/block/block-copy.h | 5 +++++
 block/block-copy.c         | 9 +++++++--
 2 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/include/block/block-copy.h b/include/block/block-copy.h
index 407de7e520..8fc27156b3 100644
--- a/include/block/block-copy.h
+++ b/include/block/block-copy.h
@@ -47,6 +47,11 @@ int coroutine_fn block_copy(BlockCopyState *s, int64_t start, uint64_t bytes,
 /*
  * Run block-copy in a coroutine, return state pointer. If finished early
  * returns NULL (@cb is called anyway).
+ *
+ * @max_workers means maximum of parallel coroutines to execute sub-requests,
+ * must be > 0.
+ *
+ * @max_chunk means maximum length for one IO operation. Zero means unlimited.
  */
 BlockCopyCallState *block_copy_async(BlockCopyState *s,
                                      int64_t offset, int64_t bytes,
diff --git a/block/block-copy.c b/block/block-copy.c
index 0b41afd30d..f28f3b8b6c 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -32,6 +32,8 @@ typedef struct BlockCopyCallState {
     BlockCopyState *s;
     int64_t offset;
     int64_t bytes;
+    int max_workers;
+    int64_t max_chunk;
     BlockCopyAsyncCallbackFunc cb;
 
     /* State */
@@ -393,7 +395,7 @@ static BlockCopyTask *block_copy_task_create(BlockCopyState *s,
 
     assert(bdrv_dirty_bitmap_get(s->copy_bitmap, offset));
 
-    bytes = MIN(bytes, s->copy_size);
+    bytes = MIN(bytes, MIN_NON_ZERO(s->copy_size, call_state->max_chunk));
     next_zero = bdrv_dirty_bitmap_next_zero(s->copy_bitmap, offset, bytes);
     if (next_zero >= 0) {
         assert(next_zero > offset); /* offset is dirty */
@@ -578,7 +580,7 @@ block_copy_dirty_clusters(BlockCopyCallState *call_state)
         co_get_from_shres(s->mem, task->bytes);
 
         if (!aio && task->bytes != bytes) {
-            aio = aio_task_pool_new(BLOCK_COPY_MAX_WORKERS);
+            aio = aio_task_pool_new(call_state->max_workers);
         }
 
         offset += task->bytes;
@@ -655,6 +657,7 @@ int coroutine_fn block_copy(BlockCopyState *s, int64_t start, uint64_t bytes,
         .s = s,
         .offset = start,
         .bytes = bytes,
+        .max_workers = BLOCK_COPY_MAX_WORKERS,
     };
 
     int ret = block_copy_common(&call_state);
@@ -686,6 +689,8 @@ BlockCopyCallState *block_copy_async(BlockCopyState *s,
         .offset = offset,
         .bytes = bytes,
         .cb = cb,
+        .max_workers = max_workers ?: BLOCK_COPY_MAX_WORKERS,
+        .max_chunk = max_chunk,
     };
 
     qemu_coroutine_enter(co);
-- 
2.21.0



  parent reply	other threads:[~2019-11-15 14:21 UTC|newest]

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-11-15 14:14 [RFC 00/24] backup performance: block_status + async Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 01/24] block/block-copy: specialcase first copy_range request Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 02/24] block/block-copy: use block_status Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 03/24] block/block-copy: factor out block_copy_find_inflight_req Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 04/24] block/block-copy: refactor interfaces to use bytes instead of end Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 05/24] block/block-copy: rename start to offset in interfaces Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 06/24] block/block-copy: reduce intersecting request lock Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 07/24] block/block-copy: hide structure definitions Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 08/24] block/block-copy: rename in-flight requests to tasks Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 09/24] block/block-copy: alloc task on each iteration Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 10/24] block/block-copy: add state pointer to BlockCopyTask Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 11/24] block/block-copy: move task size initial calculation to _task_create Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 12/24] block/block-copy: move block_copy_task_create down Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 13/24] block/block-copy: use aio-task-pool API Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 14/24] block/block-copy: More explicit call_state Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 15/24] block/block-copy: implement block_copy_async Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` Vladimir Sementsov-Ogievskiy [this message]
2019-11-15 14:14 ` [RFC 17/24] block/block-copy: add ratelimit to block-copy Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 18/24] block/block-copy: add block_copy_cancel Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 19/24] blockjob: add set_speed to BlockJobDriver Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 20/24] job: call job_enter from job_user_pause Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 21/24] backup: move to block-copy Vladimir Sementsov-Ogievskiy
2019-11-15 17:58   ` Eric Blake
2019-11-16  9:28     ` Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 22/24] python: add simplebench.py Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 23/24] python: add qemu/bench_block_job.py Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 24/24] python: benchmark new backup architecture Vladimir Sementsov-Ogievskiy
2019-11-15 17:30 ` [RFC 00/24] backup performance: block_status + async no-reply
2019-11-15 17:33   ` Vladimir Sementsov-Ogievskiy

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20191115141444.24155-17-vsementsov@virtuozzo.com \
    --to=vsementsov@virtuozzo.com \
    --cc=armbru@redhat.com \
    --cc=crosa@redhat.com \
    --cc=den@openvz.org \
    --cc=ehabkost@redhat.com \
    --cc=jsnow@redhat.com \
    --cc=kwolf@redhat.com \
    --cc=mreitz@redhat.com \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    --cc=wencongyang2@huawei.com \
    --cc=xiechanglong.d@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).