qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: Stefan Hajnoczi <stefanha@redhat.com>
To: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Cc: qemu-block@nongnu.org, qemu-devel@nongnu.org, kwolf@redhat.com,
	mreitz@redhat.com, jsnow@redhat.com, famz@redhat.com,
	den@openvz.org, pbonzini@redhat.com, jcody@redhat.com
Subject: Re: [Qemu-devel] [PATCH 00/21] new backup architecture
Date: Tue, 31 Jan 2017 10:20:35 +0000	[thread overview]
Message-ID: <20170131102035.GA18804@stefanha-x1.localdomain> (raw)
In-Reply-To: <1482503344-6424-1-git-send-email-vsementsov@virtuozzo.com>

[-- Attachment #1: Type: text/plain, Size: 1393 bytes --]

On Fri, Dec 23, 2016 at 05:28:43PM +0300, Vladimir Sementsov-Ogievskiy wrote:
> This is a new architecture for backup. It solves some current problems:
> 1. intersecting requests: for now at request start we wait for all intersecting requests, which means that
>     a. we may wait even for unrelated to our request clusters
>     b. not full async: if we are going to copy clusters 1,2,3,4, when 2 and 4 are in flight, why should we wait for 2 and 4 to be fully copied? Why not to start 1 and 3 in parallel with 2 and 4?
> 
> 2. notifier request is internally synchronous: if notifier starts copying clusters 1,2,3,4, they will be copied one by one in synchronous loop.
> 
> 3. notifier wait full copying of corresponding clusters (when actually it may wait only for _read_ operations to be finished)

Please include benchmark results since this is a performance
optimization.  I think this new level of complexity is worth it because
it should be possible to achieve significantly higher throughput, but
data is still necessary.

The cover letter mentions spawning 24 coroutines.  Did you compare the
memory footprint against the old backup architecture?  Sometimes users
complain when they notice QEMU using significantly more memory than in
previous versions.  If there's a good justification or a way to minimize
the impact then it's fine, but please check.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

      parent reply	other threads:[~2017-01-31 12:44 UTC|newest]

Thread overview: 59+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-12-23 14:28 [Qemu-devel] [PATCH 00/21] new backup architecture Vladimir Sementsov-Ogievskiy
2016-12-23 14:28 ` [Qemu-devel] [PATCH 01/21] backup: move from done_bitmap to copy_bitmap Vladimir Sementsov-Ogievskiy
2017-01-23  5:34   ` Jeff Cody
2017-01-23 12:20   ` Vladimir Sementsov-Ogievskiy
2017-01-31 10:25   ` Stefan Hajnoczi
2016-12-23 14:28 ` [Qemu-devel] [PATCH 02/21] backup: init copy_bitmap from sync_bitmap for incremental Vladimir Sementsov-Ogievskiy
2017-01-24  7:09   ` Fam Zheng
2017-01-24  9:00     ` Vladimir Sementsov-Ogievskiy
2017-01-24  9:46       ` Fam Zheng
2017-01-24 10:16         ` Vladimir Sementsov-Ogievskiy
2017-01-31 10:36   ` Stefan Hajnoczi
2016-12-23 14:28 ` [Qemu-devel] [PATCH 03/21] backup: improve non-dirty bits progress processing Vladimir Sementsov-Ogievskiy
2017-01-24  7:17   ` Fam Zheng
2017-01-24  9:12     ` Vladimir Sementsov-Ogievskiy
2017-01-31 10:56       ` Stefan Hajnoczi
2016-12-23 14:28 ` [Qemu-devel] [PATCH 04/21] backup: use copy_bitmap in incremental backup Vladimir Sementsov-Ogievskiy
2017-01-31 11:01   ` Stefan Hajnoczi
2016-12-23 14:28 ` [Qemu-devel] [PATCH 05/21] hbitmap: improve dirty iter Vladimir Sementsov-Ogievskiy
2017-01-31 11:20   ` Stefan Hajnoczi
2017-01-31 11:29   ` Stefan Hajnoczi
2016-12-23 14:28 ` [Qemu-devel] [PATCH 06/21] backup: rewrite top mode cluster skipping Vladimir Sementsov-Ogievskiy
2017-01-31 13:31   ` Stefan Hajnoczi
2016-12-23 14:28 ` [Qemu-devel] [PATCH 07/21] backup: refactor: merge top/full/incremental backup code Vladimir Sementsov-Ogievskiy
2017-01-31 14:26   ` Stefan Hajnoczi
2016-12-23 14:28 ` [Qemu-devel] [PATCH 08/21] backup: skip unallocated clusters for full mode Vladimir Sementsov-Ogievskiy
2017-01-24  7:59   ` Fam Zheng
2017-01-24  9:18     ` Vladimir Sementsov-Ogievskiy
2017-01-24  9:36       ` Fam Zheng
2017-01-24 10:13         ` Vladimir Sementsov-Ogievskiy
2017-01-24 11:12           ` Fam Zheng
2017-01-31 14:33   ` Stefan Hajnoczi
2017-01-31 14:38   ` Stefan Hajnoczi
2016-12-23 14:28 ` [Qemu-devel] [PATCH 09/21] backup: separate copy function Vladimir Sementsov-Ogievskiy
2017-01-31 14:40   ` Stefan Hajnoczi
2016-12-23 14:28 ` [Qemu-devel] [PATCH 10/21] backup: refactor backup_copy_cluster() Vladimir Sementsov-Ogievskiy
2017-01-31 14:57   ` Stefan Hajnoczi
2016-12-23 14:28 ` [Qemu-devel] [PATCH 11/21] backup: move r/w error handling code to r/w functions Vladimir Sementsov-Ogievskiy
2017-01-31 14:57   ` Stefan Hajnoczi
2016-12-23 14:28 ` [Qemu-devel] [PATCH 12/21] iotests: add supported_cache_modes to main function Vladimir Sementsov-Ogievskiy
2017-01-31 14:58   ` Stefan Hajnoczi
2016-12-23 14:28 ` [Qemu-devel] [PATCH 13/21] coroutine: add qemu_coroutine_add_next Vladimir Sementsov-Ogievskiy
2017-01-31 15:03   ` Stefan Hajnoczi
2016-12-23 14:28 ` [Qemu-devel] [PATCH 14/21] block: add trace point on bdrv_close_all Vladimir Sementsov-Ogievskiy
2017-01-31 15:03   ` Stefan Hajnoczi
2016-12-23 14:28 ` [Qemu-devel] [PATCH 15/21] bitmap: add bitmap_count_between() function Vladimir Sementsov-Ogievskiy
2017-01-31 15:15   ` Stefan Hajnoczi
2016-12-23 14:28 ` [Qemu-devel] [PATCH 16/21] hbitmap: add hbitmap_count_between() function Vladimir Sementsov-Ogievskiy
2017-01-31 15:56   ` Stefan Hajnoczi
2016-12-23 14:29 ` [Qemu-devel] [PATCH 17/21] backup: make all reads not serializing Vladimir Sementsov-Ogievskiy
2017-01-31 16:30   ` Stefan Hajnoczi
2016-12-23 14:29 ` [Qemu-devel] [PATCH 18/21] backup: new async architecture Vladimir Sementsov-Ogievskiy
2017-01-31 16:46   ` Stefan Hajnoczi
2017-02-01 16:13   ` [Qemu-devel] [Qemu-block] " Stefan Hajnoczi
2016-12-23 14:29 ` [Qemu-devel] [PATCH 20/21] backup: move bitmap handling from backup_do_cow to get_work Vladimir Sementsov-Ogievskiy
2016-12-23 14:29 ` [Qemu-devel] [PATCH 21/21] backup: refactor: remove backup_do_cow() Vladimir Sementsov-Ogievskiy
2017-01-09 11:04 ` [Qemu-devel] [PATCH 00/21] new backup architecture Stefan Hajnoczi
2017-01-10  6:05   ` Jeff Cody
2017-01-10 18:48     ` John Snow
2017-01-31 10:20 ` Stefan Hajnoczi [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170131102035.GA18804@stefanha-x1.localdomain \
    --to=stefanha@redhat.com \
    --cc=den@openvz.org \
    --cc=famz@redhat.com \
    --cc=jcody@redhat.com \
    --cc=jsnow@redhat.com \
    --cc=kwolf@redhat.com \
    --cc=mreitz@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    --cc=vsementsov@virtuozzo.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).