qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: Kevin Wolf <kwolf@redhat.com>
To: qemu-block@nongnu.org
Cc: kwolf@redhat.com, qemu-devel@nongnu.org
Subject: [Qemu-devel] [PULL 16/35] qemu-img: Speed up compare on pre-allocated larger file
Date: Thu, 26 Oct 2017 15:17:22 +0200	[thread overview]
Message-ID: <20171026131741.5059-17-kwolf@redhat.com> (raw)
In-Reply-To: <20171026131741.5059-1-kwolf@redhat.com>

From: Eric Blake <eblake@redhat.com>

Compare the following images with all-zero contents:
$ truncate --size 1M A
$ qemu-img create -f qcow2 -o preallocation=off B 1G
$ qemu-img create -f qcow2 -o preallocation=metadata C 1G

On my machine, the difference is noticeable for pre-patch speeds,
with more than an order of magnitude in difference caused by the
choice of preallocation in the qcow2 file:

$ time ./qemu-img compare -f raw -F qcow2 A B
Warning: Image size mismatch!
Images are identical.

real	0m0.014s
user	0m0.007s
sys	0m0.007s

$ time ./qemu-img compare -f raw -F qcow2 A C
Warning: Image size mismatch!
Images are identical.

real	0m0.341s
user	0m0.144s
sys	0m0.188s

Why? Because bdrv_is_allocated() returns false for image B but
true for image C, throwing away the fact that both images know
via lseek(SEEK_HOLE) that the entire image still reads as zero.
>From there, qemu-img ends up calling bdrv_pread() for every byte
of the tail, instead of quickly looking for the next allocation.
The solution: use block_status instead of is_allocated, giving:

$ time ./qemu-img compare -f raw -F qcow2 A C
Warning: Image size mismatch!
Images are identical.

real	0m0.014s
user	0m0.011s
sys	0m0.003s

which is on par with the speeds for no pre-allocation.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 qemu-img.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/qemu-img.c b/qemu-img.c
index cfa28d41d5..e4b84c4f56 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -1480,11 +1480,11 @@ static int img_compare(int argc, char **argv)
         while (sector_num < progress_base) {
             int64_t count;
 
-            ret = bdrv_is_allocated_above(blk_bs(blk_over), NULL,
+            ret = bdrv_block_status_above(blk_bs(blk_over), NULL,
                                           sector_num * BDRV_SECTOR_SIZE,
                                           (progress_base - sector_num) *
                                           BDRV_SECTOR_SIZE,
-                                          &count);
+                                          &count, NULL, NULL);
             if (ret < 0) {
                 ret = 3;
                 error_report("Sector allocation test failed for %s",
@@ -1492,11 +1492,11 @@ static int img_compare(int argc, char **argv)
                 goto out;
 
             }
-            /* TODO relax this once bdrv_is_allocated_above does not enforce
+            /* TODO relax this once bdrv_block_status_above does not enforce
              * sector alignment */
             assert(QEMU_IS_ALIGNED(count, BDRV_SECTOR_SIZE));
             nb_sectors = count >> BDRV_SECTOR_BITS;
-            if (ret) {
+            if (ret & BDRV_BLOCK_ALLOCATED && !(ret & BDRV_BLOCK_ZERO)) {
                 nb_sectors = MIN(nb_sectors, IO_BUF_SIZE >> BDRV_SECTOR_BITS);
                 ret = check_empty_sectors(blk_over, sector_num, nb_sectors,
                                           filename_over, buf1, quiet);
-- 
2.13.6

  parent reply	other threads:[~2017-10-26 13:18 UTC|newest]

Thread overview: 37+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-10-26 13:17 [Qemu-devel] [PULL 00/35] Block layer patches Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 01/35] block: don't add 'driver' to options when referring to backing via node name Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 02/35] qemu-iotests: Test backing_fmt with backing node reference Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 03/35] block: Allow NULL file for bdrv_get_block_status() Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 04/35] block: Add flag to avoid wasted work in bdrv_is_allocated() Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 05/35] block: Make bdrv_round_to_clusters() signature more useful Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 06/35] qcow2: Switch is_zero_sectors() to byte-based Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 07/35] block: Switch bdrv_make_zero() " Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 08/35] qemu-img: Switch get_block_status() " Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 09/35] block: Convert bdrv_get_block_status() to bytes Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 10/35] block: Switch bdrv_co_get_block_status() to byte-based Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 11/35] block: Switch BdrvCoGetBlockStatusData " Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 12/35] block: Switch bdrv_common_block_status_above() " Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 13/35] block: Switch bdrv_co_get_block_status_above() " Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 14/35] block: Convert bdrv_get_block_status_above() to bytes Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 15/35] qemu-img: Simplify logic in img_compare() Kevin Wolf
2017-10-26 13:17 ` Kevin Wolf [this message]
2017-10-26 13:17 ` [Qemu-devel] [PULL 17/35] qemu-img: Add find_nonzero() Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 18/35] qemu-img: Drop redundant error message in compare Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 19/35] qemu-img: Change check_empty_sectors() to byte-based Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 20/35] qemu-img: Change compare_sectors() to be byte-based Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 21/35] qemu-img: Change img_rebase() " Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 22/35] qemu-img: Change img_compare() " Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 23/35] block: Align block status requests Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 24/35] block: Reduce bdrv_aligned_preadv() rounding Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 25/35] qcow2: Reduce is_zero() rounding Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 26/35] qemu-io: Relax 'alloc' now that block-status doesn't assert Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 27/35] qemu-img.1: Image invalidation on qemu-img commit Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 28/35] qcow2: Use BDRV_SECTOR_BITS instead of its literal value Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 29/35] iotests: Add test for dataplane mirroring Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 30/35] iotests: Pull _filter_actual_image_size from 67/87 Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 31/35] iotests: Filter actual image size in 184 and 191 Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 32/35] qcow2: Emit errp when truncating the image tail Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 33/35] qcow2: Fix unaligned preallocated truncation Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 34/35] qcow2: Always execute preallocate() in a coroutine Kevin Wolf
2017-10-26 13:17 ` [Qemu-devel] [PULL 35/35] iotests: Add cluster_size=64k to 125 Kevin Wolf
2017-10-27 10:13 ` [Qemu-devel] [PULL 00/35] Block layer patches Peter Maydell

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20171026131741.5059-17-kwolf@redhat.com \
    --to=kwolf@redhat.com \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).