qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: Eric Blake <eblake@redhat.com>
To: qemu-devel@nongnu.org
Cc: kwolf@redhat.com, jsnow@redhat.com, famz@redhat.com,
	qemu-block@nongnu.org, Max Reitz <mreitz@redhat.com>
Subject: [Qemu-devel] [PATCH v5 14/23] qemu-img: Speed up compare on pre-allocated larger file
Date: Tue,  3 Oct 2017 21:00:39 -0500	[thread overview]
Message-ID: <20171004020048.26379-15-eblake@redhat.com> (raw)
In-Reply-To: <20171004020048.26379-1-eblake@redhat.com>

Compare the following images with all-zero contents:
$ truncate --size 1M A
$ qemu-img create -f qcow2 -o preallocation=off B 1G
$ qemu-img create -f qcow2 -o preallocation=metadata C 1G

On my machine, the difference is noticeable for pre-patch speeds,
with more than an order of magnitude in difference caused by the
choice of preallocation in the qcow2 file:

$ time ./qemu-img compare -f raw -F qcow2 A B
Warning: Image size mismatch!
Images are identical.

real	0m0.014s
user	0m0.007s
sys	0m0.007s

$ time ./qemu-img compare -f raw -F qcow2 A C
Warning: Image size mismatch!
Images are identical.

real	0m0.341s
user	0m0.144s
sys	0m0.188s

Why? Because bdrv_is_allocated() returns false for image B but
true for image C, throwing away the fact that both images know
via lseek(SEEK_HOLE) that the entire image still reads as zero.
>From there, qemu-img ends up calling bdrv_pread() for every byte
of the tail, instead of quickly looking for the next allocation.
The solution: use block_status instead of is_allocated, giving:

$ time ./qemu-img compare -f raw -F qcow2 A C
Warning: Image size mismatch!
Images are identical.

real	0m0.014s
user	0m0.011s
sys	0m0.003s

which is on par with the speeds for no pre-allocation.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

---
v4-v5: no change
v3: new patch
---
 qemu-img.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/qemu-img.c b/qemu-img.c
index abd289c0b5..43e3038894 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -1481,11 +1481,11 @@ static int img_compare(int argc, char **argv)
         while (sector_num < progress_base) {
             int64_t count;

-            ret = bdrv_is_allocated_above(blk_bs(blk_over), NULL,
+            ret = bdrv_block_status_above(blk_bs(blk_over), NULL,
                                           sector_num * BDRV_SECTOR_SIZE,
                                           (progress_base - sector_num) *
                                           BDRV_SECTOR_SIZE,
-                                          &count);
+                                          &count, NULL);
             if (ret < 0) {
                 ret = 3;
                 error_report("Sector allocation test failed for %s",
@@ -1493,11 +1493,11 @@ static int img_compare(int argc, char **argv)
                 goto out;

             }
-            /* TODO relax this once bdrv_is_allocated_above does not enforce
+            /* TODO relax this once bdrv_block_status_above does not enforce
              * sector alignment */
             assert(QEMU_IS_ALIGNED(count, BDRV_SECTOR_SIZE));
             nb_sectors = count >> BDRV_SECTOR_BITS;
-            if (ret) {
+            if (ret & BDRV_BLOCK_ALLOCATED && !(ret & BDRV_BLOCK_ZERO)) {
                 nb_sectors = MIN(nb_sectors, IO_BUF_SIZE >> BDRV_SECTOR_BITS);
                 ret = check_empty_sectors(blk_over, sector_num, nb_sectors,
                                           filename_over, buf1, quiet);
-- 
2.13.6

  parent reply	other threads:[~2017-10-04  2:01 UTC|newest]

Thread overview: 39+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-10-04  2:00 [Qemu-devel] [PATCH v5 00/23] make bdrv_get_block_status byte-based Eric Blake
2017-10-04  2:00 ` [Qemu-devel] [PATCH v5 01/23] block: Allow NULL file for bdrv_get_block_status() Eric Blake
2017-10-10 13:59   ` Kevin Wolf
2017-10-10 14:43     ` Eric Blake
2017-10-10 19:00       ` Eric Blake
2017-10-10 19:24         ` John Snow
2017-10-11  8:42           ` Kevin Wolf
2017-10-11 17:42             ` John Snow
2017-10-04  2:00 ` [Qemu-devel] [PATCH v5 02/23] block: Add flag to avoid wasted work in bdrv_is_allocated() Eric Blake
2017-10-04  2:00 ` [Qemu-devel] [PATCH v5 03/23] block: Make bdrv_round_to_clusters() signature more useful Eric Blake
2017-10-04  2:00 ` [Qemu-devel] [PATCH v5 04/23] qcow2: Switch is_zero_sectors() to byte-based Eric Blake
2017-10-10 14:15   ` Kevin Wolf
2017-10-10 14:47     ` Eric Blake
2017-10-04  2:00 ` [Qemu-devel] [PATCH v5 05/23] block: Switch bdrv_make_zero() " Eric Blake
2017-10-04  2:00 ` [Qemu-devel] [PATCH v5 06/23] qemu-img: Switch get_block_status() " Eric Blake
2017-10-04  2:00 ` [Qemu-devel] [PATCH v5 07/23] block: Convert bdrv_get_block_status() to bytes Eric Blake
2017-10-10 14:46   ` Kevin Wolf
2017-10-10 15:38     ` Eric Blake
2017-10-04  2:00 ` [Qemu-devel] [PATCH v5 08/23] block: Switch bdrv_co_get_block_status() to byte-based Eric Blake
2017-10-04  2:00 ` [Qemu-devel] [PATCH v5 09/23] block: Switch BdrvCoGetBlockStatusData " Eric Blake
2017-10-09 20:07   ` [Qemu-devel] [Qemu-block] " Jeff Cody
2017-10-09 21:30     ` Eric Blake
2017-10-04  2:00 ` [Qemu-devel] [PATCH v5 10/23] block: Switch bdrv_common_block_status_above() " Eric Blake
2017-10-04  2:00 ` [Qemu-devel] [PATCH v5 11/23] block: Switch bdrv_co_get_block_status_above() " Eric Blake
2017-10-04  2:00 ` [Qemu-devel] [PATCH v5 12/23] block: Convert bdrv_get_block_status_above() to bytes Eric Blake
2017-10-04  2:00 ` [Qemu-devel] [PATCH v5 13/23] qemu-img: Simplify logic in img_compare() Eric Blake
2017-10-04  2:00 ` Eric Blake [this message]
2017-10-11 18:33   ` [Qemu-devel] [PATCH v5 14/23] qemu-img: Speed up compare on pre-allocated larger file Eric Blake
2017-10-04  2:00 ` [Qemu-devel] [PATCH v5 15/23] qemu-img: Add find_nonzero() Eric Blake
2017-10-04  2:00 ` [Qemu-devel] [PATCH v5 16/23] qemu-img: Drop redundant error message in compare Eric Blake
2017-10-04  2:00 ` [Qemu-devel] [PATCH v5 17/23] qemu-img: Change check_empty_sectors() to byte-based Eric Blake
2017-10-04  2:00 ` [Qemu-devel] [PATCH v5 18/23] qemu-img: Change compare_sectors() to be byte-based Eric Blake
2017-10-04  2:00 ` [Qemu-devel] [PATCH v5 19/23] qemu-img: Change img_rebase() " Eric Blake
2017-10-04  2:00 ` [Qemu-devel] [PATCH v5 20/23] qemu-img: Change img_compare() " Eric Blake
2017-10-04  2:00 ` [Qemu-devel] [PATCH v5 21/23] block: Align block status requests Eric Blake
2017-10-04  2:00 ` [Qemu-devel] [PATCH v5 22/23] block: Relax bdrv_aligned_preadv() assertion Eric Blake
2017-10-04  2:00 ` [Qemu-devel] [PATCH v5 23/23] qemu-io: Relax 'alloc' now that block-status doesn't assert Eric Blake
2017-10-10 12:58 ` [Qemu-devel] [PATCH v5 00/23] make bdrv_get_block_status byte-based Kevin Wolf
2017-10-10 14:48   ` Eric Blake

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20171004020048.26379-15-eblake@redhat.com \
    --to=eblake@redhat.com \
    --cc=famz@redhat.com \
    --cc=jsnow@redhat.com \
    --cc=kwolf@redhat.com \
    --cc=mreitz@redhat.com \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).