From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60739) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YLYRh-00032y-Lk for qemu-devel@nongnu.org; Wed, 11 Feb 2015 09:39:22 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YLYRc-0004Nn-Lp for qemu-devel@nongnu.org; Wed, 11 Feb 2015 09:39:17 -0500 Received: from mx1.redhat.com ([209.132.183.28]:54412) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YLYRc-0004NU-FB for qemu-devel@nongnu.org; Wed, 11 Feb 2015 09:39:12 -0500 Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id t1BEdAwF031742 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Wed, 11 Feb 2015 09:39:11 -0500 Message-ID: <54DB698B.5080805@redhat.com> Date: Wed, 11 Feb 2015 09:39:07 -0500 From: Max Reitz MIME-Version: 1.0 References: <1423652937-13066-1-git-send-email-kwolf@redhat.com> In-Reply-To: <1423652937-13066-1-git-send-email-kwolf@redhat.com> Content-Type: text/plain; charset=iso-8859-15; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH] qemu-img convert: Fix copy length for the iteration List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Kevin Wolf , qemu-devel@nongnu.org Cc: stefanha@redhat.com On 2015-02-11 at 06:08, Kevin Wolf wrote: > qemu-img converts already checks the block status of the next sectors to > copy and skips them if it knows that they don't need to be copied (e.g. > because they are zero and the destination is zero-initialised). > > It doesn't, however, limit the number of sectors read to the number of > allocated sectors when copying data yet. This means that when using -B, > some data is needlessly copied from the backing file into the overlay. > For images without backing files, it means that more data is read than > is necessary, even though the zero buffer detection will still keep > those sectors sparse. > > This patch changes qemu-img to respect the number of allocated sectors > and limit its requests accordingly when processing data, which fixes -B > to expose the expected behaviour and is a performance optimisation for > other cases. > > Reported-by: Max Reitz > Signed-off-by: Kevin Wolf > --- > qemu-img.c | 1 + > tests/qemu-iotests/122 | 56 ++++++++++++++++++++++++++++++++++++++++++++++ > tests/qemu-iotests/122.out | 11 +++++++++ > tests/qemu-iotests/group | 1 + > 4 files changed, 69 insertions(+) > create mode 100755 tests/qemu-iotests/122 > create mode 100644 tests/qemu-iotests/122.out > > diff --git a/qemu-img.c b/qemu-img.c > index e148af8..b94afe2 100644 > --- a/qemu-img.c > +++ b/qemu-img.c > @@ -1809,6 +1809,7 @@ restart: > } > > n = MIN(nb_sectors, bufsectors); > + n = MIN(n, sector_num_next_status - sector_num); This needs to be done only if sector_num_next_status > sector_num, otherwise we'll run into an infinite loop. Apparently I accidentally broke -n for qemu-img convert, but once I get it working again, you can see that this command: $ qemu-img convert -n -O raw test.qcow2 nbd://localhost:10809 does not terminate (because out_baseimg is NULL and has_zero_init is false due to nbd not supporting that, which means that sector_num_next_status is never set to anything but 0). Other than that, this looks good to me. Max > /* round down request length to an aligned sector, but > * do not bother doing this on short requests. They happen > diff --git a/tests/qemu-iotests/122 b/tests/qemu-iotests/122 > new file mode 100755 > index 0000000..0106e8a > --- /dev/null > +++ b/tests/qemu-iotests/122 > @@ -0,0 +1,56 @@ > +#!/bin/bash > +# > +# Test image sparseness after qemu-img convert > +# > +# Copyright (C) 2015 Red Hat, Inc. > +# > +# This program is free software; you can redistribute it and/or modify > +# it under the terms of the GNU General Public License as published by > +# the Free Software Foundation; either version 2 of the License, or > +# (at your option) any later version. > +# > +# This program is distributed in the hope that it will be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > +# > +# You should have received a copy of the GNU General Public License > +# along with this program. If not, see . > +# > + > +# creator > +owner=kwolf@redhat.com > + > +seq="$(basename $0)" > +echo "QA output created by $seq" > + > +here="$PWD" > +tmp=/tmp/$$ > +status=1 # failure is the default! > + > +_cleanup() > +{ > + _cleanup_test_img > +} > +trap "_cleanup; exit \$status" 0 1 2 3 15 > + > +# get standard environment, filters and checks > +. ./common.rc > +. ./common.filter > + > +_supported_fmt qcow2 > +_supported_proto generic > +_supported_os Linux > + > + > +TEST_IMG=$TEST_IMG.base _make_test_img 64M > +_make_test_img -b $TEST_IMG.base > +$QEMU_IO -c "write -P 0x11 0 64M" $TEST_IMG.base 2>&1 | _filter_qemu_io | _filter_testdir > +$QEMU_IO -c "write -P 0x22 0 3M" $TEST_IMG 2>&1 | _filter_qemu_io | _filter_testdir > +$QEMU_IMG convert -O $IMGFMT -B $TEST_IMG.base $TEST_IMG $TEST_IMG.orig > +$QEMU_IMG map $TEST_IMG.orig | _filter_qemu_img_map > + > +# success, all done > +echo '*** done' > +rm -f $seq.full > +status=0 > diff --git a/tests/qemu-iotests/122.out b/tests/qemu-iotests/122.out > new file mode 100644 > index 0000000..6e4a760 > --- /dev/null > +++ b/tests/qemu-iotests/122.out > @@ -0,0 +1,11 @@ > +QA output created by 122 > +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864 > +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file='TEST_DIR/t.IMGFMT.base' > +wrote 67108864/67108864 bytes at offset 0 > +64 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +wrote 3145728/3145728 bytes at offset 0 > +3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +Offset Length File > +0 0x300000 TEST_DIR/t.IMGFMT.orig > +0x300000 0x3d00000 TEST_DIR/t.IMGFMT.base > +*** done > diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group > index 4b2b93b..28c78e5 100644 > --- a/tests/qemu-iotests/group > +++ b/tests/qemu-iotests/group > @@ -117,3 +117,4 @@ > 113 rw auto quick > 114 rw auto quick > 116 rw auto quick > +122 rw auto quick