* [Qemu-devel] [PATCH 0/3] qcow2: write_zeroes corruption fixes and tests
@ 2016-05-17 16:51 Kevin Wolf
2016-05-17 16:51 ` [Qemu-devel] [PATCH 1/3] qcow2: fix condition in is_zero_cluster Kevin Wolf
` (2 more replies)
0 siblings, 3 replies; 12+ messages in thread
From: Kevin Wolf @ 2016-05-17 16:51 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, mreitz, eblake, den, qemu-devel
These are the qcow2 write_zeroes fixes that need to be merged with the next
pull request as they can actually cause image corruption. The rest of Den's
patches are mostly cleanups, so they aren't quite as urgent.
Denis V. Lunev (1):
qcow2: fix condition in is_zero_cluster
Kevin Wolf (2):
qcow2: Fix write_zeroes with partially allocated backing file cluster
qemu-iotests: Some more write_zeroes tests
block/qcow2.c | 3 +-
tests/qemu-iotests/154 | 100 +++++++++++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/154.out | 64 +++++++++++++++++++++++++++++
tests/qemu-iotests/group | 1 +
4 files changed, 167 insertions(+), 1 deletion(-)
create mode 100755 tests/qemu-iotests/154
create mode 100644 tests/qemu-iotests/154.out
--
1.8.3.1
^ permalink raw reply [flat|nested] 12+ messages in thread* [Qemu-devel] [PATCH 1/3] qcow2: fix condition in is_zero_cluster 2016-05-17 16:51 [Qemu-devel] [PATCH 0/3] qcow2: write_zeroes corruption fixes and tests Kevin Wolf @ 2016-05-17 16:51 ` Kevin Wolf 2016-05-17 17:25 ` Eric Blake 2016-05-17 16:51 ` [Qemu-devel] [PATCH 2/3] qcow2: Fix write_zeroes with partially allocated backing file cluster Kevin Wolf 2016-05-17 16:51 ` [Qemu-devel] [PATCH 3/3] qemu-iotests: Some more write_zeroes tests Kevin Wolf 2 siblings, 1 reply; 12+ messages in thread From: Kevin Wolf @ 2016-05-17 16:51 UTC (permalink / raw) To: qemu-block; +Cc: kwolf, mreitz, eblake, den, qemu-devel From: "Denis V. Lunev" <den@openvz.org> We should check for (res & BDRV_BLOCK_ZERO) only. The situation when we will have !(res & BDRV_BLOCK_DATA) and will not have BDRV_BLOCK_ZERO is not possible for images with bdi.unallocated_blocks_are_zero == true. For those images where it's false, however, it can happen and we must not consider the data zeroed then or we would corrupt the image. Signed-off-by: Denis V. Lunev <den@openvz.org> CC: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/qcow2.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/qcow2.c b/block/qcow2.c index 62febfc..a6012dc 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2412,7 +2412,7 @@ static bool is_zero_cluster(BlockDriverState *bs, int64_t start) BlockDriverState *file; int64_t res = bdrv_get_block_status_above(bs, NULL, start, s->cluster_sectors, &nr, &file); - return res >= 0 && ((res & BDRV_BLOCK_ZERO) || !(res & BDRV_BLOCK_DATA)); + return res >= 0 && (res & BDRV_BLOCK_ZERO); } static bool is_zero_cluster_top_locked(BlockDriverState *bs, int64_t start) -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [Qemu-devel] [PATCH 1/3] qcow2: fix condition in is_zero_cluster 2016-05-17 16:51 ` [Qemu-devel] [PATCH 1/3] qcow2: fix condition in is_zero_cluster Kevin Wolf @ 2016-05-17 17:25 ` Eric Blake 0 siblings, 0 replies; 12+ messages in thread From: Eric Blake @ 2016-05-17 17:25 UTC (permalink / raw) To: Kevin Wolf, qemu-block; +Cc: mreitz, den, qemu-devel [-- Attachment #1: Type: text/plain, Size: 837 bytes --] On 05/17/2016 10:51 AM, Kevin Wolf wrote: > From: "Denis V. Lunev" <den@openvz.org> > > We should check for (res & BDRV_BLOCK_ZERO) only. The situation when we > will have !(res & BDRV_BLOCK_DATA) and will not have BDRV_BLOCK_ZERO is > not possible for images with bdi.unallocated_blocks_are_zero == true. > > For those images where it's false, however, it can happen and we must > not consider the data zeroed then or we would corrupt the image. > > Signed-off-by: Denis V. Lunev <den@openvz.org> > CC: Kevin Wolf <kwolf@redhat.com> > Signed-off-by: Kevin Wolf <kwolf@redhat.com> > --- > block/qcow2.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) Reviewed-by: Eric Blake <eblake@redhat.com> -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 604 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* [Qemu-devel] [PATCH 2/3] qcow2: Fix write_zeroes with partially allocated backing file cluster 2016-05-17 16:51 [Qemu-devel] [PATCH 0/3] qcow2: write_zeroes corruption fixes and tests Kevin Wolf 2016-05-17 16:51 ` [Qemu-devel] [PATCH 1/3] qcow2: fix condition in is_zero_cluster Kevin Wolf @ 2016-05-17 16:51 ` Kevin Wolf 2016-05-17 17:22 ` Eric Blake 2016-05-17 19:20 ` Denis V. Lunev 2016-05-17 16:51 ` [Qemu-devel] [PATCH 3/3] qemu-iotests: Some more write_zeroes tests Kevin Wolf 2 siblings, 2 replies; 12+ messages in thread From: Kevin Wolf @ 2016-05-17 16:51 UTC (permalink / raw) To: qemu-block; +Cc: kwolf, mreitz, eblake, den, qemu-devel In order to correctly check whether a given cluster is read as zero, we don't only need to check whether bdrv_get_block_status_above() sets BDRV_BLOCK_ZERO, but also if all sectors for the whole cluster have the same status. Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- block/qcow2.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/block/qcow2.c b/block/qcow2.c index a6012dc..c197ff3 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2412,7 +2412,7 @@ static bool is_zero_cluster(BlockDriverState *bs, int64_t start) BlockDriverState *file; int64_t res = bdrv_get_block_status_above(bs, NULL, start, s->cluster_sectors, &nr, &file); - return res >= 0 && (res & BDRV_BLOCK_ZERO); + return res >= 0 && (res & BDRV_BLOCK_ZERO) && nr == s->cluster_sectors; } static bool is_zero_cluster_top_locked(BlockDriverState *bs, int64_t start) @@ -2423,6 +2423,7 @@ static bool is_zero_cluster_top_locked(BlockDriverState *bs, int64_t start) int ret; ret = qcow2_get_cluster_offset(bs, start << BDRV_SECTOR_BITS, &nr, &off); + assert(nr == s->cluster_sectors); return ret == QCOW2_CLUSTER_UNALLOCATED || ret == QCOW2_CLUSTER_ZERO; } -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [Qemu-devel] [PATCH 2/3] qcow2: Fix write_zeroes with partially allocated backing file cluster 2016-05-17 16:51 ` [Qemu-devel] [PATCH 2/3] qcow2: Fix write_zeroes with partially allocated backing file cluster Kevin Wolf @ 2016-05-17 17:22 ` Eric Blake 2016-05-17 19:20 ` Denis V. Lunev 1 sibling, 0 replies; 12+ messages in thread From: Eric Blake @ 2016-05-17 17:22 UTC (permalink / raw) To: Kevin Wolf, qemu-block; +Cc: mreitz, den, qemu-devel [-- Attachment #1: Type: text/plain, Size: 578 bytes --] On 05/17/2016 10:51 AM, Kevin Wolf wrote: > In order to correctly check whether a given cluster is read as zero, we > don't only need to check whether bdrv_get_block_status_above() sets > BDRV_BLOCK_ZERO, but also if all sectors for the whole cluster have the > same status. > > Signed-off-by: Kevin Wolf <kwolf@redhat.com> > --- > block/qcow2.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > Reviewed-by: Eric Blake <eblake@redhat.com> -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 604 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Qemu-devel] [PATCH 2/3] qcow2: Fix write_zeroes with partially allocated backing file cluster 2016-05-17 16:51 ` [Qemu-devel] [PATCH 2/3] qcow2: Fix write_zeroes with partially allocated backing file cluster Kevin Wolf 2016-05-17 17:22 ` Eric Blake @ 2016-05-17 19:20 ` Denis V. Lunev 1 sibling, 0 replies; 12+ messages in thread From: Denis V. Lunev @ 2016-05-17 19:20 UTC (permalink / raw) To: Kevin Wolf, qemu-block; +Cc: mreitz, eblake, qemu-devel On 05/17/2016 07:51 PM, Kevin Wolf wrote: > In order to correctly check whether a given cluster is read as zero, we > don't only need to check whether bdrv_get_block_status_above() sets > BDRV_BLOCK_ZERO, but also if all sectors for the whole cluster have the > same status. > > Signed-off-by: Kevin Wolf <kwolf@redhat.com> > --- > block/qcow2.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/block/qcow2.c b/block/qcow2.c > index a6012dc..c197ff3 100644 > --- a/block/qcow2.c > +++ b/block/qcow2.c > @@ -2412,7 +2412,7 @@ static bool is_zero_cluster(BlockDriverState *bs, int64_t start) > BlockDriverState *file; > int64_t res = bdrv_get_block_status_above(bs, NULL, start, > s->cluster_sectors, &nr, &file); > - return res >= 0 && (res & BDRV_BLOCK_ZERO); > + return res >= 0 && (res & BDRV_BLOCK_ZERO) && nr == s->cluster_sectors; > } > > static bool is_zero_cluster_top_locked(BlockDriverState *bs, int64_t start) > @@ -2423,6 +2423,7 @@ static bool is_zero_cluster_top_locked(BlockDriverState *bs, int64_t start) > int ret; > > ret = qcow2_get_cluster_offset(bs, start << BDRV_SECTOR_BITS, &nr, &off); > + assert(nr == s->cluster_sectors); > return ret == QCOW2_CLUSTER_UNALLOCATED || ret == QCOW2_CLUSTER_ZERO; > } > Reviewed-by: Denis V. Lunev <den@openvz.org> I have had this kludge in the revision one and I have lost them in revision 3 :( ^ permalink raw reply [flat|nested] 12+ messages in thread
* [Qemu-devel] [PATCH 3/3] qemu-iotests: Some more write_zeroes tests 2016-05-17 16:51 [Qemu-devel] [PATCH 0/3] qcow2: write_zeroes corruption fixes and tests Kevin Wolf 2016-05-17 16:51 ` [Qemu-devel] [PATCH 1/3] qcow2: fix condition in is_zero_cluster Kevin Wolf 2016-05-17 16:51 ` [Qemu-devel] [PATCH 2/3] qcow2: Fix write_zeroes with partially allocated backing file cluster Kevin Wolf @ 2016-05-17 16:51 ` Kevin Wolf 2016-05-17 20:35 ` Eric Blake 2 siblings, 1 reply; 12+ messages in thread From: Kevin Wolf @ 2016-05-17 16:51 UTC (permalink / raw) To: qemu-block; +Cc: kwolf, mreitz, eblake, den, qemu-devel This covers some more write_zeroes cases which are relevant for the recent qcow2 optimisations that check the allocation status of the backing file for partial cluster write_zeroes requests. Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- tests/qemu-iotests/154 | 100 +++++++++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/154.out | 64 +++++++++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 165 insertions(+) create mode 100755 tests/qemu-iotests/154 create mode 100644 tests/qemu-iotests/154.out diff --git a/tests/qemu-iotests/154 b/tests/qemu-iotests/154 new file mode 100755 index 0000000..e9b5a8f --- /dev/null +++ b/tests/qemu-iotests/154 @@ -0,0 +1,100 @@ +#!/bin/bash +# +# qcow2 specific bdrv_write_zeroes tests with backing files (complements 034) +# +# Copyright (C) 2016 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +# creator +owner=kwolf@redhat.com + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fmt qcow2 +_supported_proto file +_supported_os Linux + +CLUSTER_SIZE=4k +size=128M + +echo +echo == backing file contains zeros == + +# Make sure that the whole cluster is allocated even for partial write_zeroes +# when the backing file contains zeros + +CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size +_make_test_img -b "$TEST_IMG.base" + +$QEMU_IO -c "write -z 0k 2k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "write -z 10k 2k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "write -z 17k 2k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map + +echo +echo == backing file contains non-zero data before write_zeros == + +CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size +_make_test_img -b "$TEST_IMG.base" + +$QEMU_IO -c "write -P 0x11 32k 1k" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z 34k 1k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0x11 32k 1k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 33k 3k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map + +echo +echo == backing file contains non-zero data after write_zeros == + +CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size +_make_test_img -b "$TEST_IMG.base" + +$QEMU_IO -c "write -P 0x11 43k 1k" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z 41k 1k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0x11 43k 1k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 40k 3k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map + +echo +echo == spanning multiple clusters == + +CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size +_make_test_img -b "$TEST_IMG.base" + +$QEMU_IO -c "write -P 0x11 74k 2k" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z 66k 7k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0x11 74k 2k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 64k 10k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map + +# success, all done +echo "*** done" +rm -f $seq.full +status=0 diff --git a/tests/qemu-iotests/154.out b/tests/qemu-iotests/154.out new file mode 100644 index 0000000..daf3f6d --- /dev/null +++ b/tests/qemu-iotests/154.out @@ -0,0 +1,64 @@ +QA output created by 154 + +== backing file contains zeros == +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base +wrote 2048/2048 bytes at offset 0 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 2048/2048 bytes at offset 10240 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 2048/2048 bytes at offset 17408 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +[{ "start": 0, "length": 4096, "depth": 0, "zero": true, "data": false}, +{ "start": 4096, "length": 4096, "depth": 1, "zero": true, "data": false}, +{ "start": 8192, "length": 4096, "depth": 0, "zero": true, "data": false}, +{ "start": 12288, "length": 4096, "depth": 1, "zero": true, "data": false}, +{ "start": 16384, "length": 4096, "depth": 0, "zero": true, "data": false}, +{ "start": 20480, "length": 134197248, "depth": 1, "zero": true, "data": false}] + +== backing file contains non-zero data before write_zeros == +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base +wrote 1024/1024 bytes at offset 32768 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1024/1024 bytes at offset 34816 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 32768 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 3072/3072 bytes at offset 33792 +3 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +[{ "start": 0, "length": 32768, "depth": 1, "zero": true, "data": false}, +{ "start": 32768, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": 20480}, +{ "start": 36864, "length": 134180864, "depth": 1, "zero": true, "data": false}] + +== backing file contains non-zero data after write_zeros == +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base +wrote 1024/1024 bytes at offset 44032 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1024/1024 bytes at offset 41984 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 44032 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 3072/3072 bytes at offset 40960 +3 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +[{ "start": 0, "length": 40960, "depth": 1, "zero": true, "data": false}, +{ "start": 40960, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": 20480}, +{ "start": 45056, "length": 134172672, "depth": 1, "zero": true, "data": false}] + +== spanning multiple clusters == +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base +wrote 2048/2048 bytes at offset 75776 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 7168/7168 bytes at offset 67584 +7 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 2048/2048 bytes at offset 75776 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 10240/10240 bytes at offset 65536 +10 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +[{ "start": 0, "length": 65536, "depth": 1, "zero": true, "data": false}, +{ "start": 65536, "length": 8192, "depth": 0, "zero": true, "data": false}, +{ "start": 73728, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": 20480}, +{ "start": 77824, "length": 134139904, "depth": 1, "zero": true, "data": false}] +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 6067673..ab1d76e 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -153,3 +153,4 @@ 149 rw auto sudo 150 rw auto quick 152 rw auto quick +154 rw auto backing quick -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [Qemu-devel] [PATCH 3/3] qemu-iotests: Some more write_zeroes tests 2016-05-17 16:51 ` [Qemu-devel] [PATCH 3/3] qemu-iotests: Some more write_zeroes tests Kevin Wolf @ 2016-05-17 20:35 ` Eric Blake 2016-05-18 10:09 ` Kevin Wolf 2016-05-18 12:27 ` [Qemu-devel] [PATCH v2 " Kevin Wolf 0 siblings, 2 replies; 12+ messages in thread From: Eric Blake @ 2016-05-17 20:35 UTC (permalink / raw) To: Kevin Wolf, qemu-block; +Cc: mreitz, den, qemu-devel [-- Attachment #1: Type: text/plain, Size: 4315 bytes --] On 05/17/2016 10:51 AM, Kevin Wolf wrote: > This covers some more write_zeroes cases which are relevant for the > recent qcow2 optimisations that check the allocation status of the > backing file for partial cluster write_zeroes requests. > > Signed-off-by: Kevin Wolf <kwolf@redhat.com> > --- > +# qcow2 specific bdrv_write_zeroes tests with backing files (complements 034) > +_supported_fmt qcow2 I guess test 034 covered cases of larger-than-sector write_zero, and that the reason you needed a new test id is because this one is qcow2 specific (the general test covers multiple formats, but the behavior on sub-cluster zeroing is specific to qcow2). Makes sense so far. > + > +CLUSTER_SIZE=4k > +size=128M > + > +echo > +echo == backing file contains zeros == > + > +# Make sure that the whole cluster is allocated even for partial write_zeroes > +# when the backing file contains zeros > + > +CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size > +_make_test_img -b "$TEST_IMG.base" > + > +$QEMU_IO -c "write -z 0k 2k" "$TEST_IMG" | _filter_qemu_io head aligned, tail is unaligned > +$QEMU_IO -c "write -z 10k 2k" "$TEST_IMG" | _filter_qemu_io head is unaligned, tail is unaligned > +$QEMU_IO -c "write -z 17k 2k" "$TEST_IMG" | _filter_qemu_io both unaligned, and completely contained within a single cluster No test of crossing a cluster boundary at this point? I would probably do: $QEMU_IO -c "write -z 27k 2k" "$TEST_IMG" | _filter_qemu_io then check that the map shows an allocation starting at 24k for 8192 bytes > +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map > + > +echo > +echo == backing file contains non-zero data before write_zeros == > + > +CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size > +_make_test_img -b "$TEST_IMG.base" > + > +$QEMU_IO -c "write -P 0x11 32k 1k" "$TEST_IMG.base" | _filter_qemu_io > +$QEMU_IO -c "write -z 34k 1k" "$TEST_IMG" | _filter_qemu_io cluster contains partial non-zero data, which does not overlap with the write zero request. Good - you're testing that we don't nuke the rest of the cluster (that is, we cannot convert an entire cluster to 0 unless all sectors read as 0 before the conversion), but instead do proper copy-on-write. > +$QEMU_IO -c "read -P 0x11 32k 1k" "$TEST_IMG" | _filter_qemu_io > +$QEMU_IO -c "read -P 0 33k 3k" "$TEST_IMG" | _filter_qemu_io > +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map > + > +echo > +echo == backing file contains non-zero data after write_zeros == > + > +CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size > +_make_test_img -b "$TEST_IMG.base" > + > +$QEMU_IO -c "write -P 0x11 43k 1k" "$TEST_IMG.base" | _filter_qemu_io > +$QEMU_IO -c "write -z 41k 1k" "$TEST_IMG" | _filter_qemu_io > +$QEMU_IO -c "read -P 0x11 43k 1k" "$TEST_IMG" | _filter_qemu_io > +$QEMU_IO -c "read -P 0 40k 3k" "$TEST_IMG" | _filter_qemu_io > +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map > + > +echo > +echo == spanning multiple clusters == > + > +CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size > +_make_test_img -b "$TEST_IMG.base" > + > +$QEMU_IO -c "write -P 0x11 74k 2k" "$TEST_IMG.base" | _filter_qemu_io > +$QEMU_IO -c "write -z 66k 7k" "$TEST_IMG" | _filter_qemu_io the tail from 72k to 73k overlaps the sector that has to be copy-on-write to pick up the data at 74k; while the earlier portion picks up the clusters at 64k and 68k. Looks good. What you have looks fine, but again, I'd consider also testing a sub-cluster size 'write -z' that spans a cluster boundary, including the case where it is overwriting data from the backing file. Something like: $QEMU_IO -c "write -P 0x11 2k 4k" "$TEST_IMG.base" | _filter_qemu_io $QEMU_IO -c "write -z 3k 2k" "$TEST_IMG" | _filter_qemu_io $QEMU_IO -c "read -P 0 0k 2k" "$TEST_IMG" | _filter_qemu_io $QEMU_IO -c "read -P 0x11 2k 1k" "$TEST_IMG" | _filter_qemu_io $QEMU_IO -c "read -P 0 3k 2k" "$TEST_IMG" | _filter_qemu_io $QEMU_IO -c "read -P 0x11 5k 1k" "$TEST_IMG" | _filter_qemu_io $QEMU_IO -c "read -P 0 6k 2k" "$TEST_IMG" | _filter_qemu_io -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 604 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Qemu-devel] [PATCH 3/3] qemu-iotests: Some more write_zeroes tests 2016-05-17 20:35 ` Eric Blake @ 2016-05-18 10:09 ` Kevin Wolf 2016-05-18 12:27 ` [Qemu-devel] [PATCH v2 " Kevin Wolf 1 sibling, 0 replies; 12+ messages in thread From: Kevin Wolf @ 2016-05-18 10:09 UTC (permalink / raw) To: Eric Blake; +Cc: qemu-block, mreitz, den, qemu-devel [-- Attachment #1: Type: text/plain, Size: 647 bytes --] Am 17.05.2016 um 22:35 hat Eric Blake geschrieben: > No test of crossing a cluster boundary at this point? I would probably do: > > $QEMU_IO -c "write -z 27k 2k" "$TEST_IMG" | _filter_qemu_io > > then check that the map shows an allocation starting at 24k for 8192 bytes > What you have looks fine, but again, I'd consider also testing a > sub-cluster size 'write -z' that spans a cluster boundary, including the > case where it is overwriting data from the backing file. Okay, I'll put some more time into this test case. What we're testing here is a source for corruption bugs, so it's probably worth doing it right. Kevin [-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* [Qemu-devel] [PATCH v2 3/3] qemu-iotests: Some more write_zeroes tests 2016-05-17 20:35 ` Eric Blake 2016-05-18 10:09 ` Kevin Wolf @ 2016-05-18 12:27 ` Kevin Wolf 2016-05-18 14:12 ` Eric Blake 1 sibling, 1 reply; 12+ messages in thread From: Kevin Wolf @ 2016-05-18 12:27 UTC (permalink / raw) To: qemu-block; +Cc: kwolf, mreitz, eblake, den, qemu-devel This covers some more write_zeroes cases which are relevant for the recent qcow2 optimisations that check the allocation status of the backing file for partial cluster write_zeroes requests. This needs to be separate from 034 because we can only support qcow2 in this test case for multiple reasons: We check the allocation status after write_zeroes with 'qemu-img map' and the optimised behaviour that produces zero clusters is only implemented in qcow2; second, the map command returns offsets that are qcow2 specific; and finally, we also use 512 byte clusters which aren't supported for formats like qed. Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- v2: - Some more test cases, including one suggested by Eric - Comments in the test script that visualise each case with ASCII art tests/qemu-iotests/154 | 265 +++++++++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/154.out | 242 +++++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 508 insertions(+) create mode 100755 tests/qemu-iotests/154 create mode 100644 tests/qemu-iotests/154.out diff --git a/tests/qemu-iotests/154 b/tests/qemu-iotests/154 new file mode 100755 index 0000000..23f1b3a --- /dev/null +++ b/tests/qemu-iotests/154 @@ -0,0 +1,265 @@ +#!/bin/bash +# +# qcow2 specific bdrv_write_zeroes tests with backing files (complements 034) +# +# Copyright (C) 2016 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +# creator +owner=kwolf@redhat.com + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fmt qcow2 +_supported_proto file +_supported_os Linux + +CLUSTER_SIZE=4k +size=128M + +echo +echo == backing file contains zeros == + +CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size +_make_test_img -b "$TEST_IMG.base" + +# Make sure that the whole cluster is allocated even for partial write_zeroes +# when the backing file contains zeros + +# X = non-zero data sector in backing file +# - = sector unallocated in whole backing chain +# 0 = sector touched by write_zeroes request + +# 1. Tail unaligned: 00 00 -- -- +# 2. Head unaligned: -- -- 00 00 +# 3. Both unaligned: -- 00 00 -- +# 4. Both, 2 clusters: -- -- -- 00 | 00 -- -- -- + +$QEMU_IO -c "write -z 0 2k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "write -z 10k 2k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "write -z 17k 2k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "write -z 27k 2k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map + +echo +echo == backing file contains non-zero data before write_zeroes == + +CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size +_make_test_img -b "$TEST_IMG.base" + +# Single cluster; non-zero data at the cluster start +# ... | XX -- 00 -- | ... +$QEMU_IO -c "write -P 0x11 32k 1k" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z 34k 1k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0x11 32k 1k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 33k 3k" "$TEST_IMG" | _filter_qemu_io + +# Single cluster; non-zero data exists, but not at the cluster start +# ... | -- XX 00 -- | ... +$QEMU_IO -c "write -P 0x11 65k 1k" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z 66k 1k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0x11 65k 1k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 64k 1k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 66k 2k" "$TEST_IMG" | _filter_qemu_io + +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map + +echo +echo == backing file contains non-zero data after write_zeroes == + +CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size +_make_test_img -b "$TEST_IMG.base" + +# Single cluster; non-zero data directly after request +# ... | -- 00 XX -- | ... +$QEMU_IO -c "write -P 0x11 34k 1k" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z 33k 1k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 32k 2k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0x11 34k 1k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 35k 1k" "$TEST_IMG" | _filter_qemu_io + +# Single cluster; non-zero data exists, but not directly after request +# ... | -- 00 -- XX | ... +$QEMU_IO -c "write -P 0x11 43k 1k" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z 41k 1k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0x11 43k 1k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 40k 3k" "$TEST_IMG" | _filter_qemu_io + +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map + +echo +echo == spanning two clusters, non-zero before request == + +CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size +_make_test_img -b "$TEST_IMG.base" + +# Two clusters; non-zero data before request: +# 1. At cluster start: 32k: XX -- -- 00 | 00 -- -- -- +# 2. Between unallocated space: 48k: -- XX -- 00 | 00 -- -- -- +# 3. Directly before request: 64k: -- -- XX 00 | 00 -- -- -- + +$QEMU_IO -c "write -P 0x11 32k 1k" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z 35k 2k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0x11 32k 1k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 33k 7k" "$TEST_IMG" | _filter_qemu_io + +$QEMU_IO -c "write -P 0x11 49k 1k" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z 51k 2k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 48k 1k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0x11 49k 1k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 50k 6k" "$TEST_IMG" | _filter_qemu_io + +$QEMU_IO -c "write -P 0x11 66k 1k" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z 67k 2k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 64k 2k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0x11 66k 1k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 67k 5k" "$TEST_IMG" | _filter_qemu_io + +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map + +echo +echo == spanning two clusters, non-zero after request == + +CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size +_make_test_img -b "$TEST_IMG.base" + +# Two clusters; non-zero data after request: +# 1. Directly after request: 32k: -- -- -- 00 | 00 XX -- -- +# 2. Between unallocated space: 48k: -- -- -- 00 | 00 -- XX -- +# 3. At cluster end: 64k: -- -- -- 00 | 00 -- -- XX + +$QEMU_IO -c "write -P 0x11 37k 1k" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z 35k 2k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 32k 5k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0x11 37k 1k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 38k 2k" "$TEST_IMG" | _filter_qemu_io + +$QEMU_IO -c "write -P 0x11 54k 1k" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z 51k 2k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 48k 6k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0x11 54k 1k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 55k 1k" "$TEST_IMG" | _filter_qemu_io + +$QEMU_IO -c "write -P 0x11 71k 1k" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z 67k 2k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 64k 7k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0x11 71k 1k" "$TEST_IMG" | _filter_qemu_io + +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map + +echo +echo == spanning two clusters, partially overwriting backing file == + +CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size +_make_test_img -b "$TEST_IMG.base" + +# Backing file: -- -- XX XX | XX XX -- -- +# Active layer: -- -- XX 00 | 00 XX -- -- + +$QEMU_IO -c "write -P 0x11 2k 4k" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z 3k 2k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 0k 2k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0x11 2k 1k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 3k 2k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0x11 5k 1k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 6k 2k" "$TEST_IMG" | _filter_qemu_io + +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map + +echo +echo == spanning multiple clusters, non-zero in first cluster == + +CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size +_make_test_img -b "$TEST_IMG.base" + +# Backing file: 64k: XX XX -- -- | -- -- -- -- | -- -- -- -- +# Active layer: 64k: XX XX 00 00 | 00 00 00 00 | 00 -- -- -- + +$QEMU_IO -c "write -P 0x11 64k 2k" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z 66k 7k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0x11 64k 2k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 66k 10k" "$TEST_IMG" | _filter_qemu_io + +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map + +echo +echo == spanning multiple clusters, non-zero in intermediate cluster == + +CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size +_make_test_img -b "$TEST_IMG.base" + +# Backing file: 64k: -- -- -- -- | -- XX XX -- | -- -- -- -- +# Active layer: 64k: -- -- 00 00 | 00 00 00 00 | 00 -- -- -- + +$QEMU_IO -c "write -P 0x11 69k 2k" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z 66k 7k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 64k 12k" "$TEST_IMG" | _filter_qemu_io + +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map + +echo +echo == spanning multiple clusters, non-zero in final cluster == + +CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size +_make_test_img -b "$TEST_IMG.base" + +# Backing file: 64k: -- -- -- -- | -- -- -- -- | -- -- XX XX +# Active layer: 64k: -- -- 00 00 | 00 00 00 00 | 00 -- XX XX + +$QEMU_IO -c "write -P 0x11 74k 2k" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z 66k 7k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 64k 10k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0x11 74k 2k" "$TEST_IMG" | _filter_qemu_io + +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map + +echo +echo == spanning multiple clusters, partially overwriting backing file == + +CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size +_make_test_img -b "$TEST_IMG.base" + +# Backing file: 64k: -- XX XX XX | XX XX XX XX | XX XX XX -- +# Active layer: 64k: -- XX 00 00 | 00 00 00 00 | 00 XX XX -- + +$QEMU_IO -c "write -P 0x11 65k 10k" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z 66k 7k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 64k 1k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0x11 65k 1k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 66k 7k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0x11 73k 2k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 75k 1k" "$TEST_IMG" | _filter_qemu_io + +$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map + +# success, all done +echo "*** done" +rm -f $seq.full +status=0 diff --git a/tests/qemu-iotests/154.out b/tests/qemu-iotests/154.out new file mode 100644 index 0000000..8946b73 --- /dev/null +++ b/tests/qemu-iotests/154.out @@ -0,0 +1,242 @@ +QA output created by 154 + +== backing file contains zeros == +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base +wrote 2048/2048 bytes at offset 0 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 2048/2048 bytes at offset 10240 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 2048/2048 bytes at offset 17408 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 2048/2048 bytes at offset 27648 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +[{ "start": 0, "length": 4096, "depth": 0, "zero": true, "data": false}, +{ "start": 4096, "length": 4096, "depth": 1, "zero": true, "data": false}, +{ "start": 8192, "length": 4096, "depth": 0, "zero": true, "data": false}, +{ "start": 12288, "length": 4096, "depth": 1, "zero": true, "data": false}, +{ "start": 16384, "length": 4096, "depth": 0, "zero": true, "data": false}, +{ "start": 20480, "length": 4096, "depth": 1, "zero": true, "data": false}, +{ "start": 24576, "length": 8192, "depth": 0, "zero": true, "data": false}, +{ "start": 32768, "length": 134184960, "depth": 1, "zero": true, "data": false}] + +== backing file contains non-zero data before write_zeroes == +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base +wrote 1024/1024 bytes at offset 32768 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1024/1024 bytes at offset 34816 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 32768 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 3072/3072 bytes at offset 33792 +3 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1024/1024 bytes at offset 66560 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1024/1024 bytes at offset 67584 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 66560 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 65536 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 2048/2048 bytes at offset 67584 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +[{ "start": 0, "length": 32768, "depth": 1, "zero": true, "data": false}, +{ "start": 32768, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": 20480}, +{ "start": 36864, "length": 28672, "depth": 1, "zero": true, "data": false}, +{ "start": 65536, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": 24576}, +{ "start": 69632, "length": 134148096, "depth": 1, "zero": true, "data": false}] + +== backing file contains non-zero data after write_zeroes == +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base +wrote 1024/1024 bytes at offset 34816 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1024/1024 bytes at offset 33792 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 2048/2048 bytes at offset 32768 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 34816 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 35840 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1024/1024 bytes at offset 44032 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1024/1024 bytes at offset 41984 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 44032 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 3072/3072 bytes at offset 40960 +3 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +[{ "start": 0, "length": 32768, "depth": 1, "zero": true, "data": false}, +{ "start": 32768, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": 20480}, +{ "start": 36864, "length": 4096, "depth": 1, "zero": true, "data": false}, +{ "start": 40960, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": 24576}, +{ "start": 45056, "length": 134172672, "depth": 1, "zero": true, "data": false}] + +== spanning two clusters, non-zero before request == +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base +wrote 1024/1024 bytes at offset 32768 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 2048/2048 bytes at offset 35840 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 32768 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 7168/7168 bytes at offset 33792 +7 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1024/1024 bytes at offset 50176 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 2048/2048 bytes at offset 52224 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 49152 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 50176 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 6144/6144 bytes at offset 51200 +6 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1024/1024 bytes at offset 67584 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 2048/2048 bytes at offset 68608 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 2048/2048 bytes at offset 65536 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 67584 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 5120/5120 bytes at offset 68608 +5 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +[{ "start": 0, "length": 32768, "depth": 1, "zero": true, "data": false}, +{ "start": 32768, "length": 8192, "depth": 0, "zero": false, "data": true, "offset": 20480}, +{ "start": 40960, "length": 8192, "depth": 1, "zero": true, "data": false}, +{ "start": 49152, "length": 8192, "depth": 0, "zero": false, "data": true, "offset": 28672}, +{ "start": 57344, "length": 8192, "depth": 1, "zero": true, "data": false}, +{ "start": 65536, "length": 8192, "depth": 0, "zero": false, "data": true, "offset": 36864}, +{ "start": 73728, "length": 134144000, "depth": 1, "zero": true, "data": false}] + +== spanning two clusters, non-zero after request == +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base +wrote 1024/1024 bytes at offset 37888 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 2048/2048 bytes at offset 35840 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 5120/5120 bytes at offset 32768 +5 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 37888 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 2048/2048 bytes at offset 38912 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1024/1024 bytes at offset 55296 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 2048/2048 bytes at offset 52224 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 6144/6144 bytes at offset 49152 +6 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 55296 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 56320 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1024/1024 bytes at offset 72704 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 2048/2048 bytes at offset 68608 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 7168/7168 bytes at offset 65536 +7 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 72704 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +[{ "start": 0, "length": 32768, "depth": 1, "zero": true, "data": false}, +{ "start": 32768, "length": 8192, "depth": 0, "zero": false, "data": true, "offset": 20480}, +{ "start": 40960, "length": 8192, "depth": 1, "zero": true, "data": false}, +{ "start": 49152, "length": 8192, "depth": 0, "zero": false, "data": true, "offset": 28672}, +{ "start": 57344, "length": 8192, "depth": 1, "zero": true, "data": false}, +{ "start": 65536, "length": 8192, "depth": 0, "zero": false, "data": true, "offset": 36864}, +{ "start": 73728, "length": 134144000, "depth": 1, "zero": true, "data": false}] + +== spanning two clusters, partially overwriting backing file == +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base +wrote 4096/4096 bytes at offset 2048 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 2048/2048 bytes at offset 3072 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 2048/2048 bytes at offset 0 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 2048 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 2048/2048 bytes at offset 3072 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 5120 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 2048/2048 bytes at offset 6144 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +[{ "start": 0, "length": 8192, "depth": 0, "zero": false, "data": true, "offset": 20480}, +{ "start": 8192, "length": 134209536, "depth": 1, "zero": true, "data": false}] + +== spanning multiple clusters, non-zero in first cluster == +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base +wrote 2048/2048 bytes at offset 65536 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 7168/7168 bytes at offset 67584 +7 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 2048/2048 bytes at offset 65536 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 10240/10240 bytes at offset 67584 +10 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +[{ "start": 0, "length": 65536, "depth": 1, "zero": true, "data": false}, +{ "start": 65536, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": 20480}, +{ "start": 69632, "length": 8192, "depth": 0, "zero": true, "data": false}, +{ "start": 77824, "length": 134139904, "depth": 1, "zero": true, "data": false}] + +== spanning multiple clusters, non-zero in intermediate cluster == +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base +wrote 2048/2048 bytes at offset 70656 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 7168/7168 bytes at offset 67584 +7 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 12288/12288 bytes at offset 65536 +12 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +[{ "start": 0, "length": 65536, "depth": 1, "zero": true, "data": false}, +{ "start": 65536, "length": 12288, "depth": 0, "zero": true, "data": false}, +{ "start": 77824, "length": 134139904, "depth": 1, "zero": true, "data": false}] + +== spanning multiple clusters, non-zero in final cluster == +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base +wrote 2048/2048 bytes at offset 75776 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 7168/7168 bytes at offset 67584 +7 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 10240/10240 bytes at offset 65536 +10 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 2048/2048 bytes at offset 75776 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +[{ "start": 0, "length": 65536, "depth": 1, "zero": true, "data": false}, +{ "start": 65536, "length": 8192, "depth": 0, "zero": true, "data": false}, +{ "start": 73728, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": 20480}, +{ "start": 77824, "length": 134139904, "depth": 1, "zero": true, "data": false}] + +== spanning multiple clusters, partially overwriting backing file == +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base +wrote 10240/10240 bytes at offset 66560 +10 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 7168/7168 bytes at offset 67584 +7 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 65536 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 66560 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 7168/7168 bytes at offset 67584 +7 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 2048/2048 bytes at offset 74752 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 76800 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +[{ "start": 0, "length": 65536, "depth": 1, "zero": true, "data": false}, +{ "start": 65536, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": 20480}, +{ "start": 69632, "length": 4096, "depth": 0, "zero": true, "data": false}, +{ "start": 73728, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": 24576}, +{ "start": 77824, "length": 134139904, "depth": 1, "zero": true, "data": false}] +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 6067673..ab1d76e 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -153,3 +153,4 @@ 149 rw auto sudo 150 rw auto quick 152 rw auto quick +154 rw auto backing quick -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [Qemu-devel] [PATCH v2 3/3] qemu-iotests: Some more write_zeroes tests 2016-05-18 12:27 ` [Qemu-devel] [PATCH v2 " Kevin Wolf @ 2016-05-18 14:12 ` Eric Blake 2016-05-18 14:22 ` Kevin Wolf 0 siblings, 1 reply; 12+ messages in thread From: Eric Blake @ 2016-05-18 14:12 UTC (permalink / raw) To: Kevin Wolf, qemu-block; +Cc: mreitz, den, qemu-devel [-- Attachment #1: Type: text/plain, Size: 1301 bytes --] On 05/18/2016 06:27 AM, Kevin Wolf wrote: > This covers some more write_zeroes cases which are relevant for the > recent qcow2 optimisations that check the allocation status of the > backing file for partial cluster write_zeroes requests. > > This needs to be separate from 034 because we can only support qcow2 in > this test case for multiple reasons: We check the allocation status > after write_zeroes with 'qemu-img map' and the optimised behaviour that > produces zero clusters is only implemented in qcow2; second, the map > command returns offsets that are qcow2 specific; and finally, we also > use 512 byte clusters which aren't supported for formats like qed. > > Signed-off-by: Kevin Wolf <kwolf@redhat.com> > --- > > v2: > - Some more test cases, including one suggested by Eric > - Comments in the test script that visualise each case with ASCII art Nice! > +# qcow2 specific bdrv_write_zeroes tests with backing files (complements 034) Maybe also worth a one line tweak to 034 to give a comment pointing to this file (so the two mutually reference each other), but that doesn't require a respin. Reviewed-by: Eric Blake <eblake@redhat.com> -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 604 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Qemu-devel] [PATCH v2 3/3] qemu-iotests: Some more write_zeroes tests 2016-05-18 14:12 ` Eric Blake @ 2016-05-18 14:22 ` Kevin Wolf 0 siblings, 0 replies; 12+ messages in thread From: Kevin Wolf @ 2016-05-18 14:22 UTC (permalink / raw) To: Eric Blake; +Cc: qemu-block, mreitz, den, qemu-devel [-- Attachment #1: Type: text/plain, Size: 1356 bytes --] Am 18.05.2016 um 16:12 hat Eric Blake geschrieben: > On 05/18/2016 06:27 AM, Kevin Wolf wrote: > > This covers some more write_zeroes cases which are relevant for the > > recent qcow2 optimisations that check the allocation status of the > > backing file for partial cluster write_zeroes requests. > > > > This needs to be separate from 034 because we can only support qcow2 in > > this test case for multiple reasons: We check the allocation status > > after write_zeroes with 'qemu-img map' and the optimised behaviour that > > produces zero clusters is only implemented in qcow2; second, the map > > command returns offsets that are qcow2 specific; and finally, we also > > use 512 byte clusters which aren't supported for formats like qed. > > > > Signed-off-by: Kevin Wolf <kwolf@redhat.com> > > --- > > > > v2: > > - Some more test cases, including one suggested by Eric > > - Comments in the test script that visualise each case with ASCII art > > Nice! > > > +# qcow2 specific bdrv_write_zeroes tests with backing files (complements 034) > > Maybe also worth a one line tweak to 034 to give a comment pointing to > this file (so the two mutually reference each other), but that doesn't > require a respin. > > Reviewed-by: Eric Blake <eblake@redhat.com> Thanks, I pushed this to my block branch now. Kevin [-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2016-05-18 14:22 UTC | newest] Thread overview: 12+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2016-05-17 16:51 [Qemu-devel] [PATCH 0/3] qcow2: write_zeroes corruption fixes and tests Kevin Wolf 2016-05-17 16:51 ` [Qemu-devel] [PATCH 1/3] qcow2: fix condition in is_zero_cluster Kevin Wolf 2016-05-17 17:25 ` Eric Blake 2016-05-17 16:51 ` [Qemu-devel] [PATCH 2/3] qcow2: Fix write_zeroes with partially allocated backing file cluster Kevin Wolf 2016-05-17 17:22 ` Eric Blake 2016-05-17 19:20 ` Denis V. Lunev 2016-05-17 16:51 ` [Qemu-devel] [PATCH 3/3] qemu-iotests: Some more write_zeroes tests Kevin Wolf 2016-05-17 20:35 ` Eric Blake 2016-05-18 10:09 ` Kevin Wolf 2016-05-18 12:27 ` [Qemu-devel] [PATCH v2 " Kevin Wolf 2016-05-18 14:12 ` Eric Blake 2016-05-18 14:22 ` Kevin Wolf
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).