[Qemu-devel] [PATCH v3 0/2] block: fix write with zero flag set and iovector provided

[Qemu-devel] [PATCH v3 0/2] block: fix write with zero flag set and iovector provided

[Anton Nefedov]

v3: patch 1: image cluster size reduced to get away with a smaller test image
    (the cluster size can be as small as 512 bytes for qcow2,
     but the test runs for all generic formats and minimum for qed is 4k)

v2: http://lists.nongnu.org/archive/html/qemu-devel/2018-02/msg03016.html

Anton Nefedov (2):
  iotest 033: add misaligned write-zeroes test via truncate
  block: fix write with zero flag set and iovector provided

 block/io.c                 |  2 +-
 tests/qemu-iotests/033     | 29 +++++++++++++++++++++++++++++
 tests/qemu-iotests/033.out | 13 +++++++++++++
 3 files changed, 43 insertions(+), 1 deletion(-)

-- 
2.7.4

[Qemu-devel] [PATCH v3 1/2] iotest 033: add misaligned write-zeroes test via truncate

[Anton Nefedov]

This new test case only makes sense for qcow2 while iotest 033 is generic;
however it matches the test purpose perfectly and also 033 contains those
do_test() tricks to pass the alignment, which won't look nice being
duplicated in other tests or moved to the common code.

Signed-off-by: Anton Nefedov <anton.nefedov@virtuozzo.com>
---
 tests/qemu-iotests/033     | 29 +++++++++++++++++++++++++++++
 tests/qemu-iotests/033.out | 13 +++++++++++++
 2 files changed, 42 insertions(+)

diff --git a/tests/qemu-iotests/033 b/tests/qemu-iotests/033
index 2cdfd13..a1d8357 100755
--- a/tests/qemu-iotests/033
+++ b/tests/qemu-iotests/033
@@ -64,6 +64,9 @@ do_test()
 	} | $QEMU_IO $IO_EXTRA_ARGS
 }
 
+echo
+echo "=== Test aligned and misaligned write zeroes operations ==="
+
 for write_zero_cmd in "write -z" "aio_write -z"; do
 for align in 512 4k; do
 	echo
@@ -102,7 +105,33 @@ for align in 512 4k; do
 done
 done
 
+
+# Trigger truncate that would shrink qcow2 L1 table, which is done by
+#   clearing one entry (8 bytes) with bdrv_co_pwrite_zeroes()
+
+echo
+echo "=== Test misaligned write zeroes via truncate ==="
+echo
+
+# any size will do, but the smaller the size the smaller the required image
+CLUSTER_SIZE=$((4 * 1024))
+L2_COVERAGE=$(($CLUSTER_SIZE * $CLUSTER_SIZE / 8))
+_make_test_img $(($L2_COVERAGE * 2))
+
+do_test 512 "write -P 1 0 0x200" "$TEST_IMG" | _filter_qemu_io
+# next L2 table
+do_test 512 "write -P 1 $L2_COVERAGE 0x200" "$TEST_IMG" | _filter_qemu_io
+
+# only interested in qcow2 here; also other formats might respond with
+#  "not supported" error message
+if [ $IMGFMT = "qcow2" ]; then
+    do_test 512 "truncate $L2_COVERAGE" "$TEST_IMG" | _filter_qemu_io
+fi
+
+do_test 512 "read -P 1 0 0x200" "$TEST_IMG" | _filter_qemu_io
+
 # success, all done
+echo
 echo "*** done"
 rm -f $seq.full
 status=0
diff --git a/tests/qemu-iotests/033.out b/tests/qemu-iotests/033.out
index 95929ef..9683f6b 100644
--- a/tests/qemu-iotests/033.out
+++ b/tests/qemu-iotests/033.out
@@ -1,6 +1,8 @@
 QA output created by 033
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
 
+=== Test aligned and misaligned write zeroes operations ===
+
 == preparing image ==
 wrote 1024/1024 bytes at offset 512
 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -164,4 +166,15 @@ read 512/512 bytes at offset 512
 read 3072/3072 bytes at offset 1024
 3 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
+
+=== Test misaligned write zeroes via truncate ===
+
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4194304
+wrote 512/512 bytes at offset 0
+512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 512/512 bytes at offset 2097152
+512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 512/512 bytes at offset 0
+512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
 *** done
-- 
2.7.4

Re: [Qemu-devel] [PATCH v3 1/2] iotest 033: add misaligned write-zeroes test via truncate

[Eric Blake]

On 02/14/2018 10:09 AM, Anton Nefedov wrote:
> This new test case only makes sense for qcow2 while iotest 033 is generic;
> however it matches the test purpose perfectly and also 033 contains those
> do_test() tricks to pass the alignment, which won't look nice being
> duplicated in other tests or moved to the common code.
> 
> Signed-off-by: Anton Nefedov <anton.nefedov@virtuozzo.com>
> ---
>   tests/qemu-iotests/033     | 29 +++++++++++++++++++++++++++++
>   tests/qemu-iotests/033.out | 13 +++++++++++++
>   2 files changed, 42 insertions(+)
> 

> +# only interested in qcow2 here; also other formats might respond with
> +#  "not supported" error message
> +if [ $IMGFMT = "qcow2" ]; then
> +    do_test 512 "truncate $L2_COVERAGE" "$TEST_IMG" | _filter_qemu_io
> +fi

But without an else branch that echoes the same text as the if branch 
generates, your .out file is now broken for other image formats.  Or 
does 'qemu-io -c truncate' not produce output?

/me goes and tests...

Okay, looks like truncate is silent; and that the truncation (or 
skipping of the truncation) doesn't affect things.

Tested-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH v3 2/2] block: fix write with zero flag set and iovector provided

[Anton Nefedov]

The normal bdrv_co_pwritev() use is either
  - BDRV_REQ_ZERO_WRITE clear and iovector provided
  - BDRV_REQ_ZERO_WRITE set and iovector == NULL

while
  - the flag clear and iovector == NULL is an assertion failure
    in bdrv_co_do_zero_pwritev()
  - the flag set and iovector provided is in fact allowed
    (the flag prevails and zeroes are written)

However the alignment logic does not support the latter case so the padding
areas get overwritten with zeroes.

Currently, general functions like bdrv_rw_co() do provide iovector
regardless of flags. So, keep it supported and use bdrv_co_do_zero_pwritev()
alignment for it which also makes the code a bit more obvious anyway.

Signed-off-by: Anton Nefedov <anton.nefedov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
---
 block/io.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/block/io.c b/block/io.c
index 89d0745..40df3be 100644
--- a/block/io.c
+++ b/block/io.c
@@ -1701,7 +1701,7 @@ int coroutine_fn bdrv_co_pwritev(BdrvChild *child,
      */
     tracked_request_begin(&req, bs, offset, bytes, BDRV_TRACKED_WRITE);
 
-    if (!qiov) {
+    if (flags & BDRV_REQ_ZERO_WRITE) {
         ret = bdrv_co_do_zero_pwritev(child, offset, bytes, flags, &req);
         goto out;
     }
-- 
2.7.4

Re: [Qemu-devel] [PATCH v3 0/2] block: fix write with zero flag set and iovector provided

[Kevin Wolf]

Am 14.02.2018 um 17:09 hat Anton Nefedov geschrieben:
> v3: patch 1: image cluster size reduced to get away with a smaller test image
>     (the cluster size can be as small as 512 bytes for qcow2,
>      but the test runs for all generic formats and minimum for qed is 4k)

Thanks, applied to the block branch (in reverse order, tests should
always pass).

Kevin