[PATCH 0/2] block/fuse: Let PUNCH_HOLE write zeroes

[PATCH 0/2] block/fuse: Let PUNCH_HOLE write zeroes

[Hanna Czenczek]

Hi,

https://gitlab.com/qemu-project/qemu/-/issues/1507 reports a bug in FUSE
exports: fallocate(PUNCH_HOLE) is implemented with blk_pdiscard(), but
its man page documents that a successful call will result in the data
being read as zero.  blk_pdiscard() does not guarantee this, so we must
use blk_pwrite_zeroes() instead (with MAY_UNMAP | NO_FALLBACK, which
differentiates it from fallocate(ZERO_RANGE)).

Patch 2 adds a regression test.


Hanna Czenczek (2):
  block/fuse: Let PUNCH_HOLE write zeroes
  iotests/308: Add test for 'write -zu'

 block/export/fuse.c        | 11 +++++++++-
 tests/qemu-iotests/308     | 43 ++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/308.out | 35 +++++++++++++++++++++++++++++++
 3 files changed, 88 insertions(+), 1 deletion(-)

-- 
2.39.1

[PATCH 1/2] block/fuse: Let PUNCH_HOLE write zeroes

[Hanna Czenczek]

fallocate(2) says about PUNCH_HOLE: "After a successful call, subsequent
reads from this range will return zeros."  As it is, PUNCH_HOLE is
implemented as a call to blk_pdiscard(), which does not guarantee this.

We must call blk_pwrite_zeroes() instead.  The difference to ZERO_RANGE
is that we pass the `BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK` flags to
the call -- the storage is supposed to be unmapped, and a slow fallback
by actually writing zeroes as data is not allowed.

Closes: https://gitlab.com/qemu-project/qemu/-/issues/1507
Signed-off-by: Hanna Czenczek <hreitz@redhat.com>
---
 block/export/fuse.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/block/export/fuse.c b/block/export/fuse.c
index e5fc4af165..06fa41079e 100644
--- a/block/export/fuse.c
+++ b/block/export/fuse.c
@@ -673,7 +673,16 @@ static void fuse_fallocate(fuse_req_t req, fuse_ino_t inode, int mode,
         do {
             int size = MIN(length, BDRV_REQUEST_MAX_BYTES);
 
-            ret = blk_pdiscard(exp->common.blk, offset, size);
+            ret = blk_pwrite_zeroes(exp->common.blk, offset, size,
+                                    BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK);
+            if (ret == -ENOTSUP) {
+                /*
+                 * fallocate() specifies to return EOPNOTSUPP for unsupported
+                 * operations
+                 */
+                ret = -EOPNOTSUPP;
+            }
+
             offset += size;
             length -= size;
         } while (ret == 0 && length > 0);
-- 
2.39.1

[PATCH 2/2] iotests/308: Add test for 'write -zu'

[Hanna Czenczek]

Try writing zeroes to a FUSE export while allowing the area to be
unmapped; block/file-posix.c generally implements writing zeroes with
BDRV_REQ_MAY_UNMAP ('write -zu') by calling fallocate(PUNCH_HOLE).  This
used to lead to a blk_pdiscard() in the FUSE export, which may or may
not lead to the area being zeroed.  HEAD^ fixed this to use
blk_pwrite_zeroes() instead (again with BDRV_REQ_MAY_UNMAP), so verify
that running `qemu-io 'write -zu'` on a FUSE exports always results in
zeroes being written.

Signed-off-by: Hanna Czenczek <hreitz@redhat.com>
---
 tests/qemu-iotests/308     | 43 ++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/308.out | 35 +++++++++++++++++++++++++++++++
 2 files changed, 78 insertions(+)

diff --git a/tests/qemu-iotests/308 b/tests/qemu-iotests/308
index 09275e9a10..de12b2b1b9 100755
--- a/tests/qemu-iotests/308
+++ b/tests/qemu-iotests/308
@@ -370,6 +370,49 @@ echo
 echo '=== Compare copy with original ==='
 
 $QEMU_IMG compare -f raw -F $IMGFMT "$COPIED_IMG" "$TEST_IMG"
+_cleanup_test_img
+
+echo
+echo '=== Writing zeroes while unmapping ==='
+# Regression test for https://gitlab.com/qemu-project/qemu/-/issues/1507
+_make_test_img 64M
+$QEMU_IO -c 'write -s /dev/urandom 0 64M' "$TEST_IMG" | _filter_qemu_io
+
+_launch_qemu
+_send_qemu_cmd $QEMU_HANDLE \
+    "{'execute': 'qmp_capabilities'}" \
+    'return'
+
+_send_qemu_cmd $QEMU_HANDLE \
+    "{'execute': 'blockdev-add',
+      'arguments': {
+          'driver': '$IMGFMT',
+          'node-name': 'node-format',
+          'file': {
+              'driver': 'file',
+              'filename': '$TEST_IMG'
+          }
+      } }" \
+    'return'
+
+fuse_export_add 'export' "'mountpoint': '$EXT_MP', 'writable': true"
+
+# Try writing zeroes by unmapping
+$QEMU_IO -f raw -c 'write -zu 0 64M' "$EXT_MP" | _filter_qemu_io
+
+# Check the result
+$QEMU_IO -f raw -c 'read -P 0 0 64M' "$EXT_MP" | _filter_qemu_io
+
+_send_qemu_cmd $QEMU_HANDLE \
+    "{'execute': 'quit'}" \
+    'return'
+
+wait=yes _cleanup_qemu
+
+# Check the original image
+$QEMU_IO -c 'read -P 0 0 64M' "$TEST_IMG" | _filter_qemu_io
+
+_cleanup_test_img
 
 # success, all done
 echo "*** done"
diff --git a/tests/qemu-iotests/308.out b/tests/qemu-iotests/308.out
index e4467a10cf..d5767133b1 100644
--- a/tests/qemu-iotests/308.out
+++ b/tests/qemu-iotests/308.out
@@ -171,4 +171,39 @@ OK: Post-truncate image size is as expected
 
 === Compare copy with original ===
 Images are identical.
+
+=== Writing zeroes while unmapping ===
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+wrote 67108864/67108864 bytes at offset 0
+64 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+{'execute': 'qmp_capabilities'}
+{"return": {}}
+{'execute': 'blockdev-add',
+      'arguments': {
+          'driver': 'IMGFMT',
+          'node-name': 'node-format',
+          'file': {
+              'driver': 'file',
+              'filename': 'TEST_DIR/t.IMGFMT'
+          }
+      } }
+{"return": {}}
+{'execute': 'block-export-add',
+          'arguments': {
+              'type': 'fuse',
+              'id': 'export',
+              'node-name': 'node-format',
+              'mountpoint': 'TEST_DIR/t.IMGFMT.fuse', 'writable': true
+          } }
+{"return": {}}
+wrote 67108864/67108864 bytes at offset 0
+64 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 67108864/67108864 bytes at offset 0
+64 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+{'execute': 'quit'}
+{"return": {}}
+{"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}}
+{"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "export"}}
+read 67108864/67108864 bytes at offset 0
+64 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 *** done
-- 
2.39.1

Re: [PATCH 0/2] block/fuse: Let PUNCH_HOLE write zeroes

[Kevin Wolf]

Am 27.02.2023 um 11:47 hat Hanna Czenczek geschrieben:
> Hi,
> 
> https://gitlab.com/qemu-project/qemu/-/issues/1507 reports a bug in FUSE
> exports: fallocate(PUNCH_HOLE) is implemented with blk_pdiscard(), but
> its man page documents that a successful call will result in the data
> being read as zero.  blk_pdiscard() does not guarantee this, so we must
> use blk_pwrite_zeroes() instead (with MAY_UNMAP | NO_FALLBACK, which
> differentiates it from fallocate(ZERO_RANGE)).
> 
> Patch 2 adds a regression test.

Thanks, applied to the block branch.

Kevin