[Qemu-devel] [PATCH v3 0/2] qcow2: include LUKS payload overhead in qemu-img measure

[Qemu-devel] [PATCH v3 0/2] qcow2: include LUKS payload overhead in qemu-img measure

[Stefan Hajnoczi]

v3:
 * Use encrypt.iter-time=10 to speed up qemu-iotests 178 [Max]
v2:
 * Take into account all "encrypt." image creation options since the LUKS
   payload size varies depending on the cipher [Max]

The LUKS payload has a significant size (>1 MB).  It must be included in the
qemu-img measure calculation so we arrive at the correct minimum image size.

Stefan Hajnoczi (2):
  qcow2: include LUKS payload overhead in qemu-img measure
  iotests: add LUKS payload overhead to 178 qemu-img measure test

 block/qcow2.c                    | 72 +++++++++++++++++++++++++++++++-
 tests/qemu-iotests/178           |  8 ++++
 tests/qemu-iotests/178.out.qcow2 | 24 +++++++++++
 3 files changed, 103 insertions(+), 1 deletion(-)

-- 
2.20.1

[Qemu-devel] [PATCH v3 1/2] qcow2: include LUKS payload overhead in qemu-img measure

[Stefan Hajnoczi]

LUKS encryption reserves clusters for its own payload data.  The size of
this area must be included in the qemu-img measure calculation so that
we arrive at the correct minimum required image size.

(Ab)use the qcrypto_block_create() API to determine the payload
overhead.  We discard the payload data that qcrypto thinks will be
written to the image.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
---
 block/qcow2.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 71 insertions(+), 1 deletion(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index 65a54c9ac6..81722177d6 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -4232,6 +4232,60 @@ static coroutine_fn int qcow2_co_flush_to_os(BlockDriverState *bs)
     return ret;
 }
 
+static ssize_t qcow2_measure_crypto_hdr_init_func(QCryptoBlock *block,
+        size_t headerlen, void *opaque, Error **errp)
+{
+    size_t *headerlenp = opaque;
+
+    /* Stash away the payload size */
+    *headerlenp = headerlen;
+    return 0;
+}
+
+static ssize_t qcow2_measure_crypto_hdr_write_func(QCryptoBlock *block,
+        size_t offset, const uint8_t *buf, size_t buflen,
+        void *opaque, Error **errp)
+{
+    /* Discard the bytes, we're not actually writing to an image */
+    return buflen;
+}
+
+/* Determine the number of bytes for the LUKS payload */
+static bool qcow2_measure_luks_headerlen(QemuOpts *opts, size_t *len,
+                                         Error **errp)
+{
+    QDict *opts_qdict;
+    QDict *cryptoopts_qdict;
+    QCryptoBlockCreateOptions *cryptoopts;
+    QCryptoBlock *crypto;
+
+    /* Extract "encrypt." options into a qdict */
+    opts_qdict = qemu_opts_to_qdict(opts, NULL);
+    qdict_extract_subqdict(opts_qdict, &cryptoopts_qdict, "encrypt.");
+    qobject_unref(opts_qdict);
+
+    /* Build QCryptoBlockCreateOptions object from qdict */
+    qdict_put_str(cryptoopts_qdict, "format", "luks");
+    cryptoopts = block_crypto_create_opts_init(cryptoopts_qdict, errp);
+    qobject_unref(cryptoopts_qdict);
+    if (!cryptoopts) {
+        return false;
+    }
+
+    /* Fake LUKS creation in order to determine the payload size */
+    crypto = qcrypto_block_create(cryptoopts, "encrypt.",
+                                  qcow2_measure_crypto_hdr_init_func,
+                                  qcow2_measure_crypto_hdr_write_func,
+                                  len, errp);
+    qapi_free_QCryptoBlockCreateOptions(cryptoopts);
+    if (!crypto) {
+        return false;
+    }
+
+    qcrypto_block_free(crypto);
+    return true;
+}
+
 static BlockMeasureInfo *qcow2_measure(QemuOpts *opts, BlockDriverState *in_bs,
                                        Error **errp)
 {
@@ -4241,11 +4295,13 @@ static BlockMeasureInfo *qcow2_measure(QemuOpts *opts, BlockDriverState *in_bs,
     uint64_t virtual_size; /* disk size as seen by guest */
     uint64_t refcount_bits;
     uint64_t l2_tables;
+    uint64_t luks_payload_size = 0;
     size_t cluster_size;
     int version;
     char *optstr;
     PreallocMode prealloc;
     bool has_backing_file;
+    bool has_luks;
 
     /* Parse image creation options */
     cluster_size = qcow2_opt_get_cluster_size_del(opts, &local_err);
@@ -4275,6 +4331,20 @@ static BlockMeasureInfo *qcow2_measure(QemuOpts *opts, BlockDriverState *in_bs,
     has_backing_file = !!optstr;
     g_free(optstr);
 
+    optstr = qemu_opt_get_del(opts, BLOCK_OPT_ENCRYPT_FORMAT);
+    has_luks = optstr && strcmp(optstr, "luks") == 0;
+    g_free(optstr);
+
+    if (has_luks) {
+        size_t headerlen;
+
+        if (!qcow2_measure_luks_headerlen(opts, &headerlen, &local_err)) {
+            goto err;
+        }
+
+        luks_payload_size = ROUND_UP(headerlen, cluster_size);
+    }
+
     virtual_size = qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0);
     virtual_size = ROUND_UP(virtual_size, cluster_size);
 
@@ -4345,7 +4415,7 @@ static BlockMeasureInfo *qcow2_measure(QemuOpts *opts, BlockDriverState *in_bs,
     info = g_new(BlockMeasureInfo, 1);
     info->fully_allocated =
         qcow2_calc_prealloc_size(virtual_size, cluster_size,
-                                 ctz32(refcount_bits));
+                                 ctz32(refcount_bits)) + luks_payload_size;
 
     /* Remove data clusters that are not required.  This overestimates the
      * required size because metadata needed for the fully allocated file is
-- 
2.20.1

[Qemu-devel] [PATCH v3 2/2] iotests: add LUKS payload overhead to 178 qemu-img measure test

[Stefan Hajnoczi]

The previous patch includes the LUKS payload overhead into the qemu-img
measure calculation for qcow2.  Update qemu-iotests 178 to exercise this
new code path.

Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 tests/qemu-iotests/178           |  8 ++++++++
 tests/qemu-iotests/178.out.qcow2 | 24 ++++++++++++++++++++++++
 2 files changed, 32 insertions(+)

diff --git a/tests/qemu-iotests/178 b/tests/qemu-iotests/178
index 3f4b4a4564..927bf06e4d 100755
--- a/tests/qemu-iotests/178
+++ b/tests/qemu-iotests/178
@@ -142,6 +142,14 @@ for ofmt in human json; do
             # The backing file doesn't need to exist :)
             $QEMU_IMG measure --output=$ofmt -o backing_file=x \
                               -f "$fmt" -O "$IMGFMT" "$TEST_IMG"
+
+            echo
+            echo "== $fmt input image and LUKS encryption =="
+            echo
+            $QEMU_IMG measure --output=$ofmt \
+                              --object secret,id=sec0,data=base \
+                              -o encrypt.format=luks,encrypt.key-secret=sec0,encrypt.iter-time=10 \
+                              -f "$fmt" -O "$IMGFMT" "$TEST_IMG"
         fi
 
         echo
diff --git a/tests/qemu-iotests/178.out.qcow2 b/tests/qemu-iotests/178.out.qcow2
index d42d4a4597..55a8dc926f 100644
--- a/tests/qemu-iotests/178.out.qcow2
+++ b/tests/qemu-iotests/178.out.qcow2
@@ -68,6 +68,11 @@ converted image file size in bytes: 458752
 required size: 1074135040
 fully allocated size: 1074135040
 
+== qcow2 input image and LUKS encryption ==
+
+required size: 2686976
+fully allocated size: 1076232192
+
 == qcow2 input image and preallocation (human) ==
 
 required size: 1074135040
@@ -114,6 +119,11 @@ converted image file size in bytes: 524288
 required size: 1074135040
 fully allocated size: 1074135040
 
+== raw input image and LUKS encryption ==
+
+required size: 2686976
+fully allocated size: 1076232192
+
 == raw input image and preallocation (human) ==
 
 required size: 1074135040
@@ -205,6 +215,13 @@ converted image file size in bytes: 458752
     "fully-allocated": 1074135040
 }
 
+== qcow2 input image and LUKS encryption ==
+
+{
+    "required": 2686976,
+    "fully-allocated": 1076232192
+}
+
 == qcow2 input image and preallocation (json) ==
 
 {
@@ -263,6 +280,13 @@ converted image file size in bytes: 524288
     "fully-allocated": 1074135040
 }
 
+== raw input image and LUKS encryption ==
+
+{
+    "required": 2686976,
+    "fully-allocated": 1076232192
+}
+
 == raw input image and preallocation (json) ==
 
 {
-- 
2.20.1

Re: [Qemu-devel] [PATCH v3 0/2] qcow2: include LUKS payload overhead in qemu-img measure

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 927 bytes --]

On 18.02.19 11:45, Stefan Hajnoczi wrote:
> v3:
>  * Use encrypt.iter-time=10 to speed up qemu-iotests 178 [Max]

Thanks!

> v2:
>  * Take into account all "encrypt." image creation options since the LUKS
>    payload size varies depending on the cipher [Max]
> 
> The LUKS payload has a significant size (>1 MB).  It must be included in the
> qemu-img measure calculation so we arrive at the correct minimum image size.
> 
> Stefan Hajnoczi (2):
>   qcow2: include LUKS payload overhead in qemu-img measure
>   iotests: add LUKS payload overhead to 178 qemu-img measure test
> 
>  block/qcow2.c                    | 72 +++++++++++++++++++++++++++++++-
>  tests/qemu-iotests/178           |  8 ++++
>  tests/qemu-iotests/178.out.qcow2 | 24 +++++++++++
>  3 files changed, 103 insertions(+), 1 deletion(-)

Applied to my block branch:

https://git.xanclic.moe/XanClic/qemu/commits/branch/block

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]