* [PULL 01/31] block/block-copy: block_copy_dirty_clusters: fix failure check
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 02/31] qcow2: Fix preallocation on images with unaligned sizes Max Reitz
` (31 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
ret may be > 0 on success path at this point. Fix assertion, which may
crash currently.
Fixes: 4ce5dd3e9b5ee0fac18625860eb3727399ee965e
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200526181347.489557-1-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/block-copy.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/block/block-copy.c b/block/block-copy.c
index bb8d0569f2..f7428a7c08 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -622,8 +622,10 @@ out:
* block_copy_task_run. If it fails, it means some task already failed
* for real reason, let's return first failure.
* Still, assert that we don't rewrite failure by success.
+ *
+ * Note: ret may be positive here because of block-status result.
*/
- assert(ret == 0 || aio_task_pool_status(aio) < 0);
+ assert(ret >= 0 || aio_task_pool_status(aio) < 0);
ret = aio_task_pool_status(aio);
aio_task_pool_free(aio);
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 02/31] qcow2: Fix preallocation on images with unaligned sizes
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
2020-07-06 10:04 ` [PULL 01/31] block/block-copy: block_copy_dirty_clusters: fix failure check Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 03/31] iotests: Make _filter_img_create more active Max Reitz
` (30 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Alberto Garcia <berto@igalia.com>
When resizing an image with qcow2_co_truncate() using the falloc or
full preallocation modes the code assumes that both the old and new
sizes are cluster-aligned.
There are two problems with this:
1) The calculation of how many clusters are involved does not always
get the right result.
Example: creating a 60KB image and resizing it (with
preallocation=full) to 80KB won't allocate the second cluster.
2) No copy-on-write is performed, so in the previous example if
there is a backing file then the first 60KB of the first cluster
won't be filled with data from the backing file.
This patch fixes both issues.
Signed-off-by: Alberto Garcia <berto@igalia.com>
Message-Id: <20200617140036.20311-1-berto@igalia.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/qcow2.c | 17 ++++++++++++++---
tests/qemu-iotests/125 | 24 ++++++++++++++++++++++++
tests/qemu-iotests/125.out | 9 +++++++++
3 files changed, 47 insertions(+), 3 deletions(-)
diff --git a/block/qcow2.c b/block/qcow2.c
index 0cd2e6757e..e20590c3b7 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -4239,8 +4239,8 @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset,
old_file_size = ROUND_UP(old_file_size, s->cluster_size);
}
- nb_new_data_clusters = DIV_ROUND_UP(offset - old_length,
- s->cluster_size);
+ nb_new_data_clusters = (ROUND_UP(offset, s->cluster_size) -
+ start_of_cluster(s, old_length)) >> s->cluster_bits;
/* This is an overestimation; we will not actually allocate space for
* these in the file but just make sure the new refcount structures are
@@ -4317,10 +4317,21 @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset,
int64_t nb_clusters = MIN(
nb_new_data_clusters,
s->l2_slice_size - offset_to_l2_slice_index(s, guest_offset));
- QCowL2Meta allocation = {
+ unsigned cow_start_length = offset_into_cluster(s, guest_offset);
+ QCowL2Meta allocation;
+ guest_offset = start_of_cluster(s, guest_offset);
+ allocation = (QCowL2Meta) {
.offset = guest_offset,
.alloc_offset = host_offset,
.nb_clusters = nb_clusters,
+ .cow_start = {
+ .offset = 0,
+ .nb_bytes = cow_start_length,
+ },
+ .cow_end = {
+ .offset = nb_clusters << s->cluster_bits,
+ .nb_bytes = 0,
+ },
};
qemu_co_queue_init(&allocation.dependent_requests);
diff --git a/tests/qemu-iotests/125 b/tests/qemu-iotests/125
index d510984045..7cb1c19730 100755
--- a/tests/qemu-iotests/125
+++ b/tests/qemu-iotests/125
@@ -164,6 +164,30 @@ for GROWTH_SIZE in 16 48 80; do
done
done
+# Test image resizing using preallocation and unaligned offsets
+$QEMU_IMG create -f raw "$TEST_IMG.base" 128k | _filter_img_create
+$QEMU_IO -c 'write -q -P 1 0 128k' -f raw "$TEST_IMG.base"
+for orig_size in 31k 33k; do
+ echo "--- Resizing image from $orig_size to 96k ---"
+ _make_test_img -F raw -b "$TEST_IMG.base" -o cluster_size=64k "$orig_size"
+ $QEMU_IMG resize -f "$IMGFMT" --preallocation=full "$TEST_IMG" 96k
+ # The first part of the image should contain data from the backing file
+ $QEMU_IO -c "read -q -P 1 0 ${orig_size}" "$TEST_IMG"
+ # The resized part of the image should contain zeroes
+ $QEMU_IO -c "read -q -P 0 ${orig_size} 63k" "$TEST_IMG"
+ # If the image does not have an external data file we can also verify its
+ # actual size. The resized image should have 7 clusters:
+ # header, L1 table, L2 table, refcount table, refcount block, 2 data clusters
+ if ! _get_data_file "$TEST_IMG" > /dev/null; then
+ expected_file_length=$((65536 * 7))
+ file_length=$(stat -c '%s' "$TEST_IMG_FILE")
+ if [ "$file_length" != "$expected_file_length" ]; then
+ echo "ERROR: file length $file_length (expected $expected_file_length)"
+ fi
+ fi
+ echo
+done
+
# success, all done
echo '*** done'
rm -f $seq.full
diff --git a/tests/qemu-iotests/125.out b/tests/qemu-iotests/125.out
index 596905f533..7f76f7af20 100644
--- a/tests/qemu-iotests/125.out
+++ b/tests/qemu-iotests/125.out
@@ -767,4 +767,13 @@ wrote 2048000/2048000 bytes at offset 0
wrote 81920/81920 bytes at offset 2048000
80 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=raw size=131072
+--- Resizing image from 31k to 96k ---
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=31744 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=raw
+Image resized.
+
+--- Resizing image from 33k to 96k ---
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33792 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=raw
+Image resized.
+
*** done
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 03/31] iotests: Make _filter_img_create more active
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
2020-07-06 10:04 ` [PULL 01/31] block/block-copy: block_copy_dirty_clusters: fix failure check Max Reitz
2020-07-06 10:04 ` [PULL 02/31] qcow2: Fix preallocation on images with unaligned sizes Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-10 15:48 ` John Snow
2020-07-06 10:04 ` [PULL 04/31] iotests: filter few more luks specific create options Max Reitz
` (29 subsequent siblings)
32 siblings, 1 reply; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
Right now, _filter_img_create just filters out everything that looks
format-dependent, and applies some filename filters. That means that we
have to add another filter line every time some format gets a new
creation option. This can be avoided by instead discarding everything
and just keeping what we know is format-independent (format, size,
backing file, encryption information[1], preallocation) or just
interesting to have in the reference output (external data file path).
Furthermore, we probably want to sort these options. Format drivers are
not required to define them in any specific order, so the output is
effectively random (although this has never bothered us until now). We
need a specific order for our reference outputs, though. Unfortunately,
just using a plain "sort" would change a lot of existing reference
outputs, so we have to pre-filter the option keys to keep our existing
order (fmt, size, backing*, data, encryption info, preallocation).
Finally, this makes it difficult for _filter_img_create to automagically
work for QMP output. Thus, this patch adds a separate
_filter_img_create_for_qmp function that echos every line verbatim that
does not start with "Formatting", and pipes those "Formatting" lines to
_filter_img_create.
[1] Actually, the only thing that is really important is whether
encryption is enabled or not. A patch by Maxim thus removes all
other "encrypt.*" options from the output:
https://lists.nongnu.org/archive/html/qemu-block/2020-06/msg00339.html
But that patch needs to come later so we can get away with changing
as few reference outputs in this patch here as possible.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200625125548.870061-2-mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
---
tests/qemu-iotests/112.out | 2 +-
tests/qemu-iotests/141 | 2 +-
tests/qemu-iotests/153 | 9 ++-
tests/qemu-iotests/common.filter | 109 ++++++++++++++++++++++++-------
4 files changed, 91 insertions(+), 31 deletions(-)
diff --git a/tests/qemu-iotests/112.out b/tests/qemu-iotests/112.out
index ae0318cabe..182655dbf6 100644
--- a/tests/qemu-iotests/112.out
+++ b/tests/qemu-iotests/112.out
@@ -5,7 +5,7 @@ QA output created by 112
qemu-img: TEST_DIR/t.IMGFMT: Refcount width must be a power of two and may not exceed 64 bits
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
qemu-img: TEST_DIR/t.IMGFMT: Refcount width must be a power of two and may not exceed 64 bits
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 refcount_bits=-1
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
qemu-img: TEST_DIR/t.IMGFMT: Refcount width must be a power of two and may not exceed 64 bits
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
qemu-img: TEST_DIR/t.IMGFMT: Refcount width must be a power of two and may not exceed 64 bits
diff --git a/tests/qemu-iotests/141 b/tests/qemu-iotests/141
index 5192d256e3..6d1b7b0d4c 100755
--- a/tests/qemu-iotests/141
+++ b/tests/qemu-iotests/141
@@ -68,7 +68,7 @@ test_blockjob()
_send_qemu_cmd $QEMU_HANDLE \
"$1" \
"$2" \
- | _filter_img_create | _filter_qmp_empty_return
+ | _filter_img_create_in_qmp | _filter_qmp_empty_return
# We want this to return an error because the block job is still running
_send_qemu_cmd $QEMU_HANDLE \
diff --git a/tests/qemu-iotests/153 b/tests/qemu-iotests/153
index cf961d3609..11e3d28841 100755
--- a/tests/qemu-iotests/153
+++ b/tests/qemu-iotests/153
@@ -167,11 +167,10 @@ done
echo
echo "== Creating ${TEST_IMG}.[abc] ==" | _filter_testdir
-(
- $QEMU_IMG create -f qcow2 "${TEST_IMG}.a" -b "${TEST_IMG}"
- $QEMU_IMG create -f qcow2 "${TEST_IMG}.b" -b "${TEST_IMG}"
- $QEMU_IMG create -f qcow2 "${TEST_IMG}.c" -b "${TEST_IMG}.b"
-) | _filter_img_create
+$QEMU_IMG create -f qcow2 "${TEST_IMG}.a" -b "${TEST_IMG}" | _filter_img_create
+$QEMU_IMG create -f qcow2 "${TEST_IMG}.b" -b "${TEST_IMG}" | _filter_img_create
+$QEMU_IMG create -f qcow2 "${TEST_IMG}.c" -b "${TEST_IMG}.b" \
+ | _filter_img_create
echo
echo "== Two devices sharing the same file in backing chain =="
diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
index 03e4f71808..f8cd80ff1f 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -122,38 +122,99 @@ _filter_actual_image_size()
# replace driver-specific options in the "Formatting..." line
_filter_img_create()
{
- data_file_filter=()
- if data_file=$(_get_data_file "$TEST_IMG"); then
- data_file_filter=(-e "s# data_file=$data_file##")
+ # Split the line into the pre-options part ($filename_part, which
+ # precedes ", fmt=") and the options part ($options, which starts
+ # with "fmt=")
+ # (And just echo everything before the first "^Formatting")
+ readarray formatting_line < <($SED -e 's/, fmt=/\n/')
+
+ filename_part=''
+ options=''
+ lines=${#formatting_line[@]}
+ for ((i = 0; i < $lines; i++)); do
+ line=${formatting_line[i]}
+ unset formatting_line[i]
+
+ filename_part="$filename_part$line"
+
+ if echo "$line" | grep -q '^Formatting'; then
+ next_i=$((i + 1))
+ if [ -n "${formatting_line[next_i]}" ]; then
+ options="fmt=${formatting_line[@]}"
+ fi
+ break
+ fi
+ done
+
+ # Set grep_data_file to '\|data_file' to keep it; make it empty
+ # to drop it.
+ # We want to drop it if it is part of the global $IMGOPTS, and we
+ # want to keep it otherwise (if the test specifically wants to
+ # test data files).
+ grep_data_file=(-e data_file)
+ if _get_data_file "$TEST_IMG" > /dev/null; then
+ grep_data_file=()
fi
- $SED "${data_file_filter[@]}" \
+ filename_filters=(
-e "s#$REMOTE_TEST_DIR#TEST_DIR#g" \
-e "s#$IMGPROTO:$TEST_DIR#TEST_DIR#g" \
-e "s#$TEST_DIR#TEST_DIR#g" \
-e "s#$SOCK_DIR#SOCK_DIR#g" \
-e "s#$IMGFMT#IMGFMT#g" \
-e 's#nbd:127.0.0.1:[0-9]\\+#TEST_DIR/t.IMGFMT#g' \
- -e 's#nbd+unix:///\??socket=SOCK_DIR/nbd#TEST_DIR/t.IMGFMT#g' \
- -e "s# encryption=off##g" \
- -e "s# cluster_size=[0-9]\\+##g" \
- -e "s# table_size=[0-9]\\+##g" \
- -e "s# compat=[^ ]*##g" \
- -e "s# compat6=\\(on\\|off\\)##g" \
- -e "s# static=\\(on\\|off\\)##g" \
- -e "s# zeroed_grain=\\(on\\|off\\)##g" \
- -e "s# subformat=[^ ]*##g" \
- -e "s# adapter_type=[^ ]*##g" \
- -e "s# hwversion=[^ ]*##g" \
- -e "s# lazy_refcounts=\\(on\\|off\\)##g" \
- -e "s# block_size=[0-9]\\+##g" \
- -e "s# block_state_zero=\\(on\\|off\\)##g" \
- -e "s# log_size=[0-9]\\+##g" \
- -e "s# refcount_bits=[0-9]\\+##g" \
- -e "s# key-secret=[a-zA-Z0-9]\\+##g" \
- -e "s# iter-time=[0-9]\\+##g" \
- -e "s# force_size=\\(on\\|off\\)##g" \
- -e "s# compression_type=[a-zA-Z0-9]\\+##g"
+ -e 's#nbd+unix:///\??socket=SOCK_DIR/nbd#TEST_DIR/t.IMGFMT#g'
+ )
+
+ filename_part=$(echo "$filename_part" | $SED "${filename_filters[@]}")
+
+ # Break the option line before each option (preserving pre-existing
+ # line breaks by replacing them by \0 and restoring them at the end),
+ # then filter out the options we want to keep and sort them according
+ # to some order that all block drivers used at the time of writing
+ # this function.
+ options=$(
+ echo "$options" \
+ | tr '\n' '\0' \
+ | $SED -e 's/ \([a-z0-9_.-]*\)=/\n\1=/g' \
+ | grep -a -e '^fmt' -e '^size' -e '^backing' -e '^preallocation' \
+ -e '^encrypt' "${grep_data_file[@]}" \
+ | $SED "${filename_filters[@]}" \
+ -e 's/^\(fmt\)/0-\1/' \
+ -e 's/^\(size\)/1-\1/' \
+ -e 's/^\(backing\)/2-\1/' \
+ -e 's/^\(data_file\)/3-\1/' \
+ -e 's/^\(encryption\)/4-\1/' \
+ -e 's/^\(encrypt\.format\)/5-\1/' \
+ -e 's/^\(encrypt\.key-secret\)/6-\1/' \
+ -e 's/^\(encrypt\.iter-time\)/7-\1/' \
+ -e 's/^\(preallocation\)/8-\1/' \
+ | sort \
+ | $SED -e 's/^[0-9]-//' \
+ | tr '\n\0' ' \n' \
+ | $SED -e 's/^ *$//' -e 's/ *$//'
+ )
+
+ if [ -n "$options" ]; then
+ echo "$filename_part, $options"
+ elif [ -n "$filename_part" ]; then
+ echo "$filename_part"
+ fi
+}
+
+# Filter the "Formatting..." line in QMP output (leaving the QMP output
+# untouched)
+# (In contrast to _filter_img_create(), this function does not support
+# multi-line Formatting output)
+_filter_img_create_in_qmp()
+{
+ while read -r line; do
+ if echo "$line" | grep -q '^Formatting'; then
+ echo "$line" | _filter_img_create
+ else
+ echo "$line"
+ fi
+ done
}
_filter_img_create_size()
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* Re: [PULL 03/31] iotests: Make _filter_img_create more active
2020-07-06 10:04 ` [PULL 03/31] iotests: Make _filter_img_create more active Max Reitz
@ 2020-07-10 15:48 ` John Snow
0 siblings, 0 replies; 45+ messages in thread
From: John Snow @ 2020-07-10 15:48 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: Kevin Wolf, Peter Maydell, Thomas Huth, Alex Bennée,
qemu-devel
On 7/6/20 6:04 AM, Max Reitz wrote:
> Right now, _filter_img_create just filters out everything that looks
> format-dependent, and applies some filename filters. That means that we
> have to add another filter line every time some format gets a new
> creation option. This can be avoided by instead discarding everything
> and just keeping what we know is format-independent (format, size,
> backing file, encryption information[1], preallocation) or just
> interesting to have in the reference output (external data file path).
>
> Furthermore, we probably want to sort these options. Format drivers are
> not required to define them in any specific order, so the output is
> effectively random (although this has never bothered us until now). We
> need a specific order for our reference outputs, though. Unfortunately,
> just using a plain "sort" would change a lot of existing reference
> outputs, so we have to pre-filter the option keys to keep our existing
> order (fmt, size, backing*, data, encryption info, preallocation).
>
> Finally, this makes it difficult for _filter_img_create to automagically
> work for QMP output. Thus, this patch adds a separate
> _filter_img_create_for_qmp function that echos every line verbatim that
> does not start with "Formatting", and pipes those "Formatting" lines to
> _filter_img_create.
>
> [1] Actually, the only thing that is really important is whether
> encryption is enabled or not. A patch by Maxim thus removes all
> other "encrypt.*" options from the output:
> https://lists.nongnu.org/archive/html/qemu-block/2020-06/msg00339.html
> But that patch needs to come later so we can get away with changing
> as few reference outputs in this patch here as possible.
>
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> Message-Id: <20200625125548.870061-2-mreitz@redhat.com>
> Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Hi, I am seeing breakage on travis-ci;
https://travis-ci.org/github/jnsnow/qemu/jobs/706915953#L2236
(Disclaimer: Has my python patches applied)
There's some ambiguity perhaps over the order required by data_file /
data_file_raw.
Also in the IRC channel, pm215 notes that there is a breakage on OSX as
'readarray' is a bash 4.0 feature, and OSX has only bash 3.2:
11:34 < stsquad> MacOSX iotest failures
11:34 < stsquad> Not run: 005 013 018 019 024 034 038 039 041 048 060
061 074 079 080 086 097 099 108 137 138 140 141 150 154 161 172 176 179 181
184 203 220 226 229 244 249 251 252 259 265 267 287 290
292z
11:34 < stsquad> Failures: 001 002 003 004 007 008 009 010 011 012 017
020 021 022 025 027 029 031 032 033 035 036 037 042 043 046 047 050 052
053 054 062 063 066 069 071 072 073 089 090 098 103 104
105 107 110 111 114 117 120 126 127 133 134 156 158 159 170 174 177 186
187 190 191 192 195 214 217 268
11:34 < stsquad> Failed 69 of 75 iotests
11:34 < stsquad> that sounds like a bash-ism has snuck in?
11:36 < pm215> stsquad: readarray is in bash 4, osx system bash is 3.2
11:36 < pm215> dunno why that didn't fail for my own osx box
11:38 < th_huth> pm215: does "make check" run the iotests on your osx
box at all? Maybe it is missing gnu-sed ?
11:39 < pm215> th_huth: ah, it doesn't: "GNU sed not available ==> Not
running the qemu-iotests"
11:39 < th_huth> too bad :-(
> ---
> tests/qemu-iotests/112.out | 2 +-
> tests/qemu-iotests/141 | 2 +-
> tests/qemu-iotests/153 | 9 ++-
> tests/qemu-iotests/common.filter | 109 ++++++++++++++++++++++++-------
> 4 files changed, 91 insertions(+), 31 deletions(-)
>
> diff --git a/tests/qemu-iotests/112.out b/tests/qemu-iotests/112.out
> index ae0318cabe..182655dbf6 100644
> --- a/tests/qemu-iotests/112.out
> +++ b/tests/qemu-iotests/112.out
> @@ -5,7 +5,7 @@ QA output created by 112
> qemu-img: TEST_DIR/t.IMGFMT: Refcount width must be a power of two and may not exceed 64 bits
> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
> qemu-img: TEST_DIR/t.IMGFMT: Refcount width must be a power of two and may not exceed 64 bits
> -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 refcount_bits=-1
> +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
> qemu-img: TEST_DIR/t.IMGFMT: Refcount width must be a power of two and may not exceed 64 bits
> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
> qemu-img: TEST_DIR/t.IMGFMT: Refcount width must be a power of two and may not exceed 64 bits
> diff --git a/tests/qemu-iotests/141 b/tests/qemu-iotests/141
> index 5192d256e3..6d1b7b0d4c 100755
> --- a/tests/qemu-iotests/141
> +++ b/tests/qemu-iotests/141
> @@ -68,7 +68,7 @@ test_blockjob()
> _send_qemu_cmd $QEMU_HANDLE \
> "$1" \
> "$2" \
> - | _filter_img_create | _filter_qmp_empty_return
> + | _filter_img_create_in_qmp | _filter_qmp_empty_return
>
> # We want this to return an error because the block job is still running
> _send_qemu_cmd $QEMU_HANDLE \
> diff --git a/tests/qemu-iotests/153 b/tests/qemu-iotests/153
> index cf961d3609..11e3d28841 100755
> --- a/tests/qemu-iotests/153
> +++ b/tests/qemu-iotests/153
> @@ -167,11 +167,10 @@ done
>
> echo
> echo "== Creating ${TEST_IMG}.[abc] ==" | _filter_testdir
> -(
> - $QEMU_IMG create -f qcow2 "${TEST_IMG}.a" -b "${TEST_IMG}"
> - $QEMU_IMG create -f qcow2 "${TEST_IMG}.b" -b "${TEST_IMG}"
> - $QEMU_IMG create -f qcow2 "${TEST_IMG}.c" -b "${TEST_IMG}.b"
> -) | _filter_img_create
> +$QEMU_IMG create -f qcow2 "${TEST_IMG}.a" -b "${TEST_IMG}" | _filter_img_create
> +$QEMU_IMG create -f qcow2 "${TEST_IMG}.b" -b "${TEST_IMG}" | _filter_img_create
> +$QEMU_IMG create -f qcow2 "${TEST_IMG}.c" -b "${TEST_IMG}.b" \
> + | _filter_img_create
>
> echo
> echo "== Two devices sharing the same file in backing chain =="
> diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
> index 03e4f71808..f8cd80ff1f 100644
> --- a/tests/qemu-iotests/common.filter
> +++ b/tests/qemu-iotests/common.filter
> @@ -122,38 +122,99 @@ _filter_actual_image_size()
> # replace driver-specific options in the "Formatting..." line
> _filter_img_create()
> {
> - data_file_filter=()
> - if data_file=$(_get_data_file "$TEST_IMG"); then
> - data_file_filter=(-e "s# data_file=$data_file##")
> + # Split the line into the pre-options part ($filename_part, which
> + # precedes ", fmt=") and the options part ($options, which starts
> + # with "fmt=")
> + # (And just echo everything before the first "^Formatting")
> + readarray formatting_line < <($SED -e 's/, fmt=/\n/')
> +
> + filename_part=''
> + options=''
> + lines=${#formatting_line[@]}
> + for ((i = 0; i < $lines; i++)); do
> + line=${formatting_line[i]}
> + unset formatting_line[i]
> +
> + filename_part="$filename_part$line"
> +
> + if echo "$line" | grep -q '^Formatting'; then
> + next_i=$((i + 1))
> + if [ -n "${formatting_line[next_i]}" ]; then
> + options="fmt=${formatting_line[@]}"
> + fi
> + break
> + fi
> + done
> +
> + # Set grep_data_file to '\|data_file' to keep it; make it empty
> + # to drop it.
> + # We want to drop it if it is part of the global $IMGOPTS, and we
> + # want to keep it otherwise (if the test specifically wants to
> + # test data files).
> + grep_data_file=(-e data_file)
> + if _get_data_file "$TEST_IMG" > /dev/null; then
> + grep_data_file=()
> fi
>
> - $SED "${data_file_filter[@]}" \
> + filename_filters=(
> -e "s#$REMOTE_TEST_DIR#TEST_DIR#g" \
> -e "s#$IMGPROTO:$TEST_DIR#TEST_DIR#g" \
> -e "s#$TEST_DIR#TEST_DIR#g" \
> -e "s#$SOCK_DIR#SOCK_DIR#g" \
> -e "s#$IMGFMT#IMGFMT#g" \
> -e 's#nbd:127.0.0.1:[0-9]\\+#TEST_DIR/t.IMGFMT#g' \
> - -e 's#nbd+unix:///\??socket=SOCK_DIR/nbd#TEST_DIR/t.IMGFMT#g' \
> - -e "s# encryption=off##g" \
> - -e "s# cluster_size=[0-9]\\+##g" \
> - -e "s# table_size=[0-9]\\+##g" \
> - -e "s# compat=[^ ]*##g" \
> - -e "s# compat6=\\(on\\|off\\)##g" \
> - -e "s# static=\\(on\\|off\\)##g" \
> - -e "s# zeroed_grain=\\(on\\|off\\)##g" \
> - -e "s# subformat=[^ ]*##g" \
> - -e "s# adapter_type=[^ ]*##g" \
> - -e "s# hwversion=[^ ]*##g" \
> - -e "s# lazy_refcounts=\\(on\\|off\\)##g" \
> - -e "s# block_size=[0-9]\\+##g" \
> - -e "s# block_state_zero=\\(on\\|off\\)##g" \
> - -e "s# log_size=[0-9]\\+##g" \
> - -e "s# refcount_bits=[0-9]\\+##g" \
> - -e "s# key-secret=[a-zA-Z0-9]\\+##g" \
> - -e "s# iter-time=[0-9]\\+##g" \
> - -e "s# force_size=\\(on\\|off\\)##g" \
> - -e "s# compression_type=[a-zA-Z0-9]\\+##g"
> + -e 's#nbd+unix:///\??socket=SOCK_DIR/nbd#TEST_DIR/t.IMGFMT#g'
> + )
> +
> + filename_part=$(echo "$filename_part" | $SED "${filename_filters[@]}")
> +
> + # Break the option line before each option (preserving pre-existing
> + # line breaks by replacing them by \0 and restoring them at the end),
> + # then filter out the options we want to keep and sort them according
> + # to some order that all block drivers used at the time of writing
> + # this function.
> + options=$(
> + echo "$options" \
> + | tr '\n' '\0' \
> + | $SED -e 's/ \([a-z0-9_.-]*\)=/\n\1=/g' \
> + | grep -a -e '^fmt' -e '^size' -e '^backing' -e '^preallocation' \
> + -e '^encrypt' "${grep_data_file[@]}" \
> + | $SED "${filename_filters[@]}" \
> + -e 's/^\(fmt\)/0-\1/' \
> + -e 's/^\(size\)/1-\1/' \
> + -e 's/^\(backing\)/2-\1/' \
> + -e 's/^\(data_file\)/3-\1/' \
> + -e 's/^\(encryption\)/4-\1/' \
> + -e 's/^\(encrypt\.format\)/5-\1/' \
> + -e 's/^\(encrypt\.key-secret\)/6-\1/' \
> + -e 's/^\(encrypt\.iter-time\)/7-\1/' \
> + -e 's/^\(preallocation\)/8-\1/' \
> + | sort \
> + | $SED -e 's/^[0-9]-//' \
> + | tr '\n\0' ' \n' \
> + | $SED -e 's/^ *$//' -e 's/ *$//'
> + )
> +
> + if [ -n "$options" ]; then
> + echo "$filename_part, $options"
> + elif [ -n "$filename_part" ]; then
> + echo "$filename_part"
> + fi
> +}
> +
> +# Filter the "Formatting..." line in QMP output (leaving the QMP output
> +# untouched)
> +# (In contrast to _filter_img_create(), this function does not support
> +# multi-line Formatting output)
> +_filter_img_create_in_qmp()
> +{
> + while read -r line; do
> + if echo "$line" | grep -q '^Formatting'; then
> + echo "$line" | _filter_img_create
> + else
> + echo "$line"
> + fi
> + done
> }
>
> _filter_img_create_size()
>
--
—js
^ permalink raw reply [flat|nested] 45+ messages in thread
* [PULL 04/31] iotests: filter few more luks specific create options
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (2 preceding siblings ...)
2020-07-06 10:04 ` [PULL 03/31] iotests: Make _filter_img_create more active Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 05/31] iotests/common.rc: Add _require_working_luks Max Reitz
` (28 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Maxim Levitsky <mlevitsk@redhat.com>
This allows more tests to be able to have same output on both qcow2 luks encrypted images
and raw luks images
Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-Id: <20200625125548.870061-3-mreitz@redhat.com>
---
tests/qemu-iotests/087.out | 6 +++---
tests/qemu-iotests/134.out | 2 +-
tests/qemu-iotests/158.out | 4 ++--
tests/qemu-iotests/188.out | 2 +-
tests/qemu-iotests/189.out | 4 ++--
tests/qemu-iotests/198.out | 4 ++--
tests/qemu-iotests/263.out | 4 ++--
tests/qemu-iotests/284.out | 6 +++---
tests/qemu-iotests/common.filter | 5 +----
9 files changed, 17 insertions(+), 20 deletions(-)
diff --git a/tests/qemu-iotests/087.out b/tests/qemu-iotests/087.out
index 2d92ea847b..b61ba638af 100644
--- a/tests/qemu-iotests/087.out
+++ b/tests/qemu-iotests/087.out
@@ -34,7 +34,7 @@ QMP_VERSION
=== Encrypted image QCow ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on
Testing:
QMP_VERSION
{"return": {}}
@@ -46,7 +46,7 @@ QMP_VERSION
=== Encrypted image LUKS ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encrypt.format=luks encrypt.key-secret=sec0
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
Testing:
QMP_VERSION
{"return": {}}
@@ -58,7 +58,7 @@ QMP_VERSION
=== Missing driver ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on
Testing: -S
QMP_VERSION
{"return": {}}
diff --git a/tests/qemu-iotests/134.out b/tests/qemu-iotests/134.out
index 09d46f6b17..4abc5b5f7d 100644
--- a/tests/qemu-iotests/134.out
+++ b/tests/qemu-iotests/134.out
@@ -1,5 +1,5 @@
QA output created by 134
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on
== reading whole image ==
read 134217728/134217728 bytes at offset 0
diff --git a/tests/qemu-iotests/158.out b/tests/qemu-iotests/158.out
index 6def216e55..f28a17626b 100644
--- a/tests/qemu-iotests/158.out
+++ b/tests/qemu-iotests/158.out
@@ -1,6 +1,6 @@
QA output created by 158
== create base ==
-Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 encryption=on
== writing whole image ==
wrote 134217728/134217728 bytes at offset 0
@@ -10,7 +10,7 @@ wrote 134217728/134217728 bytes at offset 0
read 134217728/134217728 bytes at offset 0
128 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
== create overlay ==
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base encryption=on encrypt.key-secret=sec0
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base encryption=on
== writing part of a cluster ==
wrote 1024/1024 bytes at offset 0
diff --git a/tests/qemu-iotests/188.out b/tests/qemu-iotests/188.out
index c568ef3701..5426861b18 100644
--- a/tests/qemu-iotests/188.out
+++ b/tests/qemu-iotests/188.out
@@ -1,5 +1,5 @@
QA output created by 188
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216
== reading whole image ==
read 16777216/16777216 bytes at offset 0
diff --git a/tests/qemu-iotests/189.out b/tests/qemu-iotests/189.out
index a0b7c9c24c..bc213cbe14 100644
--- a/tests/qemu-iotests/189.out
+++ b/tests/qemu-iotests/189.out
@@ -1,6 +1,6 @@
QA output created by 189
== create base ==
-Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=16777216 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=16777216
== writing whole image ==
wrote 16777216/16777216 bytes at offset 0
@@ -10,7 +10,7 @@ wrote 16777216/16777216 bytes at offset 0
read 16777216/16777216 bytes at offset 0
16 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
== create overlay ==
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 backing_file=TEST_DIR/t.IMGFMT.base encrypt.format=luks encrypt.key-secret=sec1 encrypt.iter-time=10
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 backing_file=TEST_DIR/t.IMGFMT.base
== writing part of a cluster ==
wrote 1024/1024 bytes at offset 0
diff --git a/tests/qemu-iotests/198.out b/tests/qemu-iotests/198.out
index 6280ae6eed..4b800e70db 100644
--- a/tests/qemu-iotests/198.out
+++ b/tests/qemu-iotests/198.out
@@ -1,12 +1,12 @@
QA output created by 198
== create base ==
-Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=16777216 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=16777216
== writing whole image base ==
wrote 16777216/16777216 bytes at offset 0
16 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
== create overlay ==
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 backing_file=TEST_DIR/t.IMGFMT.base encrypt.format=luks encrypt.key-secret=sec1 encrypt.iter-time=10
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 backing_file=TEST_DIR/t.IMGFMT.base
== writing whole image layer ==
wrote 16777216/16777216 bytes at offset 0
diff --git a/tests/qemu-iotests/263.out b/tests/qemu-iotests/263.out
index 0c982c55cb..54bfbeeff8 100644
--- a/tests/qemu-iotests/263.out
+++ b/tests/qemu-iotests/263.out
@@ -2,7 +2,7 @@ QA output created by 263
testing LUKS qcow2 encryption
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
== reading the whole image ==
read 1048576/1048576 bytes at offset 0
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -21,7 +21,7 @@ read 982528/982528 bytes at offset 66048
testing legacy AES qcow2 encryption
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=aes encrypt.key-secret=sec0
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
== reading the whole image ==
read 1048576/1048576 bytes at offset 0
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
diff --git a/tests/qemu-iotests/284.out b/tests/qemu-iotests/284.out
index 48216f5742..a929239302 100644
--- a/tests/qemu-iotests/284.out
+++ b/tests/qemu-iotests/284.out
@@ -2,7 +2,7 @@ QA output created by 284
testing LUKS qcow2 encryption
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
== cluster size 512
== checking image refcounts ==
@@ -21,7 +21,7 @@ wrote 1/1 bytes at offset 512
== rechecking image refcounts ==
No errors were found on the image.
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
== cluster size 2048
== checking image refcounts ==
@@ -40,7 +40,7 @@ wrote 1/1 bytes at offset 2048
== rechecking image refcounts ==
No errors were found on the image.
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
== cluster size 32768
== checking image refcounts ==
diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
index f8cd80ff1f..d967adc59a 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -178,16 +178,13 @@ _filter_img_create()
| tr '\n' '\0' \
| $SED -e 's/ \([a-z0-9_.-]*\)=/\n\1=/g' \
| grep -a -e '^fmt' -e '^size' -e '^backing' -e '^preallocation' \
- -e '^encrypt' "${grep_data_file[@]}" \
+ -e '^encryption' "${grep_data_file[@]}" \
| $SED "${filename_filters[@]}" \
-e 's/^\(fmt\)/0-\1/' \
-e 's/^\(size\)/1-\1/' \
-e 's/^\(backing\)/2-\1/' \
-e 's/^\(data_file\)/3-\1/' \
-e 's/^\(encryption\)/4-\1/' \
- -e 's/^\(encrypt\.format\)/5-\1/' \
- -e 's/^\(encrypt\.key-secret\)/6-\1/' \
- -e 's/^\(encrypt\.iter-time\)/7-\1/' \
-e 's/^\(preallocation\)/8-\1/' \
| sort \
| $SED -e 's/^[0-9]-//' \
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 05/31] iotests/common.rc: Add _require_working_luks
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (3 preceding siblings ...)
2020-07-06 10:04 ` [PULL 04/31] iotests: filter few more luks specific create options Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 06/31] iotests.py: Add qemu_img_pipe_and_status() Max Reitz
` (27 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
That the luks driver is present is little indication on whether it is
actually working. Without the crypto libraries linked in, it does not
work. So add this function, which tries to create a luks image to see
whether that actually works.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200625125548.870061-4-mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
---
tests/qemu-iotests/common.rc | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)
diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index ba912555ca..f3667f48ab 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -740,6 +740,33 @@ _unsupported_imgopts()
done
}
+# Caution: Overwrites $TEST_DIR/t.luks
+_require_working_luks()
+{
+ file="$TEST_DIR/t.luks"
+
+ output=$(
+ $QEMU_IMG create -f luks \
+ --object secret,id=sec0,data=hunter0 \
+ -o key-secret=sec0 \
+ -o iter-time=10 \
+ "$file" \
+ 1M \
+ 2>&1
+ )
+ status=$?
+
+ IMGFMT='luks' _rm_test_img "$file"
+
+ if [ $status != 0 ]; then
+ reason=$(echo "$output" | grep "$file:" | $SED -e "s#.*$file: *##")
+ if [ -z "$reason" ]; then
+ reason="Failed to create a LUKS image"
+ fi
+ _notrun "$reason"
+ fi
+}
+
# this test requires that a specified command (executable) exists
#
_require_command()
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 06/31] iotests.py: Add qemu_img_pipe_and_status()
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (4 preceding siblings ...)
2020-07-06 10:04 ` [PULL 05/31] iotests/common.rc: Add _require_working_luks Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 07/31] iotests.py: Add (verify|has)_working_luks() Max Reitz
` (26 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
This function will be used by the next patch, which intends to check
both the exit code and qemu-img's output.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200625125548.870061-5-mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
[mreitz: Rebased on 49438972b8c2e]
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
tests/qemu-iotests/iotests.py | 40 +++++++++++++++++------------------
1 file changed, 20 insertions(+), 20 deletions(-)
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index ef739dd1e3..423974d3f6 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -29,7 +29,7 @@ import struct
import subprocess
import sys
from typing import (Any, Callable, Dict, Iterable,
- List, Optional, Sequence, TypeVar)
+ List, Optional, Sequence, Tuple, TypeVar)
import unittest
# pylint: disable=import-error, wrong-import-position
@@ -90,15 +90,24 @@ luks_default_secret_object = 'secret,id=keysec0,data=' + \
luks_default_key_secret_opt = 'key-secret=keysec0'
-def qemu_img(*args):
- '''Run qemu-img and return the exit code'''
- devnull = open('/dev/null', 'r+')
- exitcode = subprocess.call(qemu_img_args + list(args),
- stdin=devnull, stdout=devnull)
- if exitcode < 0:
+def qemu_img_pipe_and_status(*args: str) -> Tuple[str, int]:
+ """
+ Run qemu-img and return both its output and its exit code
+ """
+ subp = subprocess.Popen(qemu_img_args + list(args),
+ stdout=subprocess.PIPE,
+ stderr=subprocess.STDOUT,
+ universal_newlines=True)
+ output = subp.communicate()[0]
+ if subp.returncode < 0:
sys.stderr.write('qemu-img received signal %i: %s\n'
- % (-exitcode, ' '.join(qemu_img_args + list(args))))
- return exitcode
+ % (-subp.returncode,
+ ' '.join(qemu_img_args + list(args))))
+ return (output, subp.returncode)
+
+def qemu_img(*args: str) -> int:
+ '''Run qemu-img and return the exit code'''
+ return qemu_img_pipe_and_status(*args)[1]
def ordered_qmp(qmsg, conv_keys=True):
# Dictionaries are not ordered prior to 3.6, therefore:
@@ -140,18 +149,9 @@ def qemu_img_verbose(*args):
% (-exitcode, ' '.join(qemu_img_args + list(args))))
return exitcode
-def qemu_img_pipe(*args):
+def qemu_img_pipe(*args: str) -> str:
'''Run qemu-img and return its output'''
- subp = subprocess.Popen(qemu_img_args + list(args),
- stdout=subprocess.PIPE,
- stderr=subprocess.STDOUT,
- universal_newlines=True)
- output = subp.communicate()[0]
- if subp.returncode < 0:
- sys.stderr.write('qemu-img received signal %i: %s\n'
- % (-subp.returncode,
- ' '.join(qemu_img_args + list(args))))
- return output
+ return qemu_img_pipe_and_status(*args)[0]
def qemu_img_log(*args):
result = qemu_img_pipe(*args)
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 07/31] iotests.py: Add (verify|has)_working_luks()
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (5 preceding siblings ...)
2020-07-06 10:04 ` [PULL 06/31] iotests.py: Add qemu_img_pipe_and_status() Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 08/31] iotests: Check whether luks works Max Reitz
` (25 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
Similar to _require_working_luks for bash tests, these functions can be
used to check whether our luks driver can actually create images.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200625125548.870061-6-mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
---
tests/qemu-iotests/iotests.py | 39 +++++++++++++++++++++++++++++++++++
1 file changed, 39 insertions(+)
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 423974d3f6..1d8a45c02e 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -1052,6 +1052,45 @@ def verify_quorum():
if not supports_quorum():
notrun('quorum support missing')
+def has_working_luks() -> Tuple[bool, str]:
+ """
+ Check whether our LUKS driver can actually create images
+ (this extends to LUKS encryption for qcow2).
+
+ If not, return the reason why.
+ """
+
+ img_file = f'{test_dir}/luks-test.luks'
+ (output, status) = \
+ qemu_img_pipe_and_status('create', '-f', 'luks',
+ '--object', luks_default_secret_object,
+ '-o', luks_default_key_secret_opt,
+ '-o', 'iter-time=10',
+ img_file, '1G')
+ try:
+ os.remove(img_file)
+ except OSError:
+ pass
+
+ if status != 0:
+ reason = output
+ for line in output.splitlines():
+ if img_file + ':' in line:
+ reason = line.split(img_file + ':', 1)[1].strip()
+ break
+
+ return (False, reason)
+ else:
+ return (True, '')
+
+def verify_working_luks():
+ """
+ Skip test suite if LUKS does not work
+ """
+ (working, reason) = has_working_luks()
+ if not working:
+ notrun(reason)
+
def qemu_pipe(*args):
"""
Run qemu with an option to print something and exit (e.g. a help option).
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 08/31] iotests: Check whether luks works
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (6 preceding siblings ...)
2020-07-06 10:04 ` [PULL 07/31] iotests.py: Add (verify|has)_working_luks() Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 09/31] qcrypto/core: add generic infrastructure for crypto options amendment Max Reitz
` (24 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
Whenever running an iotest for the luks format, we should check whether
luks actually really works.
Tests that try to create luks-encrypted qcow2 images should do the same.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200625125548.870061-7-mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
---
tests/qemu-iotests/087 | 1 +
tests/qemu-iotests/178 | 1 +
tests/qemu-iotests/188 | 1 +
tests/qemu-iotests/189 | 1 +
tests/qemu-iotests/198 | 1 +
tests/qemu-iotests/206 | 1 +
tests/qemu-iotests/263 | 1 +
tests/qemu-iotests/284 | 1 +
tests/qemu-iotests/common.rc | 3 +++
tests/qemu-iotests/iotests.py | 5 +++++
10 files changed, 16 insertions(+)
diff --git a/tests/qemu-iotests/087 b/tests/qemu-iotests/087
index bdfdad3454..678e748c58 100755
--- a/tests/qemu-iotests/087
+++ b/tests/qemu-iotests/087
@@ -39,6 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
_supported_fmt qcow2
_supported_proto file
_supported_os Linux
+_require_working_luks
do_run_qemu()
{
diff --git a/tests/qemu-iotests/178 b/tests/qemu-iotests/178
index 7cf0e27154..f09b27caac 100755
--- a/tests/qemu-iotests/178
+++ b/tests/qemu-iotests/178
@@ -41,6 +41,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
_supported_fmt raw qcow2
_supported_proto file
_supported_os Linux
+_require_working_luks
echo "== Input validation =="
echo
diff --git a/tests/qemu-iotests/188 b/tests/qemu-iotests/188
index 09b9b6083a..13b225fded 100755
--- a/tests/qemu-iotests/188
+++ b/tests/qemu-iotests/188
@@ -39,6 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
_supported_fmt qcow2
_supported_proto generic
_supported_os Linux
+_require_working_luks
size=16M
diff --git a/tests/qemu-iotests/189 b/tests/qemu-iotests/189
index c9ce9d3bed..e6a84b8a3b 100755
--- a/tests/qemu-iotests/189
+++ b/tests/qemu-iotests/189
@@ -39,6 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
_supported_fmt qcow2
_supported_proto generic
_supported_os Linux
+_require_working_luks
size=16M
diff --git a/tests/qemu-iotests/198 b/tests/qemu-iotests/198
index fb0d5a29d3..aeb059d5ea 100755
--- a/tests/qemu-iotests/198
+++ b/tests/qemu-iotests/198
@@ -39,6 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
_supported_fmt qcow2
_supported_proto generic
_supported_os Linux
+_require_working_luks
size=16M
diff --git a/tests/qemu-iotests/206 b/tests/qemu-iotests/206
index f42432a838..11bc51f256 100755
--- a/tests/qemu-iotests/206
+++ b/tests/qemu-iotests/206
@@ -24,6 +24,7 @@ import iotests
from iotests import imgfmt
iotests.script_initialize(supported_fmts=['qcow2'])
+iotests.verify_working_luks()
with iotests.FilePath('t.qcow2') as disk_path, \
iotests.FilePath('t.qcow2.base') as backing_path, \
diff --git a/tests/qemu-iotests/263 b/tests/qemu-iotests/263
index d2c030fae9..f598a12899 100755
--- a/tests/qemu-iotests/263
+++ b/tests/qemu-iotests/263
@@ -40,6 +40,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
_supported_fmt qcow2
_supported_proto generic
_supported_os Linux
+_require_working_luks
size=1M
diff --git a/tests/qemu-iotests/284 b/tests/qemu-iotests/284
index 071e89b33e..9f6c29a79c 100755
--- a/tests/qemu-iotests/284
+++ b/tests/qemu-iotests/284
@@ -39,6 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
_supported_fmt qcow2
_supported_proto generic
_supported_os Linux
+_require_working_luks
size=1M
diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index f3667f48ab..7ac46edc1f 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -605,6 +605,9 @@ _supported_fmt()
# setting IMGFMT_GENERIC to false.
for f; do
if [ "$f" = "$IMGFMT" -o "$f" = "generic" -a "$IMGFMT_GENERIC" = "true" ]; then
+ if [ "$IMGFMT" = "luks" ]; then
+ _require_working_luks
+ fi
return
fi
done
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 1d8a45c02e..f1e0733dda 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -1010,12 +1010,17 @@ def _verify_image_format(supported_fmts: Sequence[str] = (),
# similar to
# _supported_fmt generic
# for bash tests
+ if imgfmt == 'luks':
+ verify_working_luks()
return
not_sup = supported_fmts and (imgfmt not in supported_fmts)
if not_sup or (imgfmt in unsupported_fmts):
notrun('not suitable for this image format: %s' % imgfmt)
+ if imgfmt == 'luks':
+ verify_working_luks()
+
def _verify_protocol(supported: Sequence[str] = (),
unsupported: Sequence[str] = ()) -> None:
assert not (supported and unsupported)
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 09/31] qcrypto/core: add generic infrastructure for crypto options amendment
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (7 preceding siblings ...)
2020-07-06 10:04 ` [PULL 08/31] iotests: Check whether luks works Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 10/31] qcrypto/luks: implement encryption key management Max Reitz
` (23 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Maxim Levitsky <mlevitsk@redhat.com>
This will be used first to implement luks keyslot management.
block_crypto_amend_opts_init will be used to convert
qemu-img cmdline to QCryptoBlockAmendOptions
Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200608094030.670121-2-mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
qapi/crypto.json | 16 ++++++++++++++++
block/crypto.h | 3 +++
crypto/blockpriv.h | 8 ++++++++
include/crypto/block.h | 22 ++++++++++++++++++++++
block/crypto.c | 17 +++++++++++++++++
crypto/block.c | 29 +++++++++++++++++++++++++++++
6 files changed, 95 insertions(+)
diff --git a/qapi/crypto.json b/qapi/crypto.json
index b2a4cff683..aeb6c7ef7b 100644
--- a/qapi/crypto.json
+++ b/qapi/crypto.json
@@ -309,3 +309,19 @@
'base': 'QCryptoBlockInfoBase',
'discriminator': 'format',
'data': { 'luks': 'QCryptoBlockInfoLUKS' } }
+
+
+
+##
+# @QCryptoBlockAmendOptions:
+#
+# The options that are available for all encryption formats
+# when amending encryption settings
+#
+# Since: 5.1
+##
+{ 'union': 'QCryptoBlockAmendOptions',
+ 'base': 'QCryptoBlockOptionsBase',
+ 'discriminator': 'format',
+ 'data': {
+ } }
diff --git a/block/crypto.h b/block/crypto.h
index b935695e79..06e044c9be 100644
--- a/block/crypto.h
+++ b/block/crypto.h
@@ -91,6 +91,9 @@
QCryptoBlockCreateOptions *
block_crypto_create_opts_init(QDict *opts, Error **errp);
+QCryptoBlockAmendOptions *
+block_crypto_amend_opts_init(QDict *opts, Error **errp);
+
QCryptoBlockOpenOptions *
block_crypto_open_opts_init(QDict *opts, Error **errp);
diff --git a/crypto/blockpriv.h b/crypto/blockpriv.h
index 71c59cb542..3c7ccea504 100644
--- a/crypto/blockpriv.h
+++ b/crypto/blockpriv.h
@@ -62,6 +62,14 @@ struct QCryptoBlockDriver {
void *opaque,
Error **errp);
+ int (*amend)(QCryptoBlock *block,
+ QCryptoBlockReadFunc readfunc,
+ QCryptoBlockWriteFunc writefunc,
+ void *opaque,
+ QCryptoBlockAmendOptions *options,
+ bool force,
+ Error **errp);
+
int (*get_info)(QCryptoBlock *block,
QCryptoBlockInfo *info,
Error **errp);
diff --git a/include/crypto/block.h b/include/crypto/block.h
index c77ccaf9c0..d274819791 100644
--- a/include/crypto/block.h
+++ b/include/crypto/block.h
@@ -144,6 +144,28 @@ QCryptoBlock *qcrypto_block_create(QCryptoBlockCreateOptions *options,
void *opaque,
Error **errp);
+/**
+ * qcrypto_block_amend_options:
+ * @block: the block encryption object
+ *
+ * @readfunc: callback for reading data from the volume header
+ * @writefunc: callback for writing data to the volume header
+ * @opaque: data to pass to @readfunc and @writefunc
+ * @options: the new/amended encryption options
+ * @force: hint for the driver to allow unsafe operation
+ * @errp: error pointer
+ *
+ * Changes the crypto options of the encryption format
+ *
+ */
+int qcrypto_block_amend_options(QCryptoBlock *block,
+ QCryptoBlockReadFunc readfunc,
+ QCryptoBlockWriteFunc writefunc,
+ void *opaque,
+ QCryptoBlockAmendOptions *options,
+ bool force,
+ Error **errp);
+
/**
* qcrypto_block_calculate_payload_offset:
diff --git a/block/crypto.c b/block/crypto.c
index 973b57b3eb..dcf8b42bb2 100644
--- a/block/crypto.c
+++ b/block/crypto.c
@@ -202,6 +202,23 @@ block_crypto_create_opts_init(QDict *opts, Error **errp)
return ret;
}
+QCryptoBlockAmendOptions *
+block_crypto_amend_opts_init(QDict *opts, Error **errp)
+{
+ Visitor *v;
+ QCryptoBlockAmendOptions *ret;
+
+ v = qobject_input_visitor_new_flat_confused(opts, errp);
+ if (!v) {
+ return NULL;
+ }
+
+ visit_type_QCryptoBlockAmendOptions(v, NULL, &ret, errp);
+
+ visit_free(v);
+ return ret;
+}
+
static int block_crypto_open_generic(QCryptoBlockFormat format,
QemuOptsList *opts_spec,
diff --git a/crypto/block.c b/crypto/block.c
index 6f42b32f1e..eb057948b5 100644
--- a/crypto/block.c
+++ b/crypto/block.c
@@ -150,6 +150,35 @@ qcrypto_block_calculate_payload_offset(QCryptoBlockCreateOptions *create_opts,
return crypto != NULL;
}
+int qcrypto_block_amend_options(QCryptoBlock *block,
+ QCryptoBlockReadFunc readfunc,
+ QCryptoBlockWriteFunc writefunc,
+ void *opaque,
+ QCryptoBlockAmendOptions *options,
+ bool force,
+ Error **errp)
+{
+ if (options->format != block->format) {
+ error_setg(errp,
+ "Cannot amend encryption format");
+ return -1;
+ }
+
+ if (!block->driver->amend) {
+ error_setg(errp,
+ "Crypto format %s doesn't support format options amendment",
+ QCryptoBlockFormat_str(block->format));
+ return -1;
+ }
+
+ return block->driver->amend(block,
+ readfunc,
+ writefunc,
+ opaque,
+ options,
+ force,
+ errp);
+}
QCryptoBlockInfo *qcrypto_block_get_info(QCryptoBlock *block,
Error **errp)
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 10/31] qcrypto/luks: implement encryption key management
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (8 preceding siblings ...)
2020-07-06 10:04 ` [PULL 09/31] qcrypto/core: add generic infrastructure for crypto options amendment Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 11/31] block/amend: add 'force' option Max Reitz
` (22 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Maxim Levitsky <mlevitsk@redhat.com>
Next few patches will expose that functionality to the user.
Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200608094030.670121-3-mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
qapi/crypto.json | 59 ++++++-
crypto/block-luks.c | 416 +++++++++++++++++++++++++++++++++++++++++++-
2 files changed, 469 insertions(+), 6 deletions(-)
diff --git a/qapi/crypto.json b/qapi/crypto.json
index aeb6c7ef7b..5a68e0db25 100644
--- a/qapi/crypto.json
+++ b/qapi/crypto.json
@@ -297,7 +297,6 @@
'uuid': 'str',
'slots': [ 'QCryptoBlockInfoLUKSSlot' ] }}
-
##
# @QCryptoBlockInfo:
#
@@ -310,7 +309,63 @@
'discriminator': 'format',
'data': { 'luks': 'QCryptoBlockInfoLUKS' } }
+##
+# @QCryptoBlockLUKSKeyslotState:
+#
+# Defines state of keyslots that are affected by the update
+#
+# @active: The slots contain the given password and marked as active
+# @inactive: The slots are erased (contain garbage) and marked as inactive
+#
+# Since: 5.1
+##
+{ 'enum': 'QCryptoBlockLUKSKeyslotState',
+ 'data': [ 'active', 'inactive' ] }
+
+##
+# @QCryptoBlockAmendOptionsLUKS:
+#
+# This struct defines the update parameters that activate/de-activate set
+# of keyslots
+#
+# @state: the desired state of the keyslots
+#
+# @new-secret: The ID of a QCryptoSecret object providing the password to be
+# written into added active keyslots
+#
+# @old-secret: Optional (for deactivation only)
+# If given will deactive all keyslots that
+# match password located in QCryptoSecret with this ID
+#
+# @iter-time: Optional (for activation only)
+# Number of milliseconds to spend in
+# PBKDF passphrase processing for the newly activated keyslot.
+# Currently defaults to 2000.
+#
+# @keyslot: Optional. ID of the keyslot to activate/deactivate.
+# For keyslot activation, keyslot should not be active already
+# (this is unsafe to update an active keyslot),
+# but possible if 'force' parameter is given.
+# If keyslot is not given, first free keyslot will be written.
+#
+# For keyslot deactivation, this parameter specifies the exact
+# keyslot to deactivate
+#
+# @secret: Optional. The ID of a QCryptoSecret object providing the
+# password to use to retrive current master key.
+# Defaults to the same secret that was used to open the image
+#
+#
+# Since 5.1
+##
+{ 'struct': 'QCryptoBlockAmendOptionsLUKS',
+ 'data': { 'state': 'QCryptoBlockLUKSKeyslotState',
+ '*new-secret': 'str',
+ '*old-secret': 'str',
+ '*keyslot': 'int',
+ '*iter-time': 'int',
+ '*secret': 'str' } }
##
# @QCryptoBlockAmendOptions:
@@ -324,4 +379,4 @@
'base': 'QCryptoBlockOptionsBase',
'discriminator': 'format',
'data': {
- } }
+ 'luks': 'QCryptoBlockAmendOptionsLUKS' } }
diff --git a/crypto/block-luks.c b/crypto/block-luks.c
index 4861db810c..564caa1094 100644
--- a/crypto/block-luks.c
+++ b/crypto/block-luks.c
@@ -32,6 +32,7 @@
#include "qemu/uuid.h"
#include "qemu/coroutine.h"
+#include "qemu/bitmap.h"
/*
* Reference for the LUKS format implemented here is
@@ -70,6 +71,9 @@ typedef struct QCryptoBlockLUKSKeySlot QCryptoBlockLUKSKeySlot;
#define QCRYPTO_BLOCK_LUKS_SECTOR_SIZE 512LL
+#define QCRYPTO_BLOCK_LUKS_DEFAULT_ITER_TIME_MS 2000
+#define QCRYPTO_BLOCK_LUKS_ERASE_ITERATIONS 40
+
static const char qcrypto_block_luks_magic[QCRYPTO_BLOCK_LUKS_MAGIC_LEN] = {
'L', 'U', 'K', 'S', 0xBA, 0xBE
};
@@ -219,6 +223,9 @@ struct QCryptoBlockLUKS {
/* Hash algorithm used in pbkdf2 function */
QCryptoHashAlgorithm hash_alg;
+
+ /* Name of the secret that was used to open the image */
+ char *secret;
};
@@ -720,7 +727,7 @@ qcrypto_block_luks_store_key(QCryptoBlock *block,
Error **errp)
{
QCryptoBlockLUKS *luks = block->opaque;
- QCryptoBlockLUKSKeySlot *slot = &luks->header.key_slots[slot_idx];
+ QCryptoBlockLUKSKeySlot *slot;
g_autofree uint8_t *splitkey = NULL;
size_t splitkeylen;
g_autofree uint8_t *slotkey = NULL;
@@ -730,6 +737,8 @@ qcrypto_block_luks_store_key(QCryptoBlock *block,
uint64_t iters;
int ret = -1;
+ assert(slot_idx < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS);
+ slot = &luks->header.key_slots[slot_idx];
if (qcrypto_random_bytes(slot->salt,
QCRYPTO_BLOCK_LUKS_SALT_LEN,
errp) < 0) {
@@ -890,7 +899,7 @@ qcrypto_block_luks_load_key(QCryptoBlock *block,
Error **errp)
{
QCryptoBlockLUKS *luks = block->opaque;
- const QCryptoBlockLUKSKeySlot *slot = &luks->header.key_slots[slot_idx];
+ const QCryptoBlockLUKSKeySlot *slot;
g_autofree uint8_t *splitkey = NULL;
size_t splitkeylen;
g_autofree uint8_t *possiblekey = NULL;
@@ -900,6 +909,8 @@ qcrypto_block_luks_load_key(QCryptoBlock *block,
g_autoptr(QCryptoIVGen) ivgen = NULL;
size_t niv;
+ assert(slot_idx < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS);
+ slot = &luks->header.key_slots[slot_idx];
if (slot->active != QCRYPTO_BLOCK_LUKS_KEY_SLOT_ENABLED) {
return 0;
}
@@ -1069,6 +1080,126 @@ qcrypto_block_luks_find_key(QCryptoBlock *block,
return -1;
}
+/*
+ * Returns true if a slot i is marked as active
+ * (contains encrypted copy of the master key)
+ */
+static bool
+qcrypto_block_luks_slot_active(const QCryptoBlockLUKS *luks,
+ unsigned int slot_idx)
+{
+ uint32_t val;
+
+ assert(slot_idx < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS);
+ val = luks->header.key_slots[slot_idx].active;
+ return val == QCRYPTO_BLOCK_LUKS_KEY_SLOT_ENABLED;
+}
+
+/*
+ * Returns the number of slots that are marked as active
+ * (slots that contain encrypted copy of the master key)
+ */
+static unsigned int
+qcrypto_block_luks_count_active_slots(const QCryptoBlockLUKS *luks)
+{
+ size_t i = 0;
+ unsigned int ret = 0;
+
+ for (i = 0; i < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS; i++) {
+ if (qcrypto_block_luks_slot_active(luks, i)) {
+ ret++;
+ }
+ }
+ return ret;
+}
+
+/*
+ * Finds first key slot which is not active
+ * Returns the key slot index, or -1 if it doesn't exist
+ */
+static int
+qcrypto_block_luks_find_free_keyslot(const QCryptoBlockLUKS *luks)
+{
+ size_t i;
+
+ for (i = 0; i < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS; i++) {
+ if (!qcrypto_block_luks_slot_active(luks, i)) {
+ return i;
+ }
+ }
+ return -1;
+}
+
+/*
+ * Erases an keyslot given its index
+ * Returns:
+ * 0 if the keyslot was erased successfully
+ * -1 if a error occurred while erasing the keyslot
+ *
+ */
+static int
+qcrypto_block_luks_erase_key(QCryptoBlock *block,
+ unsigned int slot_idx,
+ QCryptoBlockWriteFunc writefunc,
+ void *opaque,
+ Error **errp)
+{
+ QCryptoBlockLUKS *luks = block->opaque;
+ QCryptoBlockLUKSKeySlot *slot;
+ g_autofree uint8_t *garbagesplitkey = NULL;
+ size_t splitkeylen;
+ size_t i;
+ Error *local_err = NULL;
+ int ret;
+
+ assert(slot_idx < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS);
+ slot = &luks->header.key_slots[slot_idx];
+
+ splitkeylen = luks->header.master_key_len * slot->stripes;
+ assert(splitkeylen > 0);
+
+ garbagesplitkey = g_new0(uint8_t, splitkeylen);
+
+ /* Reset the key slot header */
+ memset(slot->salt, 0, QCRYPTO_BLOCK_LUKS_SALT_LEN);
+ slot->iterations = 0;
+ slot->active = QCRYPTO_BLOCK_LUKS_KEY_SLOT_DISABLED;
+
+ ret = qcrypto_block_luks_store_header(block, writefunc,
+ opaque, &local_err);
+
+ if (ret < 0) {
+ error_propagate(errp, local_err);
+ }
+ /*
+ * Now try to erase the key material, even if the header
+ * update failed
+ */
+ for (i = 0; i < QCRYPTO_BLOCK_LUKS_ERASE_ITERATIONS; i++) {
+ if (qcrypto_random_bytes(garbagesplitkey,
+ splitkeylen, &local_err) < 0) {
+ /*
+ * If we failed to get the random data, still write
+ * at least zeros to the key slot at least once
+ */
+ error_propagate(errp, local_err);
+
+ if (i > 0) {
+ return -1;
+ }
+ }
+ if (writefunc(block,
+ slot->key_offset_sector * QCRYPTO_BLOCK_LUKS_SECTOR_SIZE,
+ garbagesplitkey,
+ splitkeylen,
+ opaque,
+ &local_err) != splitkeylen) {
+ error_propagate(errp, local_err);
+ return -1;
+ }
+ }
+ return ret;
+}
static int
qcrypto_block_luks_open(QCryptoBlock *block,
@@ -1099,6 +1230,7 @@ qcrypto_block_luks_open(QCryptoBlock *block,
luks = g_new0(QCryptoBlockLUKS, 1);
block->opaque = luks;
+ luks->secret = g_strdup(options->u.luks.key_secret);
if (qcrypto_block_luks_load_header(block, readfunc, opaque, errp) < 0) {
goto fail;
@@ -1164,6 +1296,7 @@ qcrypto_block_luks_open(QCryptoBlock *block,
fail:
qcrypto_block_free_cipher(block);
qcrypto_ivgen_free(block->ivgen);
+ g_free(luks->secret);
g_free(luks);
return -1;
}
@@ -1204,7 +1337,7 @@ qcrypto_block_luks_create(QCryptoBlock *block,
memcpy(&luks_opts, &options->u.luks, sizeof(luks_opts));
if (!luks_opts.has_iter_time) {
- luks_opts.iter_time = 2000;
+ luks_opts.iter_time = QCRYPTO_BLOCK_LUKS_DEFAULT_ITER_TIME_MS;
}
if (!luks_opts.has_cipher_alg) {
luks_opts.cipher_alg = QCRYPTO_CIPHER_ALG_AES_256;
@@ -1244,6 +1377,8 @@ qcrypto_block_luks_create(QCryptoBlock *block,
optprefix ? optprefix : "");
goto error;
}
+ luks->secret = g_strdup(options->u.luks.key_secret);
+
password = qcrypto_secret_lookup_as_utf8(luks_opts.key_secret, errp);
if (!password) {
goto error;
@@ -1471,10 +1606,278 @@ qcrypto_block_luks_create(QCryptoBlock *block,
qcrypto_block_free_cipher(block);
qcrypto_ivgen_free(block->ivgen);
+ g_free(luks->secret);
g_free(luks);
return -1;
}
+static int
+qcrypto_block_luks_amend_add_keyslot(QCryptoBlock *block,
+ QCryptoBlockReadFunc readfunc,
+ QCryptoBlockWriteFunc writefunc,
+ void *opaque,
+ QCryptoBlockAmendOptionsLUKS *opts_luks,
+ bool force,
+ Error **errp)
+{
+ QCryptoBlockLUKS *luks = block->opaque;
+ uint64_t iter_time = opts_luks->has_iter_time ?
+ opts_luks->iter_time :
+ QCRYPTO_BLOCK_LUKS_DEFAULT_ITER_TIME_MS;
+ int keyslot;
+ g_autofree char *old_password = NULL;
+ g_autofree char *new_password = NULL;
+ g_autofree uint8_t *master_key = NULL;
+
+ char *secret = opts_luks->has_secret ? opts_luks->secret : luks->secret;
+
+ if (!opts_luks->has_new_secret) {
+ error_setg(errp, "'new-secret' is required to activate a keyslot");
+ return -1;
+ }
+ if (opts_luks->has_old_secret) {
+ error_setg(errp,
+ "'old-secret' must not be given when activating keyslots");
+ return -1;
+ }
+
+ if (opts_luks->has_keyslot) {
+ keyslot = opts_luks->keyslot;
+ if (keyslot < 0 || keyslot >= QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS) {
+ error_setg(errp,
+ "Invalid keyslot %u specified, must be between 0 and %u",
+ keyslot, QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS - 1);
+ return -1;
+ }
+ } else {
+ keyslot = qcrypto_block_luks_find_free_keyslot(luks);
+ if (keyslot == -1) {
+ error_setg(errp,
+ "Can't add a keyslot - all keyslots are in use");
+ return -1;
+ }
+ }
+
+ if (!force && qcrypto_block_luks_slot_active(luks, keyslot)) {
+ error_setg(errp,
+ "Refusing to overwrite active keyslot %i - "
+ "please erase it first",
+ keyslot);
+ return -1;
+ }
+
+ /* Locate the password that will be used to retrieve the master key */
+ old_password = qcrypto_secret_lookup_as_utf8(secret, errp);
+ if (!old_password) {
+ return -1;
+ }
+
+ /* Retrieve the master key */
+ master_key = g_new0(uint8_t, luks->header.master_key_len);
+
+ if (qcrypto_block_luks_find_key(block, old_password, master_key,
+ readfunc, opaque, errp) < 0) {
+ error_append_hint(errp, "Failed to retrieve the master key");
+ return -1;
+ }
+
+ /* Locate the new password*/
+ new_password = qcrypto_secret_lookup_as_utf8(opts_luks->new_secret, errp);
+ if (!new_password) {
+ return -1;
+ }
+
+ /* Now set the new keyslots */
+ if (qcrypto_block_luks_store_key(block, keyslot, new_password, master_key,
+ iter_time, writefunc, opaque, errp)) {
+ error_append_hint(errp, "Failed to write to keyslot %i", keyslot);
+ return -1;
+ }
+ return 0;
+}
+
+static int
+qcrypto_block_luks_amend_erase_keyslots(QCryptoBlock *block,
+ QCryptoBlockReadFunc readfunc,
+ QCryptoBlockWriteFunc writefunc,
+ void *opaque,
+ QCryptoBlockAmendOptionsLUKS *opts_luks,
+ bool force,
+ Error **errp)
+{
+ QCryptoBlockLUKS *luks = block->opaque;
+ g_autofree uint8_t *tmpkey = NULL;
+ g_autofree char *old_password = NULL;
+
+ if (opts_luks->has_new_secret) {
+ error_setg(errp,
+ "'new-secret' must not be given when erasing keyslots");
+ return -1;
+ }
+ if (opts_luks->has_iter_time) {
+ error_setg(errp,
+ "'iter-time' must not be given when erasing keyslots");
+ return -1;
+ }
+ if (opts_luks->has_secret) {
+ error_setg(errp,
+ "'secret' must not be given when erasing keyslots");
+ return -1;
+ }
+
+ /* Load the old password if given */
+ if (opts_luks->has_old_secret) {
+ old_password = qcrypto_secret_lookup_as_utf8(opts_luks->old_secret,
+ errp);
+ if (!old_password) {
+ return -1;
+ }
+
+ /*
+ * Allocate a temporary key buffer that we will need when
+ * checking if slot matches the given old password
+ */
+ tmpkey = g_new0(uint8_t, luks->header.master_key_len);
+ }
+
+ /* Erase an explicitly given keyslot */
+ if (opts_luks->has_keyslot) {
+ int keyslot = opts_luks->keyslot;
+
+ if (keyslot < 0 || keyslot >= QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS) {
+ error_setg(errp,
+ "Invalid keyslot %i specified, must be between 0 and %i",
+ keyslot, QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS - 1);
+ return -1;
+ }
+
+ if (opts_luks->has_old_secret) {
+ int rv = qcrypto_block_luks_load_key(block,
+ keyslot,
+ old_password,
+ tmpkey,
+ readfunc,
+ opaque,
+ errp);
+ if (rv == -1) {
+ return -1;
+ } else if (rv == 0) {
+ error_setg(errp,
+ "Given keyslot %i doesn't contain the given "
+ "old password for erase operation",
+ keyslot);
+ return -1;
+ }
+ }
+
+ if (!force && !qcrypto_block_luks_slot_active(luks, keyslot)) {
+ error_setg(errp,
+ "Given keyslot %i is already erased (inactive) ",
+ keyslot);
+ return -1;
+ }
+
+ if (!force && qcrypto_block_luks_count_active_slots(luks) == 1) {
+ error_setg(errp,
+ "Attempt to erase the only active keyslot %i "
+ "which will erase all the data in the image "
+ "irreversibly - refusing operation",
+ keyslot);
+ return -1;
+ }
+
+ if (qcrypto_block_luks_erase_key(block, keyslot,
+ writefunc, opaque, errp)) {
+ error_append_hint(errp, "Failed to erase keyslot %i", keyslot);
+ return -1;
+ }
+
+ /* Erase all keyslots that match the given old password */
+ } else if (opts_luks->has_old_secret) {
+
+ unsigned long slots_to_erase_bitmap = 0;
+ size_t i;
+ int slot_count;
+
+ assert(QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS <=
+ sizeof(slots_to_erase_bitmap) * 8);
+
+ for (i = 0; i < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS; i++) {
+ int rv = qcrypto_block_luks_load_key(block,
+ i,
+ old_password,
+ tmpkey,
+ readfunc,
+ opaque,
+ errp);
+ if (rv == -1) {
+ return -1;
+ } else if (rv == 1) {
+ bitmap_set(&slots_to_erase_bitmap, i, 1);
+ }
+ }
+
+ slot_count = bitmap_count_one(&slots_to_erase_bitmap,
+ QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS);
+ if (slot_count == 0) {
+ error_setg(errp,
+ "No keyslots match given (old) password for erase operation");
+ return -1;
+ }
+
+ if (!force &&
+ slot_count == qcrypto_block_luks_count_active_slots(luks)) {
+ error_setg(errp,
+ "All the active keyslots match the (old) password that "
+ "was given and erasing them will erase all the data in "
+ "the image irreversibly - refusing operation");
+ return -1;
+ }
+
+ /* Now apply the update */
+ for (i = 0; i < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS; i++) {
+ if (!test_bit(i, &slots_to_erase_bitmap)) {
+ continue;
+ }
+ if (qcrypto_block_luks_erase_key(block, i, writefunc,
+ opaque, errp)) {
+ error_append_hint(errp, "Failed to erase keyslot %zu", i);
+ return -1;
+ }
+ }
+ } else {
+ error_setg(errp,
+ "To erase keyslot(s), either explicit keyslot index "
+ "or the password currently contained in them must be given");
+ return -1;
+ }
+ return 0;
+}
+
+static int
+qcrypto_block_luks_amend_options(QCryptoBlock *block,
+ QCryptoBlockReadFunc readfunc,
+ QCryptoBlockWriteFunc writefunc,
+ void *opaque,
+ QCryptoBlockAmendOptions *options,
+ bool force,
+ Error **errp)
+{
+ QCryptoBlockAmendOptionsLUKS *opts_luks = &options->u.luks;
+
+ switch (opts_luks->state) {
+ case Q_CRYPTO_BLOCKLUKS_KEYSLOT_STATE_ACTIVE:
+ return qcrypto_block_luks_amend_add_keyslot(block, readfunc,
+ writefunc, opaque,
+ opts_luks, force, errp);
+ case Q_CRYPTO_BLOCKLUKS_KEYSLOT_STATE_INACTIVE:
+ return qcrypto_block_luks_amend_erase_keyslots(block, readfunc,
+ writefunc, opaque,
+ opts_luks, force, errp);
+ default:
+ g_assert_not_reached();
+ }
+}
static int qcrypto_block_luks_get_info(QCryptoBlock *block,
QCryptoBlockInfo *info,
@@ -1523,7 +1926,11 @@ static int qcrypto_block_luks_get_info(QCryptoBlock *block,
static void qcrypto_block_luks_cleanup(QCryptoBlock *block)
{
- g_free(block->opaque);
+ QCryptoBlockLUKS *luks = block->opaque;
+ if (luks) {
+ g_free(luks->secret);
+ g_free(luks);
+ }
}
@@ -1560,6 +1967,7 @@ qcrypto_block_luks_encrypt(QCryptoBlock *block,
const QCryptoBlockDriver qcrypto_block_driver_luks = {
.open = qcrypto_block_luks_open,
.create = qcrypto_block_luks_create,
+ .amend = qcrypto_block_luks_amend_options,
.get_info = qcrypto_block_luks_get_info,
.cleanup = qcrypto_block_luks_cleanup,
.decrypt = qcrypto_block_luks_decrypt,
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 11/31] block/amend: add 'force' option
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (9 preceding siblings ...)
2020-07-06 10:04 ` [PULL 10/31] qcrypto/luks: implement encryption key management Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 12/31] block/amend: separate amend and create options for qemu-img Max Reitz
` (21 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Maxim Levitsky <mlevitsk@redhat.com>
'force' option will be used for some unsafe amend operations.
This includes things like erasing last keyslot in luks based formats
(which destroys the data, unless the master key is backed up
by external means), but that _might_ be desired result.
Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200608094030.670121-4-mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
docs/tools/qemu-img.rst | 5 ++++-
include/block/block.h | 1 +
include/block/block_int.h | 1 +
block.c | 4 +++-
block/qcow2.c | 1 +
qemu-img.c | 8 +++++++-
qemu-img-cmds.hx | 4 ++--
7 files changed, 19 insertions(+), 5 deletions(-)
diff --git a/docs/tools/qemu-img.rst b/docs/tools/qemu-img.rst
index 7f0737488a..e33f5575e3 100644
--- a/docs/tools/qemu-img.rst
+++ b/docs/tools/qemu-img.rst
@@ -253,11 +253,14 @@ Command description:
.. program:: qemu-img-commands
-.. option:: amend [--object OBJECTDEF] [--image-opts] [-p] [-q] [-f FMT] [-t CACHE] -o OPTIONS FILENAME
+.. option:: amend [--object OBJECTDEF] [--image-opts] [-p] [-q] [-f FMT] [-t CACHE] [--force] -o OPTIONS FILENAME
Amends the image format specific *OPTIONS* for the image file
*FILENAME*. Not all file formats support this operation.
+ --force allows some unsafe operations. Currently for -f luks, it allows to
+ erase the last encryption key, and to overwrite an active encryption key.
+
.. option:: bench [-c COUNT] [-d DEPTH] [-f FMT] [--flush-interval=FLUSH_INTERVAL] [-i AIO] [-n] [--no-drain] [-o OFFSET] [--pattern=PATTERN] [-q] [-s BUFFER_SIZE] [-S STEP_SIZE] [-t CACHE] [-w] [-U] FILENAME
Run a simple sequential I/O benchmark on the specified image. If ``-w`` is
diff --git a/include/block/block.h b/include/block/block.h
index e8fc814996..a2414a58c5 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -450,6 +450,7 @@ typedef void BlockDriverAmendStatusCB(BlockDriverState *bs, int64_t offset,
int64_t total_work_size, void *opaque);
int bdrv_amend_options(BlockDriverState *bs_new, QemuOpts *opts,
BlockDriverAmendStatusCB *status_cb, void *cb_opaque,
+ bool force,
Error **errp);
/* check if a named node can be replaced when doing drive-mirror */
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 791de6a59c..066b9eaa40 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -440,6 +440,7 @@ struct BlockDriver {
int (*bdrv_amend_options)(BlockDriverState *bs, QemuOpts *opts,
BlockDriverAmendStatusCB *status_cb,
void *cb_opaque,
+ bool force,
Error **errp);
void (*bdrv_debug_event)(BlockDriverState *bs, BlkdebugEvent event);
diff --git a/block.c b/block.c
index 6dbcb7e083..144f52e413 100644
--- a/block.c
+++ b/block.c
@@ -6482,6 +6482,7 @@ void bdrv_remove_aio_context_notifier(BlockDriverState *bs,
int bdrv_amend_options(BlockDriverState *bs, QemuOpts *opts,
BlockDriverAmendStatusCB *status_cb, void *cb_opaque,
+ bool force,
Error **errp)
{
if (!bs->drv) {
@@ -6493,7 +6494,8 @@ int bdrv_amend_options(BlockDriverState *bs, QemuOpts *opts,
bs->drv->format_name);
return -ENOTSUP;
}
- return bs->drv->bdrv_amend_options(bs, opts, status_cb, cb_opaque, errp);
+ return bs->drv->bdrv_amend_options(bs, opts, status_cb,
+ cb_opaque, force, errp);
}
/*
diff --git a/block/qcow2.c b/block/qcow2.c
index e20590c3b7..f6c9207312 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -5351,6 +5351,7 @@ static void qcow2_amend_helper_cb(BlockDriverState *bs,
static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
BlockDriverAmendStatusCB *status_cb,
void *cb_opaque,
+ bool force,
Error **errp)
{
BDRVQcow2State *s = bs->opaque;
diff --git a/qemu-img.c b/qemu-img.c
index bdb9f6aa46..8c26bfafc6 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -79,6 +79,7 @@ enum {
OPTION_DISABLE = 273,
OPTION_MERGE = 274,
OPTION_BITMAPS = 275,
+ OPTION_FORCE = 276,
};
typedef enum OutputFormat {
@@ -4090,6 +4091,7 @@ static int img_amend(int argc, char **argv)
BlockBackend *blk = NULL;
BlockDriverState *bs = NULL;
bool image_opts = false;
+ bool force = false;
cache = BDRV_DEFAULT_CACHE;
for (;;) {
@@ -4097,6 +4099,7 @@ static int img_amend(int argc, char **argv)
{"help", no_argument, 0, 'h'},
{"object", required_argument, 0, OPTION_OBJECT},
{"image-opts", no_argument, 0, OPTION_IMAGE_OPTS},
+ {"force", no_argument, 0, OPTION_FORCE},
{0, 0, 0, 0}
};
c = getopt_long(argc, argv, ":ho:f:t:pq",
@@ -4144,6 +4147,9 @@ static int img_amend(int argc, char **argv)
case OPTION_IMAGE_OPTS:
image_opts = true;
break;
+ case OPTION_FORCE:
+ force = true;
+ break;
}
}
@@ -4221,7 +4227,7 @@ static int img_amend(int argc, char **argv)
/* In case the driver does not call amend_status_cb() */
qemu_progress_print(0.f, 0);
- ret = bdrv_amend_options(bs, opts, &amend_status_cb, NULL, &err);
+ ret = bdrv_amend_options(bs, opts, &amend_status_cb, NULL, force, &err);
qemu_progress_print(100.f, 0);
if (ret < 0) {
error_report_err(err);
diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
index 10b910b67c..b89c019b76 100644
--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -10,9 +10,9 @@ HXCOMM When amending the rST sections, please remember to copy the usage
HXCOMM over to the per-command sections in docs/tools/qemu-img.rst.
DEF("amend", img_amend,
- "amend [--object objectdef] [--image-opts] [-p] [-q] [-f fmt] [-t cache] -o options filename")
+ "amend [--object objectdef] [--image-opts] [-p] [-q] [-f fmt] [-t cache] [--force] -o options filename")
SRST
-.. option:: amend [--object OBJECTDEF] [--image-opts] [-p] [-q] [-f FMT] [-t CACHE] -o OPTIONS FILENAME
+.. option:: amend [--object OBJECTDEF] [--image-opts] [-p] [-q] [-f FMT] [-t CACHE] [--force] -o OPTIONS FILENAME
ERST
DEF("bench", img_bench,
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 12/31] block/amend: separate amend and create options for qemu-img
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (10 preceding siblings ...)
2020-07-06 10:04 ` [PULL 11/31] block/amend: add 'force' option Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 13/31] block/amend: refactor qcow2 amend options Max Reitz
` (20 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Maxim Levitsky <mlevitsk@redhat.com>
Some options are only useful for creation
(or hard to be amended, like cluster size for qcow2), while some other
options are only useful for amend, like upcoming keyslot management
options for luks
Since currently only qcow2 supports amend, move all its options
to a common macro and then include it in each action option list.
In future it might be useful to remove some options which are
not supported anyway from amend list, which currently
cause an error message if amended.
Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200608094030.670121-5-mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
include/block/block_int.h | 4 +
block/qcow2.c | 173 +++++++++++++++++++++-----------------
qemu-img.c | 18 ++--
3 files changed, 107 insertions(+), 88 deletions(-)
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 066b9eaa40..ed335519cc 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -420,6 +420,10 @@ struct BlockDriver {
/* List of options for creating images, terminated by name == NULL */
QemuOptsList *create_opts;
+
+ /* List of options for image amend */
+ QemuOptsList *amend_opts;
+
/*
* If this driver supports reopening images this contains a
* NULL-terminated list of the runtime options that can be
diff --git a/block/qcow2.c b/block/qcow2.c
index f6c9207312..3898853ef8 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -5660,89 +5660,103 @@ void qcow2_signal_corruption(BlockDriverState *bs, bool fatal, int64_t offset,
s->signaled_corruption = true;
}
+#define QCOW_COMMON_OPTIONS \
+ { \
+ .name = BLOCK_OPT_SIZE, \
+ .type = QEMU_OPT_SIZE, \
+ .help = "Virtual disk size" \
+ }, \
+ { \
+ .name = BLOCK_OPT_COMPAT_LEVEL, \
+ .type = QEMU_OPT_STRING, \
+ .help = "Compatibility level (v2 [0.10] or v3 [1.1])" \
+ }, \
+ { \
+ .name = BLOCK_OPT_BACKING_FILE, \
+ .type = QEMU_OPT_STRING, \
+ .help = "File name of a base image" \
+ }, \
+ { \
+ .name = BLOCK_OPT_BACKING_FMT, \
+ .type = QEMU_OPT_STRING, \
+ .help = "Image format of the base image" \
+ }, \
+ { \
+ .name = BLOCK_OPT_DATA_FILE, \
+ .type = QEMU_OPT_STRING, \
+ .help = "File name of an external data file" \
+ }, \
+ { \
+ .name = BLOCK_OPT_DATA_FILE_RAW, \
+ .type = QEMU_OPT_BOOL, \
+ .help = "The external data file must stay valid " \
+ "as a raw image" \
+ }, \
+ { \
+ .name = BLOCK_OPT_ENCRYPT, \
+ .type = QEMU_OPT_BOOL, \
+ .help = "Encrypt the image with format 'aes'. (Deprecated " \
+ "in favor of " BLOCK_OPT_ENCRYPT_FORMAT "=aes)", \
+ }, \
+ { \
+ .name = BLOCK_OPT_ENCRYPT_FORMAT, \
+ .type = QEMU_OPT_STRING, \
+ .help = "Encrypt the image, format choices: 'aes', 'luks'", \
+ }, \
+ BLOCK_CRYPTO_OPT_DEF_KEY_SECRET("encrypt.", \
+ "ID of secret providing qcow AES key or LUKS passphrase"), \
+ BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_ALG("encrypt."), \
+ BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_MODE("encrypt."), \
+ BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_ALG("encrypt."), \
+ BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG("encrypt."), \
+ BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG("encrypt."), \
+ BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME("encrypt."), \
+ { \
+ .name = BLOCK_OPT_CLUSTER_SIZE, \
+ .type = QEMU_OPT_SIZE, \
+ .help = "qcow2 cluster size", \
+ .def_value_str = stringify(DEFAULT_CLUSTER_SIZE) \
+ }, \
+ { \
+ .name = BLOCK_OPT_PREALLOC, \
+ .type = QEMU_OPT_STRING, \
+ .help = "Preallocation mode (allowed values: off, " \
+ "metadata, falloc, full)" \
+ }, \
+ { \
+ .name = BLOCK_OPT_LAZY_REFCOUNTS, \
+ .type = QEMU_OPT_BOOL, \
+ .help = "Postpone refcount updates", \
+ .def_value_str = "off" \
+ }, \
+ { \
+ .name = BLOCK_OPT_REFCOUNT_BITS, \
+ .type = QEMU_OPT_NUMBER, \
+ .help = "Width of a reference count entry in bits", \
+ .def_value_str = "16" \
+ }, \
+ { \
+ .name = BLOCK_OPT_COMPRESSION_TYPE, \
+ .type = QEMU_OPT_STRING, \
+ .help = "Compression method used for image cluster " \
+ "compression", \
+ .def_value_str = "zlib" \
+ }
+
static QemuOptsList qcow2_create_opts = {
.name = "qcow2-create-opts",
.head = QTAILQ_HEAD_INITIALIZER(qcow2_create_opts.head),
.desc = {
- {
- .name = BLOCK_OPT_SIZE,
- .type = QEMU_OPT_SIZE,
- .help = "Virtual disk size"
- },
- {
- .name = BLOCK_OPT_COMPAT_LEVEL,
- .type = QEMU_OPT_STRING,
- .help = "Compatibility level (v2 [0.10] or v3 [1.1])"
- },
- {
- .name = BLOCK_OPT_BACKING_FILE,
- .type = QEMU_OPT_STRING,
- .help = "File name of a base image"
- },
- {
- .name = BLOCK_OPT_BACKING_FMT,
- .type = QEMU_OPT_STRING,
- .help = "Image format of the base image"
- },
- {
- .name = BLOCK_OPT_DATA_FILE,
- .type = QEMU_OPT_STRING,
- .help = "File name of an external data file"
- },
- {
- .name = BLOCK_OPT_DATA_FILE_RAW,
- .type = QEMU_OPT_BOOL,
- .help = "The external data file must stay valid as a raw image"
- },
- {
- .name = BLOCK_OPT_ENCRYPT,
- .type = QEMU_OPT_BOOL,
- .help = "Encrypt the image with format 'aes'. (Deprecated "
- "in favor of " BLOCK_OPT_ENCRYPT_FORMAT "=aes)",
- },
- {
- .name = BLOCK_OPT_ENCRYPT_FORMAT,
- .type = QEMU_OPT_STRING,
- .help = "Encrypt the image, format choices: 'aes', 'luks'",
- },
- BLOCK_CRYPTO_OPT_DEF_KEY_SECRET("encrypt.",
- "ID of secret providing qcow AES key or LUKS passphrase"),
- BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_ALG("encrypt."),
- BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_MODE("encrypt."),
- BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_ALG("encrypt."),
- BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG("encrypt."),
- BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG("encrypt."),
- BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME("encrypt."),
- {
- .name = BLOCK_OPT_CLUSTER_SIZE,
- .type = QEMU_OPT_SIZE,
- .help = "qcow2 cluster size",
- .def_value_str = stringify(DEFAULT_CLUSTER_SIZE)
- },
- {
- .name = BLOCK_OPT_PREALLOC,
- .type = QEMU_OPT_STRING,
- .help = "Preallocation mode (allowed values: off, metadata, "
- "falloc, full)"
- },
- {
- .name = BLOCK_OPT_LAZY_REFCOUNTS,
- .type = QEMU_OPT_BOOL,
- .help = "Postpone refcount updates",
- .def_value_str = "off"
- },
- {
- .name = BLOCK_OPT_REFCOUNT_BITS,
- .type = QEMU_OPT_NUMBER,
- .help = "Width of a reference count entry in bits",
- .def_value_str = "16"
- },
- {
- .name = BLOCK_OPT_COMPRESSION_TYPE,
- .type = QEMU_OPT_STRING,
- .help = "Compression method used for image cluster compression",
- .def_value_str = "zlib"
- },
+ QCOW_COMMON_OPTIONS,
+ { /* end of list */ }
+ }
+};
+
+static QemuOptsList qcow2_amend_opts = {
+ .name = "qcow2-amend-opts",
+ .head = QTAILQ_HEAD_INITIALIZER(qcow2_amend_opts.head),
+ .desc = {
+ QCOW_COMMON_OPTIONS,
{ /* end of list */ }
}
};
@@ -5803,6 +5817,7 @@ BlockDriver bdrv_qcow2 = {
.bdrv_inactivate = qcow2_inactivate,
.create_opts = &qcow2_create_opts,
+ .amend_opts = &qcow2_amend_opts,
.strong_runtime_opts = qcow2_strong_runtime_opts,
.mutable_opts = mutable_opts,
.bdrv_co_check = qcow2_co_check,
diff --git a/qemu-img.c b/qemu-img.c
index 8c26bfafc6..1a0a85089b 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -4068,11 +4068,11 @@ static int print_amend_option_help(const char *format)
return 1;
}
- /* Every driver supporting amendment must have create_opts */
- assert(drv->create_opts);
+ /* Every driver supporting amendment must have amend_opts */
+ assert(drv->amend_opts);
printf("Creation options for '%s':\n", format);
- qemu_opts_print_help(drv->create_opts, false);
+ qemu_opts_print_help(drv->amend_opts, false);
printf("\nNote that not all of these options may be amendable.\n");
return 0;
}
@@ -4082,7 +4082,7 @@ static int img_amend(int argc, char **argv)
Error *err = NULL;
int c, ret = 0;
char *options = NULL;
- QemuOptsList *create_opts = NULL;
+ QemuOptsList *amend_opts = NULL;
QemuOpts *opts = NULL;
const char *fmt = NULL, *filename, *cache;
int flags;
@@ -4213,11 +4213,11 @@ static int img_amend(int argc, char **argv)
goto out;
}
- /* Every driver supporting amendment must have create_opts */
- assert(bs->drv->create_opts);
+ /* Every driver supporting amendment must have amend_opts */
+ assert(bs->drv->amend_opts);
- create_opts = qemu_opts_append(create_opts, bs->drv->create_opts);
- opts = qemu_opts_create(create_opts, NULL, 0, &error_abort);
+ amend_opts = qemu_opts_append(amend_opts, bs->drv->amend_opts);
+ opts = qemu_opts_create(amend_opts, NULL, 0, &error_abort);
qemu_opts_do_parse(opts, options, NULL, &err);
if (err) {
error_report_err(err);
@@ -4240,7 +4240,7 @@ out:
out_no_progress:
blk_unref(blk);
qemu_opts_del(opts);
- qemu_opts_free(create_opts);
+ qemu_opts_free(amend_opts);
g_free(options);
if (ret) {
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 13/31] block/amend: refactor qcow2 amend options
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (11 preceding siblings ...)
2020-07-06 10:04 ` [PULL 12/31] block/amend: separate amend and create options for qemu-img Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 14/31] block/crypto: rename two functions Max Reitz
` (19 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Maxim Levitsky <mlevitsk@redhat.com>
Some qcow2 create options can't be used for amend.
Remove them from the qcow2 create options and add generic logic to detect
such options in qemu-img
Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
[mreitz: Dropped some iotests reference output hunks that became
unnecessary thanks to
"iotests: Make _filter_img_create more active"]
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200625125548.870061-12-mreitz@redhat.com>
---
block/qcow2.c | 138 +++++++++-----------------------
qemu-img.c | 18 ++++-
tests/qemu-iotests/049.out | 102 ++++++++++++------------
tests/qemu-iotests/061.out | 12 ++-
tests/qemu-iotests/082.out | 158 ++++---------------------------------
tests/qemu-iotests/085.out | 38 ++++-----
tests/qemu-iotests/144.out | 4 +-
tests/qemu-iotests/182.out | 2 +-
tests/qemu-iotests/185.out | 8 +-
tests/qemu-iotests/255.out | 8 +-
tests/qemu-iotests/274.out | 46 +++++------
tests/qemu-iotests/280.out | 2 +-
12 files changed, 183 insertions(+), 353 deletions(-)
diff --git a/block/qcow2.c b/block/qcow2.c
index 3898853ef8..389205dffd 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -3042,17 +3042,6 @@ static int qcow2_change_backing_file(BlockDriverState *bs,
return qcow2_update_header(bs);
}
-static int qcow2_crypt_method_from_format(const char *encryptfmt)
-{
- if (g_str_equal(encryptfmt, "luks")) {
- return QCOW_CRYPT_LUKS;
- } else if (g_str_equal(encryptfmt, "aes")) {
- return QCOW_CRYPT_AES;
- } else {
- return -EINVAL;
- }
-}
-
static int qcow2_set_up_encryption(BlockDriverState *bs,
QCryptoBlockCreateOptions *cryptoopts,
Error **errp)
@@ -5361,9 +5350,6 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
bool lazy_refcounts = s->use_lazy_refcounts;
bool data_file_raw = data_file_is_raw(bs);
const char *compat = NULL;
- uint64_t cluster_size = s->cluster_size;
- bool encrypt;
- int encformat;
int refcount_bits = s->refcount_bits;
int ret;
QemuOptDesc *desc = opts->list->desc;
@@ -5388,44 +5374,12 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
error_setg(errp, "Unknown compatibility level %s", compat);
return -EINVAL;
}
- } else if (!strcmp(desc->name, BLOCK_OPT_PREALLOC)) {
- error_setg(errp, "Cannot change preallocation mode");
- return -ENOTSUP;
} else if (!strcmp(desc->name, BLOCK_OPT_SIZE)) {
new_size = qemu_opt_get_size(opts, BLOCK_OPT_SIZE, 0);
} else if (!strcmp(desc->name, BLOCK_OPT_BACKING_FILE)) {
backing_file = qemu_opt_get(opts, BLOCK_OPT_BACKING_FILE);
} else if (!strcmp(desc->name, BLOCK_OPT_BACKING_FMT)) {
backing_format = qemu_opt_get(opts, BLOCK_OPT_BACKING_FMT);
- } else if (!strcmp(desc->name, BLOCK_OPT_ENCRYPT)) {
- encrypt = qemu_opt_get_bool(opts, BLOCK_OPT_ENCRYPT,
- !!s->crypto);
-
- if (encrypt != !!s->crypto) {
- error_setg(errp,
- "Changing the encryption flag is not supported");
- return -ENOTSUP;
- }
- } else if (!strcmp(desc->name, BLOCK_OPT_ENCRYPT_FORMAT)) {
- encformat = qcow2_crypt_method_from_format(
- qemu_opt_get(opts, BLOCK_OPT_ENCRYPT_FORMAT));
-
- if (encformat != s->crypt_method_header) {
- error_setg(errp,
- "Changing the encryption format is not supported");
- return -ENOTSUP;
- }
- } else if (g_str_has_prefix(desc->name, "encrypt.")) {
- error_setg(errp,
- "Changing the encryption parameters is not supported");
- return -ENOTSUP;
- } else if (!strcmp(desc->name, BLOCK_OPT_CLUSTER_SIZE)) {
- cluster_size = qemu_opt_get_size(opts, BLOCK_OPT_CLUSTER_SIZE,
- cluster_size);
- if (cluster_size != s->cluster_size) {
- error_setg(errp, "Changing the cluster size is not supported");
- return -ENOTSUP;
- }
} else if (!strcmp(desc->name, BLOCK_OPT_LAZY_REFCOUNTS)) {
lazy_refcounts = qemu_opt_get_bool(opts, BLOCK_OPT_LAZY_REFCOUNTS,
lazy_refcounts);
@@ -5455,22 +5409,6 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
"images");
return -EINVAL;
}
- } else if (!strcmp(desc->name, BLOCK_OPT_COMPRESSION_TYPE)) {
- const char *ct_name =
- qemu_opt_get(opts, BLOCK_OPT_COMPRESSION_TYPE);
- int compression_type =
- qapi_enum_parse(&Qcow2CompressionType_lookup, ct_name, -1,
- NULL);
- if (compression_type == -1) {
- error_setg(errp, "Unknown compression type: %s", ct_name);
- return -ENOTSUP;
- }
-
- if (compression_type != s->compression_type) {
- error_setg(errp, "Changing the compression type "
- "is not supported");
- return -ENOTSUP;
- }
} else {
/* if this point is reached, this probably means a new option was
* added without having it covered here */
@@ -5692,37 +5630,6 @@ void qcow2_signal_corruption(BlockDriverState *bs, bool fatal, int64_t offset,
.help = "The external data file must stay valid " \
"as a raw image" \
}, \
- { \
- .name = BLOCK_OPT_ENCRYPT, \
- .type = QEMU_OPT_BOOL, \
- .help = "Encrypt the image with format 'aes'. (Deprecated " \
- "in favor of " BLOCK_OPT_ENCRYPT_FORMAT "=aes)", \
- }, \
- { \
- .name = BLOCK_OPT_ENCRYPT_FORMAT, \
- .type = QEMU_OPT_STRING, \
- .help = "Encrypt the image, format choices: 'aes', 'luks'", \
- }, \
- BLOCK_CRYPTO_OPT_DEF_KEY_SECRET("encrypt.", \
- "ID of secret providing qcow AES key or LUKS passphrase"), \
- BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_ALG("encrypt."), \
- BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_MODE("encrypt."), \
- BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_ALG("encrypt."), \
- BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG("encrypt."), \
- BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG("encrypt."), \
- BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME("encrypt."), \
- { \
- .name = BLOCK_OPT_CLUSTER_SIZE, \
- .type = QEMU_OPT_SIZE, \
- .help = "qcow2 cluster size", \
- .def_value_str = stringify(DEFAULT_CLUSTER_SIZE) \
- }, \
- { \
- .name = BLOCK_OPT_PREALLOC, \
- .type = QEMU_OPT_STRING, \
- .help = "Preallocation mode (allowed values: off, " \
- "metadata, falloc, full)" \
- }, \
{ \
.name = BLOCK_OPT_LAZY_REFCOUNTS, \
.type = QEMU_OPT_BOOL, \
@@ -5734,19 +5641,50 @@ void qcow2_signal_corruption(BlockDriverState *bs, bool fatal, int64_t offset,
.type = QEMU_OPT_NUMBER, \
.help = "Width of a reference count entry in bits", \
.def_value_str = "16" \
- }, \
- { \
- .name = BLOCK_OPT_COMPRESSION_TYPE, \
- .type = QEMU_OPT_STRING, \
- .help = "Compression method used for image cluster " \
- "compression", \
- .def_value_str = "zlib" \
}
static QemuOptsList qcow2_create_opts = {
.name = "qcow2-create-opts",
.head = QTAILQ_HEAD_INITIALIZER(qcow2_create_opts.head),
.desc = {
+ { \
+ .name = BLOCK_OPT_ENCRYPT, \
+ .type = QEMU_OPT_BOOL, \
+ .help = "Encrypt the image with format 'aes'. (Deprecated " \
+ "in favor of " BLOCK_OPT_ENCRYPT_FORMAT "=aes)", \
+ }, \
+ { \
+ .name = BLOCK_OPT_ENCRYPT_FORMAT, \
+ .type = QEMU_OPT_STRING, \
+ .help = "Encrypt the image, format choices: 'aes', 'luks'", \
+ }, \
+ BLOCK_CRYPTO_OPT_DEF_KEY_SECRET("encrypt.", \
+ "ID of secret providing qcow AES key or LUKS passphrase"), \
+ BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_ALG("encrypt."), \
+ BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_MODE("encrypt."), \
+ BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_ALG("encrypt."), \
+ BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG("encrypt."), \
+ BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG("encrypt."), \
+ BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME("encrypt."), \
+ { \
+ .name = BLOCK_OPT_CLUSTER_SIZE, \
+ .type = QEMU_OPT_SIZE, \
+ .help = "qcow2 cluster size", \
+ .def_value_str = stringify(DEFAULT_CLUSTER_SIZE) \
+ }, \
+ { \
+ .name = BLOCK_OPT_PREALLOC, \
+ .type = QEMU_OPT_STRING, \
+ .help = "Preallocation mode (allowed values: off, " \
+ "metadata, falloc, full)" \
+ }, \
+ { \
+ .name = BLOCK_OPT_COMPRESSION_TYPE, \
+ .type = QEMU_OPT_STRING, \
+ .help = "Compression method used for image cluster " \
+ "compression", \
+ .def_value_str = "zlib" \
+ },
QCOW_COMMON_OPTIONS,
{ /* end of list */ }
}
diff --git a/qemu-img.c b/qemu-img.c
index 1a0a85089b..7f4938a5ef 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -4071,9 +4071,8 @@ static int print_amend_option_help(const char *format)
/* Every driver supporting amendment must have amend_opts */
assert(drv->amend_opts);
- printf("Creation options for '%s':\n", format);
+ printf("Amend options for '%s':\n", format);
qemu_opts_print_help(drv->amend_opts, false);
- printf("\nNote that not all of these options may be amendable.\n");
return 0;
}
@@ -4219,7 +4218,22 @@ static int img_amend(int argc, char **argv)
amend_opts = qemu_opts_append(amend_opts, bs->drv->amend_opts);
opts = qemu_opts_create(amend_opts, NULL, 0, &error_abort);
qemu_opts_do_parse(opts, options, NULL, &err);
+
if (err) {
+ /* Try to parse options using the create options */
+ Error *err1 = NULL;
+ amend_opts = qemu_opts_append(amend_opts, bs->drv->create_opts);
+ qemu_opts_del(opts);
+ opts = qemu_opts_create(amend_opts, NULL, 0, &error_abort);
+ qemu_opts_do_parse(opts, options, NULL, &err1);
+
+ if (!err1) {
+ error_append_hint(&err,
+ "This option is only supported for image creation\n");
+ } else {
+ error_free(err1);
+ }
+
error_report_err(err);
ret = -1;
goto out;
diff --git a/tests/qemu-iotests/049.out b/tests/qemu-iotests/049.out
index c54ae21b86..e77966446b 100644
--- a/tests/qemu-iotests/049.out
+++ b/tests/qemu-iotests/049.out
@@ -4,90 +4,90 @@ QA output created by 049
== 1. Traditional size parameter ==
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024b
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1k
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1K
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1048576 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1G
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1073741824 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1073741824 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1T
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1099511627776 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1099511627776 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024.0
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024.0b
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5k
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5K
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1572864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1572864 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5G
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1610612736 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1610612736 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5T
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1649267441664 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1649267441664 lazy_refcounts=off refcount_bits=16
== 2. Specifying size via -o ==
qemu-img create -f qcow2 -o size=1024 TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o size=1024b TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o size=1k TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o size=1K TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o size=1M TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1048576 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o size=1G TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1073741824 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1073741824 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o size=1T TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1099511627776 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1099511627776 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o size=1024.0 TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o size=1024.0b TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o size=1.5k TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o size=1.5K TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o size=1.5M TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1572864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1572864 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o size=1.5G TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1610612736 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1610612736 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o size=1.5T TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1649267441664 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1649267441664 lazy_refcounts=off refcount_bits=16
== 3. Invalid sizes ==
@@ -129,84 +129,84 @@ qemu-img: TEST_DIR/t.qcow2: The image size must be specified only once
== Check correct interpretation of suffixes for cluster size ==
qemu-img create -f qcow2 -o cluster_size=1024 TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o cluster_size=1024b TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o cluster_size=1k TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o cluster_size=1K TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o cluster_size=1M TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1048576 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1048576 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o cluster_size=1024.0 TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o cluster_size=1024.0b TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o cluster_size=0.5k TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=512 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=512 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o cluster_size=0.5K TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=512 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=512 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o cluster_size=0.5M TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=524288 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=524288 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
== Check compat level option ==
qemu-img create -f qcow2 -o compat=0.10 TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.10 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=0.10 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o compat=1.1 TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=1.1 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=1.1 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o compat=0.42 TEST_DIR/t.qcow2 64M
qemu-img: TEST_DIR/t.qcow2: Invalid parameter '0.42'
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.42 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=0.42 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o compat=foobar TEST_DIR/t.qcow2 64M
qemu-img: TEST_DIR/t.qcow2: Invalid parameter 'foobar'
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=foobar cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=foobar lazy_refcounts=off refcount_bits=16
== Check preallocation option ==
qemu-img create -f qcow2 -o preallocation=off TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=65536 preallocation=off lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 preallocation=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o preallocation=metadata TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=65536 preallocation=metadata lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 preallocation=metadata compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o preallocation=1234 TEST_DIR/t.qcow2 64M
qemu-img: TEST_DIR/t.qcow2: Invalid parameter '1234'
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=65536 preallocation=1234 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 preallocation=1234 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
== Check encryption option ==
qemu-img create -f qcow2 -o encryption=off TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 encryption=off cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 --object secret,id=sec0,data=123456 -o encryption=on,encrypt.key-secret=sec0 TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=on encrypt.key-secret=sec0 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 encryption=on encrypt.key-secret=sec0 cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
== Check lazy_refcounts option (only with v3) ==
qemu-img create -f qcow2 -o compat=1.1,lazy_refcounts=off TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=1.1 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=1.1 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o compat=1.1,lazy_refcounts=on TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=1.1 cluster_size=65536 lazy_refcounts=on refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=1.1 lazy_refcounts=on refcount_bits=16
qemu-img create -f qcow2 -o compat=0.10,lazy_refcounts=off TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.10 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=0.10 lazy_refcounts=off refcount_bits=16
qemu-img create -f qcow2 -o compat=0.10,lazy_refcounts=on TEST_DIR/t.qcow2 64M
qemu-img: TEST_DIR/t.qcow2: Lazy refcounts only supported with compatibility level 1.1 and above (use version=v3 or greater)
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.10 cluster_size=65536 lazy_refcounts=on refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=0.10 lazy_refcounts=on refcount_bits=16
*** done
diff --git a/tests/qemu-iotests/061.out b/tests/qemu-iotests/061.out
index 2f03cf045c..b0f8befe30 100644
--- a/tests/qemu-iotests/061.out
+++ b/tests/qemu-iotests/061.out
@@ -381,16 +381,20 @@ qemu-img: Lazy refcounts only supported with compatibility level 1.1 and above (
qemu-img: Lazy refcounts only supported with compatibility level 1.1 and above (use compat=1.1 or greater)
qemu-img: Unknown compatibility level 0.42
qemu-img: Invalid parameter 'foo'
-qemu-img: Changing the cluster size is not supported
-qemu-img: Changing the encryption flag is not supported
-qemu-img: Cannot change preallocation mode
+qemu-img: Invalid parameter 'cluster_size'
+This option is only supported for image creation
+qemu-img: Invalid parameter 'encryption'
+This option is only supported for image creation
+qemu-img: Invalid parameter 'preallocation'
+This option is only supported for image creation
=== Testing correct handling of unset value ===
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
Should work:
Should not work:
-qemu-img: Changing the cluster size is not supported
+qemu-img: Invalid parameter 'cluster_size'
+This option is only supported for image creation
=== Testing zero expansion on inactive clusters ===
diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out
index 529a1214e1..b1cf5dfe43 100644
--- a/tests/qemu-iotests/082.out
+++ b/tests/qemu-iotests/082.out
@@ -3,14 +3,14 @@ QA output created by 082
=== create: Options specified more than once ===
Testing: create -f foo -f qcow2 TEST_DIR/t.qcow2 128M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
image: TEST_DIR/t.IMGFMT
file format: IMGFMT
virtual size: 128 MiB (134217728 bytes)
cluster_size: 65536
Testing: create -f qcow2 -o cluster_size=4k -o lazy_refcounts=on TEST_DIR/t.qcow2 128M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=4096 lazy_refcounts=on refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=4096 compression_type=zlib size=134217728 lazy_refcounts=on refcount_bits=16
image: TEST_DIR/t.IMGFMT
file format: IMGFMT
virtual size: 128 MiB (134217728 bytes)
@@ -23,7 +23,7 @@ Format specific information:
corrupt: false
Testing: create -f qcow2 -o cluster_size=4k -o lazy_refcounts=on -o cluster_size=8k TEST_DIR/t.qcow2 128M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=8192 lazy_refcounts=on refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=8192 compression_type=zlib size=134217728 lazy_refcounts=on refcount_bits=16
image: TEST_DIR/t.IMGFMT
file format: IMGFMT
virtual size: 128 MiB (134217728 bytes)
@@ -36,7 +36,7 @@ Format specific information:
corrupt: false
Testing: create -f qcow2 -o cluster_size=4k,cluster_size=8k TEST_DIR/t.qcow2 128M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=8192 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=8192 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
image: TEST_DIR/t.IMGFMT
file format: IMGFMT
virtual size: 128 MiB (134217728 bytes)
@@ -237,10 +237,10 @@ Supported options:
size=<size> - Virtual disk size
Testing: create -f qcow2 -u -o backing_file=TEST_DIR/t.qcow2,,help TEST_DIR/t.qcow2 128M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2,,help cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2,,help lazy_refcounts=off refcount_bits=16
Testing: create -f qcow2 -u -o backing_file=TEST_DIR/t.qcow2,,? TEST_DIR/t.qcow2 128M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2,,? cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2,,? lazy_refcounts=off refcount_bits=16
Testing: create -f qcow2 -o backing_file=TEST_DIR/t.qcow2, -o help TEST_DIR/t.qcow2 128M
qemu-img: Invalid option list: backing_file=TEST_DIR/t.qcow2,
@@ -290,7 +290,7 @@ qemu-img: Format driver 'bochs' does not support image creation
=== convert: Options specified more than once ===
Testing: create -f qcow2 TEST_DIR/t.qcow2 128M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
Testing: convert -f foo -f qcow2 TEST_DIR/t.qcow2 TEST_DIR/t.qcow2.base
image: TEST_DIR/t.IMGFMT.base
@@ -639,205 +639,93 @@ cluster_size: 65536
=== amend: help for -o ===
Testing: amend -f qcow2 -o help TEST_DIR/t.qcow2
-Creation options for 'qcow2':
+Amend options for 'qcow2':
backing_file=<str> - File name of a base image
backing_fmt=<str> - Image format of the base image
- cluster_size=<size> - qcow2 cluster size
compat=<str> - Compatibility level (v2 [0.10] or v3 [1.1])
- compression_type=<str> - Compression method used for image cluster compression
data_file=<str> - File name of an external data file
data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
- encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
- encrypt.cipher-mode=<str> - Name of encryption cipher mode
- encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
- encrypt.hash-alg=<str> - Name of encryption hash algorithm
- encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
- encrypt.ivgen-alg=<str> - Name of IV generator algorithm
- encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
- encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
- encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
- preallocation=<str> - Preallocation mode (allowed values: off, metadata, falloc, full)
refcount_bits=<num> - Width of a reference count entry in bits
size=<size> - Virtual disk size
-Note that not all of these options may be amendable.
-
Testing: amend -f qcow2 -o ? TEST_DIR/t.qcow2
-Creation options for 'qcow2':
+Amend options for 'qcow2':
backing_file=<str> - File name of a base image
backing_fmt=<str> - Image format of the base image
- cluster_size=<size> - qcow2 cluster size
compat=<str> - Compatibility level (v2 [0.10] or v3 [1.1])
- compression_type=<str> - Compression method used for image cluster compression
data_file=<str> - File name of an external data file
data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
- encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
- encrypt.cipher-mode=<str> - Name of encryption cipher mode
- encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
- encrypt.hash-alg=<str> - Name of encryption hash algorithm
- encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
- encrypt.ivgen-alg=<str> - Name of IV generator algorithm
- encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
- encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
- encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
- preallocation=<str> - Preallocation mode (allowed values: off, metadata, falloc, full)
refcount_bits=<num> - Width of a reference count entry in bits
size=<size> - Virtual disk size
-Note that not all of these options may be amendable.
-
Testing: amend -f qcow2 -o cluster_size=4k,help TEST_DIR/t.qcow2
-Creation options for 'qcow2':
+Amend options for 'qcow2':
backing_file=<str> - File name of a base image
backing_fmt=<str> - Image format of the base image
- cluster_size=<size> - qcow2 cluster size
compat=<str> - Compatibility level (v2 [0.10] or v3 [1.1])
- compression_type=<str> - Compression method used for image cluster compression
data_file=<str> - File name of an external data file
data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
- encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
- encrypt.cipher-mode=<str> - Name of encryption cipher mode
- encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
- encrypt.hash-alg=<str> - Name of encryption hash algorithm
- encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
- encrypt.ivgen-alg=<str> - Name of IV generator algorithm
- encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
- encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
- encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
- preallocation=<str> - Preallocation mode (allowed values: off, metadata, falloc, full)
refcount_bits=<num> - Width of a reference count entry in bits
size=<size> - Virtual disk size
-Note that not all of these options may be amendable.
-
Testing: amend -f qcow2 -o cluster_size=4k,? TEST_DIR/t.qcow2
-Creation options for 'qcow2':
+Amend options for 'qcow2':
backing_file=<str> - File name of a base image
backing_fmt=<str> - Image format of the base image
- cluster_size=<size> - qcow2 cluster size
compat=<str> - Compatibility level (v2 [0.10] or v3 [1.1])
- compression_type=<str> - Compression method used for image cluster compression
data_file=<str> - File name of an external data file
data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
- encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
- encrypt.cipher-mode=<str> - Name of encryption cipher mode
- encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
- encrypt.hash-alg=<str> - Name of encryption hash algorithm
- encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
- encrypt.ivgen-alg=<str> - Name of IV generator algorithm
- encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
- encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
- encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
- preallocation=<str> - Preallocation mode (allowed values: off, metadata, falloc, full)
refcount_bits=<num> - Width of a reference count entry in bits
size=<size> - Virtual disk size
-Note that not all of these options may be amendable.
-
Testing: amend -f qcow2 -o help,cluster_size=4k TEST_DIR/t.qcow2
-Creation options for 'qcow2':
+Amend options for 'qcow2':
backing_file=<str> - File name of a base image
backing_fmt=<str> - Image format of the base image
- cluster_size=<size> - qcow2 cluster size
compat=<str> - Compatibility level (v2 [0.10] or v3 [1.1])
- compression_type=<str> - Compression method used for image cluster compression
data_file=<str> - File name of an external data file
data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
- encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
- encrypt.cipher-mode=<str> - Name of encryption cipher mode
- encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
- encrypt.hash-alg=<str> - Name of encryption hash algorithm
- encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
- encrypt.ivgen-alg=<str> - Name of IV generator algorithm
- encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
- encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
- encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
- preallocation=<str> - Preallocation mode (allowed values: off, metadata, falloc, full)
refcount_bits=<num> - Width of a reference count entry in bits
size=<size> - Virtual disk size
-Note that not all of these options may be amendable.
-
Testing: amend -f qcow2 -o ?,cluster_size=4k TEST_DIR/t.qcow2
-Creation options for 'qcow2':
+Amend options for 'qcow2':
backing_file=<str> - File name of a base image
backing_fmt=<str> - Image format of the base image
- cluster_size=<size> - qcow2 cluster size
compat=<str> - Compatibility level (v2 [0.10] or v3 [1.1])
- compression_type=<str> - Compression method used for image cluster compression
data_file=<str> - File name of an external data file
data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
- encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
- encrypt.cipher-mode=<str> - Name of encryption cipher mode
- encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
- encrypt.hash-alg=<str> - Name of encryption hash algorithm
- encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
- encrypt.ivgen-alg=<str> - Name of IV generator algorithm
- encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
- encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
- encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
- preallocation=<str> - Preallocation mode (allowed values: off, metadata, falloc, full)
refcount_bits=<num> - Width of a reference count entry in bits
size=<size> - Virtual disk size
-Note that not all of these options may be amendable.
-
Testing: amend -f qcow2 -o cluster_size=4k -o help TEST_DIR/t.qcow2
-Creation options for 'qcow2':
+Amend options for 'qcow2':
backing_file=<str> - File name of a base image
backing_fmt=<str> - Image format of the base image
- cluster_size=<size> - qcow2 cluster size
compat=<str> - Compatibility level (v2 [0.10] or v3 [1.1])
- compression_type=<str> - Compression method used for image cluster compression
data_file=<str> - File name of an external data file
data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
- encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
- encrypt.cipher-mode=<str> - Name of encryption cipher mode
- encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
- encrypt.hash-alg=<str> - Name of encryption hash algorithm
- encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
- encrypt.ivgen-alg=<str> - Name of IV generator algorithm
- encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
- encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
- encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
- preallocation=<str> - Preallocation mode (allowed values: off, metadata, falloc, full)
refcount_bits=<num> - Width of a reference count entry in bits
size=<size> - Virtual disk size
-Note that not all of these options may be amendable.
-
Testing: amend -f qcow2 -o cluster_size=4k -o ? TEST_DIR/t.qcow2
-Creation options for 'qcow2':
+Amend options for 'qcow2':
backing_file=<str> - File name of a base image
backing_fmt=<str> - Image format of the base image
- cluster_size=<size> - qcow2 cluster size
compat=<str> - Compatibility level (v2 [0.10] or v3 [1.1])
- compression_type=<str> - Compression method used for image cluster compression
data_file=<str> - File name of an external data file
data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
- encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
- encrypt.cipher-mode=<str> - Name of encryption cipher mode
- encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
- encrypt.hash-alg=<str> - Name of encryption hash algorithm
- encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
- encrypt.ivgen-alg=<str> - Name of IV generator algorithm
- encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
- encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
- encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
- preallocation=<str> - Preallocation mode (allowed values: off, metadata, falloc, full)
refcount_bits=<num> - Width of a reference count entry in bits
size=<size> - Virtual disk size
-Note that not all of these options may be amendable.
-
Testing: amend -f qcow2 -o backing_file=TEST_DIR/t.qcow2,,help TEST_DIR/t.qcow2
Testing: rebase -u -b -f qcow2 TEST_DIR/t.qcow2
@@ -856,30 +744,16 @@ Testing: amend -f qcow2 -o backing_file=TEST_DIR/t.qcow2 -o ,, -o help TEST_DIR/
qemu-img: Invalid option list: ,,
Testing: amend -f qcow2 -o help
-Creation options for 'qcow2':
+Amend options for 'qcow2':
backing_file=<str> - File name of a base image
backing_fmt=<str> - Image format of the base image
- cluster_size=<size> - qcow2 cluster size
compat=<str> - Compatibility level (v2 [0.10] or v3 [1.1])
- compression_type=<str> - Compression method used for image cluster compression
data_file=<str> - File name of an external data file
data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
- encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
- encrypt.cipher-mode=<str> - Name of encryption cipher mode
- encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
- encrypt.hash-alg=<str> - Name of encryption hash algorithm
- encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
- encrypt.ivgen-alg=<str> - Name of IV generator algorithm
- encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
- encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
- encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
- preallocation=<str> - Preallocation mode (allowed values: off, metadata, falloc, full)
refcount_bits=<num> - Width of a reference count entry in bits
size=<size> - Virtual disk size
-Note that not all of these options may be amendable.
-
Testing: amend -o help
qemu-img: Expecting one image file name
diff --git a/tests/qemu-iotests/085.out b/tests/qemu-iotests/085.out
index a822ff4ef6..d68c06efdf 100644
--- a/tests/qemu-iotests/085.out
+++ b/tests/qemu-iotests/085.out
@@ -13,7 +13,7 @@ Formatting 'TEST_DIR/t.IMGFMT.2', fmt=IMGFMT size=134217728
=== Create a single snapshot on virtio0 ===
{ 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'virtio0', 'snapshot-file':'TEST_DIR/1-snapshot-v0.IMGFMT', 'format': 'IMGFMT' } }
-Formatting 'TEST_DIR/1-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2.1 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/1-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2.1 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
{"return": {}}
=== Invalid command - missing device and nodename ===
@@ -30,40 +30,40 @@ Formatting 'TEST_DIR/1-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file
=== Create several transactional group snapshots ===
{ 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/2-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/2-snapshot-v1.IMGFMT' } } ] } }
-Formatting 'TEST_DIR/2-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/1-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
-Formatting 'TEST_DIR/2-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2.2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/2-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/1-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+Formatting 'TEST_DIR/2-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2.2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
{"return": {}}
{ 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/3-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/3-snapshot-v1.IMGFMT' } } ] } }
-Formatting 'TEST_DIR/3-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/2-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
-Formatting 'TEST_DIR/3-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/2-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/3-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/2-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+Formatting 'TEST_DIR/3-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/2-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
{"return": {}}
{ 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/4-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/4-snapshot-v1.IMGFMT' } } ] } }
-Formatting 'TEST_DIR/4-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/3-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
-Formatting 'TEST_DIR/4-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/3-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/4-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/3-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+Formatting 'TEST_DIR/4-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/3-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
{"return": {}}
{ 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/5-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/5-snapshot-v1.IMGFMT' } } ] } }
-Formatting 'TEST_DIR/5-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/4-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
-Formatting 'TEST_DIR/5-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/4-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/5-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/4-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+Formatting 'TEST_DIR/5-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/4-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
{"return": {}}
{ 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/6-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/6-snapshot-v1.IMGFMT' } } ] } }
-Formatting 'TEST_DIR/6-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/5-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
-Formatting 'TEST_DIR/6-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/5-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/6-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/5-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+Formatting 'TEST_DIR/6-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/5-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
{"return": {}}
{ 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/7-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/7-snapshot-v1.IMGFMT' } } ] } }
-Formatting 'TEST_DIR/7-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/6-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
-Formatting 'TEST_DIR/7-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/6-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/7-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/6-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+Formatting 'TEST_DIR/7-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/6-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
{"return": {}}
{ 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/8-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/8-snapshot-v1.IMGFMT' } } ] } }
-Formatting 'TEST_DIR/8-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/7-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
-Formatting 'TEST_DIR/8-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/7-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/8-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/7-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+Formatting 'TEST_DIR/8-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/7-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
{"return": {}}
{ 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/9-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/9-snapshot-v1.IMGFMT' } } ] } }
-Formatting 'TEST_DIR/9-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/8-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
-Formatting 'TEST_DIR/9-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/8-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/9-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/8-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+Formatting 'TEST_DIR/9-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/8-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
{"return": {}}
{ 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/10-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/10-snapshot-v1.IMGFMT' } } ] } }
-Formatting 'TEST_DIR/10-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/9-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
-Formatting 'TEST_DIR/10-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/9-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/10-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/9-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+Formatting 'TEST_DIR/10-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/9-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
{"return": {}}
=== Create a couple of snapshots using blockdev-snapshot ===
diff --git a/tests/qemu-iotests/144.out b/tests/qemu-iotests/144.out
index 885a8874a5..a2172a1308 100644
--- a/tests/qemu-iotests/144.out
+++ b/tests/qemu-iotests/144.out
@@ -9,7 +9,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=536870912
{ 'execute': 'qmp_capabilities' }
{"return": {}}
{ 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'virtio0', 'snapshot-file':'TEST_DIR/tmp.IMGFMT', 'format': 'IMGFMT' } }
-Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
{"return": {}}
=== Performing block-commit on active layer ===
@@ -31,6 +31,6 @@ Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 size=536870912 backing_file=TEST_DIR/
=== Performing Live Snapshot 2 ===
{ 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'virtio0', 'snapshot-file':'TEST_DIR/tmp2.IMGFMT', 'format': 'IMGFMT' } }
-Formatting 'TEST_DIR/tmp2.qcow2', fmt=qcow2 size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/tmp2.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
{"return": {}}
*** done
diff --git a/tests/qemu-iotests/182.out b/tests/qemu-iotests/182.out
index ae43654d32..29e9db3497 100644
--- a/tests/qemu-iotests/182.out
+++ b/tests/qemu-iotests/182.out
@@ -13,7 +13,7 @@ Is another process using the image [TEST_DIR/t.qcow2]?
{'execute': 'blockdev-add', 'arguments': { 'node-name': 'node0', 'driver': 'file', 'filename': 'TEST_DIR/t.IMGFMT', 'locking': 'on' } }
{"return": {}}
{'execute': 'blockdev-snapshot-sync', 'arguments': { 'node-name': 'node0', 'snapshot-file': 'TEST_DIR/t.IMGFMT.overlay', 'snapshot-node-name': 'node1' } }
-Formatting 'TEST_DIR/t.qcow2.overlay', fmt=qcow2 size=197120 backing_file=TEST_DIR/t.qcow2 backing_fmt=file cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2.overlay', fmt=qcow2 cluster_size=65536 compression_type=zlib size=197120 backing_file=TEST_DIR/t.qcow2 backing_fmt=file lazy_refcounts=off refcount_bits=16
{"return": {}}
{'execute': 'blockdev-add', 'arguments': { 'node-name': 'node1', 'driver': 'file', 'filename': 'TEST_DIR/t.IMGFMT', 'locking': 'on' } }
{"return": {}}
diff --git a/tests/qemu-iotests/185.out b/tests/qemu-iotests/185.out
index ac5ab16bc8..62d1ab74d3 100644
--- a/tests/qemu-iotests/185.out
+++ b/tests/qemu-iotests/185.out
@@ -9,14 +9,14 @@ Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
=== Creating backing chain ===
{ 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'disk', 'snapshot-file': 'TEST_DIR/t.IMGFMT.mid', 'format': 'IMGFMT', 'mode': 'absolute-paths' } }
-Formatting 'TEST_DIR/t.qcow2.mid', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.qcow2.base backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2.mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 backing_file=TEST_DIR/t.qcow2.base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
{"return": {}}
{ 'execute': 'human-monitor-command', 'arguments': { 'command-line': 'qemu-io disk "write 0 4M"' } }
wrote 4194304/4194304 bytes at offset 0
4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
{"return": ""}
{ 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'disk', 'snapshot-file': 'TEST_DIR/t.IMGFMT', 'format': 'IMGFMT', 'mode': 'absolute-paths' } }
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.qcow2.mid backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 backing_file=TEST_DIR/t.qcow2.mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
{"return": {}}
=== Start commit job and exit qemu ===
@@ -48,7 +48,7 @@ Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.q
{ 'execute': 'qmp_capabilities' }
{"return": {}}
{ 'execute': 'drive-mirror', 'arguments': { 'device': 'disk', 'target': 'TEST_DIR/t.IMGFMT.copy', 'format': 'IMGFMT', 'sync': 'full', 'speed': 65536 } }
-Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "disk"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}}
{"return": {}}
@@ -62,7 +62,7 @@ Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 l
{ 'execute': 'qmp_capabilities' }
{"return": {}}
{ 'execute': 'drive-backup', 'arguments': { 'device': 'disk', 'target': 'TEST_DIR/t.IMGFMT.copy', 'format': 'IMGFMT', 'sync': 'full', 'speed': 65536 } }
-Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "disk"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "paused", "id": "disk"}}
diff --git a/tests/qemu-iotests/255.out b/tests/qemu-iotests/255.out
index a3c99fd62e..d74903db99 100644
--- a/tests/qemu-iotests/255.out
+++ b/tests/qemu-iotests/255.out
@@ -3,9 +3,9 @@ Finishing a commit job with background reads
=== Create backing chain and start VM ===
-Formatting 'TEST_DIR/PID-t.qcow2.mid', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-t.qcow2.mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-t.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
=== Start background read requests ===
@@ -23,9 +23,9 @@ Closing the VM while a job is being cancelled
=== Create images and start VM ===
-Formatting 'TEST_DIR/PID-src.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-src.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-dst.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-dst.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
wrote 1048576/1048576 bytes at offset 0
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
diff --git a/tests/qemu-iotests/274.out b/tests/qemu-iotests/274.out
index d24ff681af..d248a1e21b 100644
--- a/tests/qemu-iotests/274.out
+++ b/tests/qemu-iotests/274.out
@@ -1,9 +1,9 @@
== Commit tests ==
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=2097152 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-mid', fmt=qcow2 size=1048576 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=2097152 backing_file=TEST_DIR/PID-mid cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid lazy_refcounts=off refcount_bits=16
wrote 2097152/2097152 bytes at offset 0
2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -64,11 +64,11 @@ read 1048576/1048576 bytes at offset 1048576
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
=== Testing HMP commit (top -> mid) ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=2097152 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-mid', fmt=qcow2 size=1048576 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=2097152 backing_file=TEST_DIR/PID-mid cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid lazy_refcounts=off refcount_bits=16
wrote 2097152/2097152 bytes at offset 0
2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -94,11 +94,11 @@ read 1048576/1048576 bytes at offset 1048576
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
=== Testing QMP active commit (top -> mid) ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=2097152 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-mid', fmt=qcow2 size=1048576 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=2097152 backing_file=TEST_DIR/PID-mid cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid lazy_refcounts=off refcount_bits=16
wrote 2097152/2097152 bytes at offset 0
2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -131,9 +131,9 @@ read 1048576/1048576 bytes at offset 1048576
== Resize tests ==
=== preallocation=off ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=6442450944 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=6442450944 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=1073741824 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1073741824 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
wrote 65536/65536 bytes at offset 5368709120
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -150,9 +150,9 @@ read 65536/65536 bytes at offset 5368709120
{ "start": 1073741824, "length": 7516192768, "depth": 0, "zero": true, "data": false}]
=== preallocation=metadata ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=34359738368 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=34359738368 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=32212254720 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=32212254720 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
wrote 65536/65536 bytes at offset 33285996544
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -174,9 +174,9 @@ read 65536/65536 bytes at offset 33285996544
{ "start": 34896609280, "length": 536870912, "depth": 0, "zero": true, "data": false, "offset": 2685075456}]
=== preallocation=falloc ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=10485760 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=10485760 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=5242880 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=5242880 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
wrote 65536/65536 bytes at offset 9437184
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -193,9 +193,9 @@ read 65536/65536 bytes at offset 9437184
{ "start": 5242880, "length": 10485760, "depth": 0, "zero": false, "data": true, "offset": 327680}]
=== preallocation=full ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=16777216 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=16777216 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=8388608 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=8388608 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
wrote 65536/65536 bytes at offset 11534336
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -212,9 +212,9 @@ read 65536/65536 bytes at offset 11534336
{ "start": 8388608, "length": 4194304, "depth": 0, "zero": false, "data": true, "offset": 327680}]
=== preallocation=off ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=393216 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=393216 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=259072 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=259072 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
wrote 65536/65536 bytes at offset 259072
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -232,9 +232,9 @@ read 65536/65536 bytes at offset 259072
{ "start": 262144, "length": 262144, "depth": 0, "zero": true, "data": false}]
=== preallocation=off ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=409600 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=409600 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=262144 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=262144 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
wrote 65536/65536 bytes at offset 344064
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -251,9 +251,9 @@ read 65536/65536 bytes at offset 344064
{ "start": 262144, "length": 262144, "depth": 0, "zero": true, "data": false}]
=== preallocation=off ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=524288 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=524288 lazy_refcounts=off refcount_bits=16
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=262144 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=262144 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
wrote 65536/65536 bytes at offset 446464
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
diff --git a/tests/qemu-iotests/280.out b/tests/qemu-iotests/280.out
index 92e4d14079..fc59b9bc5c 100644
--- a/tests/qemu-iotests/280.out
+++ b/tests/qemu-iotests/280.out
@@ -1,4 +1,4 @@
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
=== Launch VM ===
Enabling migration QMP events on VM...
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 14/31] block/crypto: rename two functions
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (12 preceding siblings ...)
2020-07-06 10:04 ` [PULL 13/31] block/amend: refactor qcow2 amend options Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 15/31] block/crypto: implement the encryption key management Max Reitz
` (18 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Maxim Levitsky <mlevitsk@redhat.com>
rename the write_func to create_write_func, and init_func to create_init_func.
This is preparation for other write_func that will be used to update the encryption keys.
No functional changes
Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200608094030.670121-7-mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/crypto.c | 25 ++++++++++++-------------
1 file changed, 12 insertions(+), 13 deletions(-)
diff --git a/block/crypto.c b/block/crypto.c
index dcf8b42bb2..1960b47ceb 100644
--- a/block/crypto.c
+++ b/block/crypto.c
@@ -79,12 +79,12 @@ struct BlockCryptoCreateData {
};
-static ssize_t block_crypto_write_func(QCryptoBlock *block,
- size_t offset,
- const uint8_t *buf,
- size_t buflen,
- void *opaque,
- Error **errp)
+static ssize_t block_crypto_create_write_func(QCryptoBlock *block,
+ size_t offset,
+ const uint8_t *buf,
+ size_t buflen,
+ void *opaque,
+ Error **errp)
{
struct BlockCryptoCreateData *data = opaque;
ssize_t ret;
@@ -97,11 +97,10 @@ static ssize_t block_crypto_write_func(QCryptoBlock *block,
return ret;
}
-
-static ssize_t block_crypto_init_func(QCryptoBlock *block,
- size_t headerlen,
- void *opaque,
- Error **errp)
+static ssize_t block_crypto_create_init_func(QCryptoBlock *block,
+ size_t headerlen,
+ void *opaque,
+ Error **errp)
{
struct BlockCryptoCreateData *data = opaque;
Error *local_error = NULL;
@@ -313,8 +312,8 @@ static int block_crypto_co_create_generic(BlockDriverState *bs,
};
crypto = qcrypto_block_create(opts, NULL,
- block_crypto_init_func,
- block_crypto_write_func,
+ block_crypto_create_init_func,
+ block_crypto_create_write_func,
&data,
errp);
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 15/31] block/crypto: implement the encryption key management
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (13 preceding siblings ...)
2020-07-06 10:04 ` [PULL 14/31] block/crypto: rename two functions Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 16/31] block/qcow2: extend qemu-img amend interface with crypto options Max Reitz
` (17 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Maxim Levitsky <mlevitsk@redhat.com>
This implements the encryption key management using the generic code in
qcrypto layer and exposes it to the user via qemu-img
This code adds another 'write_func' because the initialization
write_func works directly on the underlying file, and amend
works on instance of luks device.
This commit also adds a 'hack/workaround' I and Kevin Wolf (thanks)
made to make the driver both support write sharing (to avoid breaking the users),
and be safe against concurrent metadata update (the keyslots)
Eventually the write sharing for luks driver will be deprecated
and removed together with this hack.
The hack is that we ask (as a format driver) for BLK_PERM_CONSISTENT_READ
and then when we want to update the keys, we unshare that permission.
So if someone else has the image open, even readonly, encryption
key update will fail gracefully.
Also thanks to Daniel Berrange for the idea of
unsharing read, rather that write permission which allows
to avoid cases when the other user had opened the image read-only.
Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200608094030.670121-8-mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/crypto.h | 34 +++++++++++++
block/crypto.c | 130 +++++++++++++++++++++++++++++++++++++++++++++++--
2 files changed, 161 insertions(+), 3 deletions(-)
diff --git a/block/crypto.h b/block/crypto.h
index 06e044c9be..c72c3dec61 100644
--- a/block/crypto.h
+++ b/block/crypto.h
@@ -41,6 +41,11 @@
#define BLOCK_CRYPTO_OPT_LUKS_IVGEN_HASH_ALG "ivgen-hash-alg"
#define BLOCK_CRYPTO_OPT_LUKS_HASH_ALG "hash-alg"
#define BLOCK_CRYPTO_OPT_LUKS_ITER_TIME "iter-time"
+#define BLOCK_CRYPTO_OPT_LUKS_KEYSLOT "keyslot"
+#define BLOCK_CRYPTO_OPT_LUKS_STATE "state"
+#define BLOCK_CRYPTO_OPT_LUKS_OLD_SECRET "old-secret"
+#define BLOCK_CRYPTO_OPT_LUKS_NEW_SECRET "new-secret"
+
#define BLOCK_CRYPTO_OPT_DEF_LUKS_KEY_SECRET(prefix) \
BLOCK_CRYPTO_OPT_DEF_KEY_SECRET(prefix, \
@@ -88,6 +93,35 @@
.help = "Time to spend in PBKDF in milliseconds", \
}
+#define BLOCK_CRYPTO_OPT_DEF_LUKS_STATE(prefix) \
+ { \
+ .name = prefix BLOCK_CRYPTO_OPT_LUKS_STATE, \
+ .type = QEMU_OPT_STRING, \
+ .help = "Select new state of affected keyslots (active/inactive)",\
+ }
+
+#define BLOCK_CRYPTO_OPT_DEF_LUKS_KEYSLOT(prefix) \
+ { \
+ .name = prefix BLOCK_CRYPTO_OPT_LUKS_KEYSLOT, \
+ .type = QEMU_OPT_NUMBER, \
+ .help = "Select a single keyslot to modify explicitly",\
+ }
+
+#define BLOCK_CRYPTO_OPT_DEF_LUKS_OLD_SECRET(prefix) \
+ { \
+ .name = prefix BLOCK_CRYPTO_OPT_LUKS_OLD_SECRET, \
+ .type = QEMU_OPT_STRING, \
+ .help = "Select all keyslots that match this password", \
+ }
+
+#define BLOCK_CRYPTO_OPT_DEF_LUKS_NEW_SECRET(prefix) \
+ { \
+ .name = prefix BLOCK_CRYPTO_OPT_LUKS_NEW_SECRET, \
+ .type = QEMU_OPT_STRING, \
+ .help = "New secret to set in the matching keyslots. " \
+ "Empty string to erase", \
+ }
+
QCryptoBlockCreateOptions *
block_crypto_create_opts_init(QDict *opts, Error **errp);
diff --git a/block/crypto.c b/block/crypto.c
index 1960b47ceb..b9c40e6922 100644
--- a/block/crypto.c
+++ b/block/crypto.c
@@ -37,6 +37,7 @@ typedef struct BlockCrypto BlockCrypto;
struct BlockCrypto {
QCryptoBlock *block;
+ bool updating_keys;
};
@@ -71,6 +72,24 @@ static ssize_t block_crypto_read_func(QCryptoBlock *block,
return ret;
}
+static ssize_t block_crypto_write_func(QCryptoBlock *block,
+ size_t offset,
+ const uint8_t *buf,
+ size_t buflen,
+ void *opaque,
+ Error **errp)
+{
+ BlockDriverState *bs = opaque;
+ ssize_t ret;
+
+ ret = bdrv_pwrite(bs->file, offset, buf, buflen);
+ if (ret < 0) {
+ error_setg_errno(errp, -ret, "Could not write encryption header");
+ return ret;
+ }
+ return ret;
+}
+
struct BlockCryptoCreateData {
BlockBackend *blk;
@@ -166,6 +185,19 @@ static QemuOptsList block_crypto_create_opts_luks = {
};
+static QemuOptsList block_crypto_amend_opts_luks = {
+ .name = "crypto",
+ .head = QTAILQ_HEAD_INITIALIZER(block_crypto_create_opts_luks.head),
+ .desc = {
+ BLOCK_CRYPTO_OPT_DEF_LUKS_STATE(""),
+ BLOCK_CRYPTO_OPT_DEF_LUKS_KEYSLOT(""),
+ BLOCK_CRYPTO_OPT_DEF_LUKS_OLD_SECRET(""),
+ BLOCK_CRYPTO_OPT_DEF_LUKS_NEW_SECRET(""),
+ BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME(""),
+ { /* end of list */ }
+ },
+};
+
QCryptoBlockOpenOptions *
block_crypto_open_opts_init(QDict *opts, Error **errp)
{
@@ -758,6 +790,98 @@ block_crypto_get_specific_info_luks(BlockDriverState *bs, Error **errp)
return spec_info;
}
+static int
+block_crypto_amend_options_luks(BlockDriverState *bs,
+ QemuOpts *opts,
+ BlockDriverAmendStatusCB *status_cb,
+ void *cb_opaque,
+ bool force,
+ Error **errp)
+{
+ BlockCrypto *crypto = bs->opaque;
+ QDict *cryptoopts = NULL;
+ QCryptoBlockAmendOptions *amend_options = NULL;
+ int ret;
+
+ assert(crypto);
+ assert(crypto->block);
+ crypto->updating_keys = true;
+
+ ret = bdrv_child_refresh_perms(bs, bs->file, errp);
+ if (ret < 0) {
+ goto cleanup;
+ }
+
+ cryptoopts = qemu_opts_to_qdict(opts, NULL);
+ qdict_put_str(cryptoopts, "format", "luks");
+ amend_options = block_crypto_amend_opts_init(cryptoopts, errp);
+ if (!amend_options) {
+ ret = -EINVAL;
+ goto cleanup;
+ }
+
+ ret = qcrypto_block_amend_options(crypto->block,
+ block_crypto_read_func,
+ block_crypto_write_func,
+ bs,
+ amend_options,
+ force,
+ errp);
+cleanup:
+ crypto->updating_keys = false;
+ bdrv_child_refresh_perms(bs, bs->file, errp);
+ qapi_free_QCryptoBlockAmendOptions(amend_options);
+ qobject_unref(cryptoopts);
+ return ret;
+}
+
+
+static void
+block_crypto_child_perms(BlockDriverState *bs, BdrvChild *c,
+ const BdrvChildRole role,
+ BlockReopenQueue *reopen_queue,
+ uint64_t perm, uint64_t shared,
+ uint64_t *nperm, uint64_t *nshared)
+{
+
+ BlockCrypto *crypto = bs->opaque;
+
+ bdrv_default_perms(bs, c, role, reopen_queue, perm, shared, nperm, nshared);
+
+ /*
+ * For backward compatibility, manually share the write
+ * and resize permission
+ */
+ *nshared |= (BLK_PERM_WRITE | BLK_PERM_RESIZE);
+ /*
+ * Since we are not fully a format driver, don't always request
+ * the read/resize permission but only when explicitly
+ * requested
+ */
+ *nperm &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE);
+ *nperm |= perm & (BLK_PERM_WRITE | BLK_PERM_RESIZE);
+
+ /*
+ * This driver doesn't modify LUKS metadata except
+ * when updating the encryption slots.
+ * Thus unlike a proper format driver we don't ask for
+ * shared write/read permission. However we need it
+ * when we are updating the keys, to ensure that only we
+ * have access to the device.
+ *
+ * Encryption update will set the crypto->updating_keys
+ * during that period and refresh permissions
+ *
+ */
+ if (crypto->updating_keys) {
+ /* need exclusive write access for header update */
+ *nperm |= BLK_PERM_WRITE;
+ /* unshare read and write permission */
+ *nshared &= ~(BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE);
+ }
+}
+
+
static const char *const block_crypto_strong_runtime_opts[] = {
BLOCK_CRYPTO_OPT_LUKS_KEY_SECRET,
@@ -770,13 +894,12 @@ static BlockDriver bdrv_crypto_luks = {
.bdrv_probe = block_crypto_probe_luks,
.bdrv_open = block_crypto_open_luks,
.bdrv_close = block_crypto_close,
- /* This driver doesn't modify LUKS metadata except when creating image.
- * Allow share-rw=on as a special case. */
- .bdrv_child_perm = bdrv_default_perms,
+ .bdrv_child_perm = block_crypto_child_perms,
.bdrv_co_create = block_crypto_co_create_luks,
.bdrv_co_create_opts = block_crypto_co_create_opts_luks,
.bdrv_co_truncate = block_crypto_co_truncate,
.create_opts = &block_crypto_create_opts_luks,
+ .amend_opts = &block_crypto_amend_opts_luks,
.bdrv_reopen_prepare = block_crypto_reopen_prepare,
.bdrv_refresh_limits = block_crypto_refresh_limits,
@@ -786,6 +909,7 @@ static BlockDriver bdrv_crypto_luks = {
.bdrv_measure = block_crypto_measure,
.bdrv_get_info = block_crypto_get_info_luks,
.bdrv_get_specific_info = block_crypto_get_specific_info_luks,
+ .bdrv_amend_options = block_crypto_amend_options_luks,
.is_format = true,
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 16/31] block/qcow2: extend qemu-img amend interface with crypto options
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (14 preceding siblings ...)
2020-07-06 10:04 ` [PULL 15/31] block/crypto: implement the encryption key management Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 17/31] iotests: qemu-img tests for luks key management Max Reitz
` (16 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Maxim Levitsky <mlevitsk@redhat.com>
Now that we have all the infrastructure in place,
wire it in the qcow2 driver and expose this to the user.
Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200608094030.670121-9-mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/qcow2.c | 71 +++++++++++++++++++++++++++++++++-----
tests/qemu-iotests/082.out | 45 ++++++++++++++++++++++++
2 files changed, 107 insertions(+), 9 deletions(-)
diff --git a/block/qcow2.c b/block/qcow2.c
index 389205dffd..750e9c4842 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -176,6 +176,19 @@ static ssize_t qcow2_crypto_hdr_write_func(QCryptoBlock *block, size_t offset,
return ret;
}
+static QDict*
+qcow2_extract_crypto_opts(QemuOpts *opts, const char *fmt, Error **errp)
+{
+ QDict *cryptoopts_qdict;
+ QDict *opts_qdict;
+
+ /* Extract "encrypt." options into a qdict */
+ opts_qdict = qemu_opts_to_qdict(opts, NULL);
+ qdict_extract_subqdict(opts_qdict, &cryptoopts_qdict, "encrypt.");
+ qobject_unref(opts_qdict);
+ qdict_put_str(cryptoopts_qdict, "format", fmt);
+ return cryptoopts_qdict;
+}
/*
* read qcow2 extension and fill bs
@@ -4860,16 +4873,9 @@ static BlockMeasureInfo *qcow2_measure(QemuOpts *opts, BlockDriverState *in_bs,
if (has_luks) {
g_autoptr(QCryptoBlockCreateOptions) create_opts = NULL;
- QDict *opts_qdict;
- QDict *cryptoopts;
+ QDict *cryptoopts = qcow2_extract_crypto_opts(opts, "luks", errp);
size_t headerlen;
- opts_qdict = qemu_opts_to_qdict(opts, NULL);
- qdict_extract_subqdict(opts_qdict, &cryptoopts, "encrypt.");
- qobject_unref(opts_qdict);
-
- qdict_put_str(cryptoopts, "format", "luks");
-
create_opts = block_crypto_create_opts_init(cryptoopts, errp);
qobject_unref(cryptoopts);
if (!create_opts) {
@@ -5273,6 +5279,7 @@ typedef enum Qcow2AmendOperation {
QCOW2_NO_OPERATION = 0,
QCOW2_UPGRADING,
+ QCOW2_UPDATING_ENCRYPTION,
QCOW2_CHANGING_REFCOUNT_ORDER,
QCOW2_DOWNGRADING,
} Qcow2AmendOperation;
@@ -5354,6 +5361,7 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
int ret;
QemuOptDesc *desc = opts->list->desc;
Qcow2AmendHelperCBInfo helper_cb_info;
+ bool encryption_update = false;
while (desc && desc->name) {
if (!qemu_opt_find(opts, desc->name)) {
@@ -5380,6 +5388,18 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
backing_file = qemu_opt_get(opts, BLOCK_OPT_BACKING_FILE);
} else if (!strcmp(desc->name, BLOCK_OPT_BACKING_FMT)) {
backing_format = qemu_opt_get(opts, BLOCK_OPT_BACKING_FMT);
+ } else if (g_str_has_prefix(desc->name, "encrypt.")) {
+ if (!s->crypto) {
+ error_setg(errp,
+ "Can't amend encryption options - encryption not present");
+ return -EINVAL;
+ }
+ if (s->crypt_method_header != QCOW_CRYPT_LUKS) {
+ error_setg(errp,
+ "Only LUKS encryption options can be amended");
+ return -ENOTSUP;
+ }
+ encryption_update = true;
} else if (!strcmp(desc->name, BLOCK_OPT_LAZY_REFCOUNTS)) {
lazy_refcounts = qemu_opt_get_bool(opts, BLOCK_OPT_LAZY_REFCOUNTS,
lazy_refcounts);
@@ -5422,7 +5442,8 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
.original_status_cb = status_cb,
.original_cb_opaque = cb_opaque,
.total_operations = (new_version != old_version)
- + (s->refcount_bits != refcount_bits)
+ + (s->refcount_bits != refcount_bits) +
+ (encryption_update == true)
};
/* Upgrade first (some features may require compat=1.1) */
@@ -5435,6 +5456,33 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
}
}
+ if (encryption_update) {
+ QDict *amend_opts_dict;
+ QCryptoBlockAmendOptions *amend_opts;
+
+ helper_cb_info.current_operation = QCOW2_UPDATING_ENCRYPTION;
+ amend_opts_dict = qcow2_extract_crypto_opts(opts, "luks", errp);
+ if (!amend_opts_dict) {
+ return -EINVAL;
+ }
+ amend_opts = block_crypto_amend_opts_init(amend_opts_dict, errp);
+ qobject_unref(amend_opts_dict);
+ if (!amend_opts) {
+ return -EINVAL;
+ }
+ ret = qcrypto_block_amend_options(s->crypto,
+ qcow2_crypto_hdr_read_func,
+ qcow2_crypto_hdr_write_func,
+ bs,
+ amend_opts,
+ force,
+ errp);
+ qapi_free_QCryptoBlockAmendOptions(amend_opts);
+ if (ret < 0) {
+ return ret;
+ }
+ }
+
if (s->refcount_bits != refcount_bits) {
int refcount_order = ctz32(refcount_bits);
@@ -5694,6 +5742,11 @@ static QemuOptsList qcow2_amend_opts = {
.name = "qcow2-amend-opts",
.head = QTAILQ_HEAD_INITIALIZER(qcow2_amend_opts.head),
.desc = {
+ BLOCK_CRYPTO_OPT_DEF_LUKS_STATE("encrypt."),
+ BLOCK_CRYPTO_OPT_DEF_LUKS_KEYSLOT("encrypt."),
+ BLOCK_CRYPTO_OPT_DEF_LUKS_OLD_SECRET("encrypt."),
+ BLOCK_CRYPTO_OPT_DEF_LUKS_NEW_SECRET("encrypt."),
+ BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME("encrypt."),
QCOW_COMMON_OPTIONS,
{ /* end of list */ }
}
diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out
index b1cf5dfe43..a4a2b69030 100644
--- a/tests/qemu-iotests/082.out
+++ b/tests/qemu-iotests/082.out
@@ -645,6 +645,11 @@ Amend options for 'qcow2':
compat=<str> - Compatibility level (v2 [0.10] or v3 [1.1])
data_file=<str> - File name of an external data file
data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
+ encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
+ encrypt.keyslot=<num> - Select a single keyslot to modify explicitly
+ encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
+ encrypt.old-secret=<str> - Select all keyslots that match this password
+ encrypt.state=<str> - Select new state of affected keyslots (active/inactive)
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
refcount_bits=<num> - Width of a reference count entry in bits
size=<size> - Virtual disk size
@@ -656,6 +661,11 @@ Amend options for 'qcow2':
compat=<str> - Compatibility level (v2 [0.10] or v3 [1.1])
data_file=<str> - File name of an external data file
data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
+ encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
+ encrypt.keyslot=<num> - Select a single keyslot to modify explicitly
+ encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
+ encrypt.old-secret=<str> - Select all keyslots that match this password
+ encrypt.state=<str> - Select new state of affected keyslots (active/inactive)
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
refcount_bits=<num> - Width of a reference count entry in bits
size=<size> - Virtual disk size
@@ -667,6 +677,11 @@ Amend options for 'qcow2':
compat=<str> - Compatibility level (v2 [0.10] or v3 [1.1])
data_file=<str> - File name of an external data file
data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
+ encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
+ encrypt.keyslot=<num> - Select a single keyslot to modify explicitly
+ encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
+ encrypt.old-secret=<str> - Select all keyslots that match this password
+ encrypt.state=<str> - Select new state of affected keyslots (active/inactive)
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
refcount_bits=<num> - Width of a reference count entry in bits
size=<size> - Virtual disk size
@@ -678,6 +693,11 @@ Amend options for 'qcow2':
compat=<str> - Compatibility level (v2 [0.10] or v3 [1.1])
data_file=<str> - File name of an external data file
data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
+ encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
+ encrypt.keyslot=<num> - Select a single keyslot to modify explicitly
+ encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
+ encrypt.old-secret=<str> - Select all keyslots that match this password
+ encrypt.state=<str> - Select new state of affected keyslots (active/inactive)
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
refcount_bits=<num> - Width of a reference count entry in bits
size=<size> - Virtual disk size
@@ -689,6 +709,11 @@ Amend options for 'qcow2':
compat=<str> - Compatibility level (v2 [0.10] or v3 [1.1])
data_file=<str> - File name of an external data file
data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
+ encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
+ encrypt.keyslot=<num> - Select a single keyslot to modify explicitly
+ encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
+ encrypt.old-secret=<str> - Select all keyslots that match this password
+ encrypt.state=<str> - Select new state of affected keyslots (active/inactive)
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
refcount_bits=<num> - Width of a reference count entry in bits
size=<size> - Virtual disk size
@@ -700,6 +725,11 @@ Amend options for 'qcow2':
compat=<str> - Compatibility level (v2 [0.10] or v3 [1.1])
data_file=<str> - File name of an external data file
data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
+ encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
+ encrypt.keyslot=<num> - Select a single keyslot to modify explicitly
+ encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
+ encrypt.old-secret=<str> - Select all keyslots that match this password
+ encrypt.state=<str> - Select new state of affected keyslots (active/inactive)
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
refcount_bits=<num> - Width of a reference count entry in bits
size=<size> - Virtual disk size
@@ -711,6 +741,11 @@ Amend options for 'qcow2':
compat=<str> - Compatibility level (v2 [0.10] or v3 [1.1])
data_file=<str> - File name of an external data file
data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
+ encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
+ encrypt.keyslot=<num> - Select a single keyslot to modify explicitly
+ encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
+ encrypt.old-secret=<str> - Select all keyslots that match this password
+ encrypt.state=<str> - Select new state of affected keyslots (active/inactive)
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
refcount_bits=<num> - Width of a reference count entry in bits
size=<size> - Virtual disk size
@@ -722,6 +757,11 @@ Amend options for 'qcow2':
compat=<str> - Compatibility level (v2 [0.10] or v3 [1.1])
data_file=<str> - File name of an external data file
data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
+ encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
+ encrypt.keyslot=<num> - Select a single keyslot to modify explicitly
+ encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
+ encrypt.old-secret=<str> - Select all keyslots that match this password
+ encrypt.state=<str> - Select new state of affected keyslots (active/inactive)
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
refcount_bits=<num> - Width of a reference count entry in bits
size=<size> - Virtual disk size
@@ -750,6 +790,11 @@ Amend options for 'qcow2':
compat=<str> - Compatibility level (v2 [0.10] or v3 [1.1])
data_file=<str> - File name of an external data file
data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
+ encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
+ encrypt.keyslot=<num> - Select a single keyslot to modify explicitly
+ encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
+ encrypt.old-secret=<str> - Select all keyslots that match this password
+ encrypt.state=<str> - Select new state of affected keyslots (active/inactive)
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
refcount_bits=<num> - Width of a reference count entry in bits
size=<size> - Virtual disk size
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 17/31] iotests: qemu-img tests for luks key management
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (15 preceding siblings ...)
2020-07-06 10:04 ` [PULL 16/31] block/qcow2: extend qemu-img amend interface with crypto options Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 18/31] block/core: add generic infrastructure for x-blockdev-amend qmp command Max Reitz
` (15 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Maxim Levitsky <mlevitsk@redhat.com>
This commit adds two tests, which test the new amend interface
of both luks raw images and qcow2 luks encrypted images.
Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
[mreitz: Let 293 verify that LUKS works; drop $(seq) usage from 293;
drop 293 and 294 from the auto group]
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200625125548.870061-16-mreitz@redhat.com>
---
tests/qemu-iotests/293 | 208 +++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/293.out | 99 ++++++++++++++++++
tests/qemu-iotests/294 | 90 ++++++++++++++++
tests/qemu-iotests/294.out | 30 ++++++
tests/qemu-iotests/group | 2 +
5 files changed, 429 insertions(+)
create mode 100755 tests/qemu-iotests/293
create mode 100644 tests/qemu-iotests/293.out
create mode 100755 tests/qemu-iotests/294
create mode 100644 tests/qemu-iotests/294.out
diff --git a/tests/qemu-iotests/293 b/tests/qemu-iotests/293
new file mode 100755
index 0000000000..f86fe3b413
--- /dev/null
+++ b/tests/qemu-iotests/293
@@ -0,0 +1,208 @@
+#!/usr/bin/env bash
+#
+# Test encryption key management with luks
+# Based on 134
+#
+# Copyright (C) 2019 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+# creator
+owner=mlevitsk@redhat.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+status=1 # failure is the default!
+
+_cleanup()
+{
+ _cleanup_test_img
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_supported_fmt qcow2 luks
+_supported_proto file #TODO
+_require_working_luks
+
+QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT
+
+if [ "$IMGFMT" = "qcow2" ] ; then
+ PR="encrypt."
+ EXTRA_IMG_ARGS="-o encrypt.format=luks"
+fi
+
+
+# secrets: you are supposed to see the password as *******, see :-)
+S0="--object secret,id=sec0,data=hunter0"
+S1="--object secret,id=sec1,data=hunter1"
+S2="--object secret,id=sec2,data=hunter2"
+S3="--object secret,id=sec3,data=hunter3"
+S4="--object secret,id=sec4,data=hunter4"
+SECRETS="$S0 $S1 $S2 $S3 $S4"
+
+# image with given secret
+IMGS0="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec0"
+IMGS1="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec1"
+IMGS2="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec2"
+IMGS3="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec3"
+IMGS4="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec4"
+
+
+echo "== creating a test image =="
+_make_test_img $S0 $EXTRA_IMG_ARGS -o ${PR}key-secret=sec0,${PR}iter-time=10 32M
+
+echo
+echo "== test that key 0 opens the image =="
+$QEMU_IO $S0 -c "read 0 4096" $IMGS0 | _filter_qemu_io | _filter_testdir
+
+echo
+echo "== adding a password to slot 4 =="
+$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=active,${PR}new-secret=sec4,${PR}iter-time=10,${PR}keyslot=4
+echo "== adding a password to slot 1 =="
+$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=active,${PR}new-secret=sec1,${PR}iter-time=10
+echo "== adding a password to slot 3 =="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=active,${PR}new-secret=sec3,${PR}iter-time=10,${PR}keyslot=3
+
+echo "== adding a password to slot 2 =="
+$QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec2,${PR}iter-time=10
+
+
+echo "== erase slot 4 =="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=4 | _filter_img_create
+
+
+echo
+echo "== all secrets should work =="
+for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do
+ $QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir
+done
+
+echo
+echo "== erase slot 0 and try it =="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec0 | _filter_img_create
+$QEMU_IO $SECRETS -c "read 0 4096" $IMGS0 | _filter_qemu_io | _filter_testdir
+
+echo
+echo "== erase slot 2 and try it =="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=2 | _filter_img_create
+$QEMU_IO $SECRETS -c "read 0 4096" $IMGS2 | _filter_qemu_io | _filter_testdir
+
+
+# at this point slots 1 and 3 should be active
+
+echo
+echo "== filling 4 slots with secret 2 =="
+for ((i = 0; i < 4; i++)); do
+ $QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec2,${PR}iter-time=10
+done
+
+echo
+echo "== adding secret 0 =="
+ $QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec0,${PR}iter-time=10
+
+echo
+echo "== adding secret 3 (last slot) =="
+ $QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec3,${PR}iter-time=10
+
+echo
+echo "== trying to add another slot (should fail) =="
+$QEMU_IMG amend $SECRETS $IMGS2 -o ${PR}state=active,${PR}new-secret=sec3,${PR}iter-time=10
+
+echo
+echo "== all secrets should work again =="
+for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do
+ $QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir
+done
+
+
+echo
+
+echo "== erase all keys of secret 2=="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec2
+
+echo "== erase all keys of secret 1=="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec1
+
+echo "== erase all keys of secret 0=="
+$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=inactive,${PR}old-secret=sec0
+
+echo "== erasing secret3 will fail now since it is the only secret (in 3 slots) =="
+$QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=inactive,${PR}old-secret=sec3
+
+echo
+echo "== only secret3 should work now =="
+for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do
+ $QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir
+done
+
+echo
+echo "== add secret0 =="
+$QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec0,${PR}iter-time=10
+
+echo "== erase secret3 =="
+$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=inactive,${PR}old-secret=sec3
+
+echo
+echo "== only secret0 should work now =="
+for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do
+ $QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir
+done
+
+echo
+echo "== replace secret0 with secret1 (should fail) =="
+$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=active,${PR}new-secret=sec1,${PR}keyslot=0
+
+echo
+echo "== replace secret0 with secret1 with force (should work) =="
+$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=active,${PR}new-secret=sec1,${PR}iter-time=10,${PR}keyslot=0 --force
+
+echo
+echo "== only secret1 should work now =="
+for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do
+ $QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir
+done
+
+
+echo
+echo "== erase last secret (should fail) =="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=0
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec1
+
+
+echo "== erase non existing secrets (should fail) =="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec5 --force
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec0 --force
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=1 --force
+
+echo
+echo "== erase last secret with force by slot (should work) =="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=0 --force
+
+echo
+echo "== we have no secrets now, data is lost forever =="
+for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do
+ $QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir
+done
+
+# success, all done
+echo "*** done"
+rm -f $seq.full
+status=0
+
diff --git a/tests/qemu-iotests/293.out b/tests/qemu-iotests/293.out
new file mode 100644
index 0000000000..7260783126
--- /dev/null
+++ b/tests/qemu-iotests/293.out
@@ -0,0 +1,99 @@
+QA output created by 293
+== creating a test image ==
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33554432
+
+== test that key 0 opens the image ==
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== adding a password to slot 4 ==
+== adding a password to slot 1 ==
+== adding a password to slot 3 ==
+== adding a password to slot 2 ==
+== erase slot 4 ==
+
+== all secrets should work ==
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== erase slot 0 and try it ==
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+
+== erase slot 2 and try it ==
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+
+== filling 4 slots with secret 2 ==
+
+== adding secret 0 ==
+
+== adding secret 3 (last slot) ==
+
+== trying to add another slot (should fail) ==
+qemu-img: Can't add a keyslot - all keyslots are in use
+
+== all secrets should work again ==
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== erase all keys of secret 2==
+== erase all keys of secret 1==
+== erase all keys of secret 0==
+== erasing secret3 will fail now since it is the only secret (in 3 slots) ==
+qemu-img: All the active keyslots match the (old) password that was given and erasing them will erase all the data in the image irreversibly - refusing operation
+
+== only secret3 should work now ==
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== add secret0 ==
+== erase secret3 ==
+
+== only secret0 should work now ==
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+
+== replace secret0 with secret1 (should fail) ==
+qemu-img: Refusing to overwrite active keyslot 0 - please erase it first
+
+== replace secret0 with secret1 with force (should work) ==
+
+== only secret1 should work now ==
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+
+== erase last secret (should fail) ==
+qemu-img: Attempt to erase the only active keyslot 0 which will erase all the data in the image irreversibly - refusing operation
+qemu-img: All the active keyslots match the (old) password that was given and erasing them will erase all the data in the image irreversibly - refusing operation
+== erase non existing secrets (should fail) ==
+qemu-img: No secret with id 'sec5'
+qemu-img: No keyslots match given (old) password for erase operation
+
+== erase last secret with force by slot (should work) ==
+
+== we have no secrets now, data is lost forever ==
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+*** done
diff --git a/tests/qemu-iotests/294 b/tests/qemu-iotests/294
new file mode 100755
index 0000000000..9c95ed8c9a
--- /dev/null
+++ b/tests/qemu-iotests/294
@@ -0,0 +1,90 @@
+#
+# Copyright (C) 2019 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+# creator
+owner=mlevitsk@redhat.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+status=1 # failure is the default!
+
+_cleanup()
+{
+ _cleanup_test_img
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_supported_fmt luks
+_supported_proto file #TODO
+
+QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT
+
+# you are supposed to see the password as *******, see :-)
+S0="--object secret,id=sec0,data=hunter0"
+S1="--object secret,id=sec1,data=hunter1"
+SECRETS="$S0 $S1"
+
+
+IMGS0="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,key-secret=sec0"
+IMGS1="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,key-secret=sec1"
+
+echo "== creating a test image =="
+_make_test_img $S0 -o "key-secret=sec0,iter-time=10" 32M
+
+echo
+echo "== test that key 0 opens the image =="
+$QEMU_IO $S0 -c "read 0 4096" $IMGS0 | _filter_qemu_io | _filter_testdir
+
+echo
+echo "== adding a password to slot 1 =="
+$QEMU_IMG amend $SECRETS $IMGS0 -o state=active,new-secret=sec1,keyslot=1,iter-time=10
+
+echo
+echo "== 'backup' the image header =="
+dd if=$TEST_IMG_FILE of=${TEST_IMG_FILE}.bk bs=4K skip=0 count=1
+
+echo
+echo "== erase slot 0 =="
+$QEMU_IMG amend $SECRETS $IMGS1 -o state=inactive,keyslot=0 | _filter_img_create
+
+echo
+echo "== test that key 0 doesn't open the image =="
+$QEMU_IO $S0 -c "read 0 4096" $IMGS0 | _filter_qemu_io | _filter_testdir
+
+echo
+echo "== 'restore' the image header =="
+dd if=${TEST_IMG_FILE}.bk of=${TEST_IMG_FILE} bs=4K skip=0 count=1 conv=notrunc
+
+echo
+echo "== test that key 0 still doesn't open the image (key material is erased) =="
+$QEMU_IO $SECRETS -c "read 0 4096" $IMGS0 | _filter_qemu_io | _filter_testdir
+
+echo
+echo "== test that key 1 still works =="
+$QEMU_IO $SECRETS -c "read 0 4096" $IMGS1 | _filter_qemu_io | _filter_testdir
+
+echo "*** done"
+rm -f $seq.full
+status=0
+
+
+exit 0
diff --git a/tests/qemu-iotests/294.out b/tests/qemu-iotests/294.out
new file mode 100644
index 0000000000..994ae87308
--- /dev/null
+++ b/tests/qemu-iotests/294.out
@@ -0,0 +1,30 @@
+QA output created by 294
+== creating a test image ==
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33554432
+
+== test that key 0 opens the image ==
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== adding a password to slot 1 ==
+
+== 'backup' the image header ==
+1+0 records in
+1+0 records out
+
+== erase slot 0 ==
+
+== test that key 0 doesn't open the image ==
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+
+== 'restore' the image header ==
+1+0 records in
+1+0 records out
+
+== test that key 0 still doesn't open the image (key material is erased) ==
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+
+== test that key 1 still works ==
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+*** done
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index d886fa0cb3..b945dd4f20 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -301,4 +301,6 @@
290 rw auto quick
291 rw quick
292 rw auto quick
+293 rw
+294 rw quick
297 meta
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 18/31] block/core: add generic infrastructure for x-blockdev-amend qmp command
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (16 preceding siblings ...)
2020-07-06 10:04 ` [PULL 17/31] iotests: qemu-img tests for luks key management Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-09 15:09 ` Peter Maydell
2020-07-06 10:04 ` [PULL 19/31] block/crypto: implement blockdev-amend Max Reitz
` (14 subsequent siblings)
32 siblings, 1 reply; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Maxim Levitsky <mlevitsk@redhat.com>
blockdev-amend will be used similiar to blockdev-create
to allow on the fly changes of the structure of the format based block devices.
Current plan is to first support encryption keyslot management for luks
based formats (raw and embedded in qcow2)
Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200608094030.670121-12-mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
qapi/block-core.json | 42 ++++++++++++++
qapi/job.json | 4 +-
include/block/block_int.h | 21 +++++--
block/amend.c | 113 ++++++++++++++++++++++++++++++++++++++
block/Makefile.objs | 2 +-
5 files changed, 174 insertions(+), 8 deletions(-)
create mode 100644 block/amend.c
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 0e1c6a59f2..c22996282f 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -4674,6 +4674,48 @@
'data': { 'job-id': 'str',
'options': 'BlockdevCreateOptions' } }
+##
+# @BlockdevAmendOptions:
+#
+# Options for amending an image format
+#
+# @driver: Block driver of the node to amend.
+#
+# Since: 5.1
+##
+{ 'union': 'BlockdevAmendOptions',
+ 'base': {
+ 'driver': 'BlockdevDriver' },
+ 'discriminator': 'driver',
+ 'data': {
+ } }
+
+##
+# @x-blockdev-amend:
+#
+# Starts a job to amend format specific options of an existing open block device
+# The job is automatically finalized, but a manual job-dismiss is required.
+#
+# @job-id: Identifier for the newly created job.
+#
+# @node-name: Name of the block node to work on
+#
+# @options: Options (driver specific)
+#
+# @force: Allow unsafe operations, format specific
+# For luks that allows erase of the last active keyslot
+# (permanent loss of data),
+# and replacement of an active keyslot
+# (possible loss of data if IO error happens)
+#
+# Since: 5.1
+##
+{ 'command': 'x-blockdev-amend',
+ 'data': { 'job-id': 'str',
+ 'node-name': 'str',
+ 'options': 'BlockdevAmendOptions',
+ '*force': 'bool' } }
+
##
# @BlockErrorAction:
#
diff --git a/qapi/job.json b/qapi/job.json
index 5e658281f5..c48a0c3e34 100644
--- a/qapi/job.json
+++ b/qapi/job.json
@@ -19,10 +19,12 @@
#
# @create: image creation job type, see "blockdev-create" (since 3.0)
#
+# @amend: image options amend job type, see "x-blockdev-amend" (since 5.1)
+#
# Since: 1.7
##
{ 'enum': 'JobType',
- 'data': ['commit', 'stream', 'mirror', 'backup', 'create'] }
+ 'data': ['commit', 'stream', 'mirror', 'backup', 'create', 'amend'] }
##
# @JobStatus:
diff --git a/include/block/block_int.h b/include/block/block_int.h
index ed335519cc..1b86b59af1 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -141,12 +141,27 @@ struct BlockDriver {
int (*bdrv_file_open)(BlockDriverState *bs, QDict *options, int flags,
Error **errp);
void (*bdrv_close)(BlockDriverState *bs);
+
+
int coroutine_fn (*bdrv_co_create)(BlockdevCreateOptions *opts,
Error **errp);
int coroutine_fn (*bdrv_co_create_opts)(BlockDriver *drv,
const char *filename,
QemuOpts *opts,
Error **errp);
+
+ int coroutine_fn (*bdrv_co_amend)(BlockDriverState *bs,
+ BlockdevAmendOptions *opts,
+ bool force,
+ Error **errp);
+
+ int (*bdrv_amend_options)(BlockDriverState *bs,
+ QemuOpts *opts,
+ BlockDriverAmendStatusCB *status_cb,
+ void *cb_opaque,
+ bool force,
+ Error **errp);
+
int (*bdrv_make_empty)(BlockDriverState *bs);
/*
@@ -441,12 +456,6 @@ struct BlockDriver {
BdrvCheckResult *result,
BdrvCheckMode fix);
- int (*bdrv_amend_options)(BlockDriverState *bs, QemuOpts *opts,
- BlockDriverAmendStatusCB *status_cb,
- void *cb_opaque,
- bool force,
- Error **errp);
-
void (*bdrv_debug_event)(BlockDriverState *bs, BlkdebugEvent event);
/* TODO Better pass a option string/QDict/QemuOpts to add any rule? */
diff --git a/block/amend.c b/block/amend.c
new file mode 100644
index 0000000000..f4612dcf08
--- /dev/null
+++ b/block/amend.c
@@ -0,0 +1,113 @@
+/*
+ * Block layer code related to image options amend
+ *
+ * Copyright (c) 2018 Kevin Wolf <kwolf@redhat.com>
+ * Copyright (c) 2020 Red Hat. Inc
+ *
+ * Heavily based on create.c
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "qemu/osdep.h"
+#include "block/block_int.h"
+#include "qemu/job.h"
+#include "qemu/main-loop.h"
+#include "qapi/qapi-commands-block-core.h"
+#include "qapi/qapi-visit-block-core.h"
+#include "qapi/clone-visitor.h"
+#include "qapi/error.h"
+
+typedef struct BlockdevAmendJob {
+ Job common;
+ BlockdevAmendOptions *opts;
+ BlockDriverState *bs;
+ bool force;
+} BlockdevAmendJob;
+
+static int coroutine_fn blockdev_amend_run(Job *job, Error **errp)
+{
+ BlockdevAmendJob *s = container_of(job, BlockdevAmendJob, common);
+ int ret;
+
+ job_progress_set_remaining(&s->common, 1);
+ ret = s->bs->drv->bdrv_co_amend(s->bs, s->opts, s->force, errp);
+ job_progress_update(&s->common, 1);
+ qapi_free_BlockdevAmendOptions(s->opts);
+ return ret;
+}
+
+static const JobDriver blockdev_amend_job_driver = {
+ .instance_size = sizeof(BlockdevAmendJob),
+ .job_type = JOB_TYPE_AMEND,
+ .run = blockdev_amend_run,
+};
+
+void qmp_x_blockdev_amend(const char *job_id,
+ const char *node_name,
+ BlockdevAmendOptions *options,
+ bool has_force,
+ bool force,
+ Error **errp)
+{
+ BlockdevAmendJob *s;
+ const char *fmt = BlockdevDriver_str(options->driver);
+ BlockDriver *drv = bdrv_find_format(fmt);
+ BlockDriverState *bs = bdrv_find_node(node_name);
+
+
+ if (!drv) {
+ error_setg(errp, "Block driver '%s' not found or not supported", fmt);
+ return;
+ }
+
+ /*
+ * If the driver is in the schema, we know that it exists. But it may not
+ * be whitelisted.
+ */
+ if (bdrv_uses_whitelist() && !bdrv_is_whitelisted(drv, false)) {
+ error_setg(errp, "Driver is not whitelisted");
+ return;
+ }
+
+ if (bs->drv != drv) {
+ error_setg(errp,
+ "x-blockdev-amend doesn't support changing the block driver");
+ return;
+ }
+
+ /* Error out if the driver doesn't support .bdrv_co_amend */
+ if (!drv->bdrv_co_amend) {
+ error_setg(errp, "Driver does not support x-blockdev-amend");
+ return;
+ }
+
+ /* Create the block job */
+ s = job_create(job_id, &blockdev_amend_job_driver, NULL,
+ bdrv_get_aio_context(bs), JOB_DEFAULT | JOB_MANUAL_DISMISS,
+ NULL, NULL, errp);
+ if (!s) {
+ return;
+ }
+
+ s->bs = bs,
+ s->opts = QAPI_CLONE(BlockdevAmendOptions, options),
+ s->force = has_force ? force : false;
+ job_start(&s->common);
+}
diff --git a/block/Makefile.objs b/block/Makefile.objs
index 96028eedce..577e578bc2 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -19,7 +19,7 @@ block-obj-$(CONFIG_WIN32) += file-win32.o win32-aio.o
block-obj-$(CONFIG_POSIX) += file-posix.o
block-obj-$(CONFIG_LINUX_AIO) += linux-aio.o
block-obj-$(CONFIG_LINUX_IO_URING) += io_uring.o
-block-obj-y += null.o mirror.o commit.o io.o create.o
+block-obj-y += null.o mirror.o commit.o io.o create.o amend.o
block-obj-y += throttle-groups.o
block-obj-$(CONFIG_LINUX) += nvme.o
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* Re: [PULL 18/31] block/core: add generic infrastructure for x-blockdev-amend qmp command
2020-07-06 10:04 ` [PULL 18/31] block/core: add generic infrastructure for x-blockdev-amend qmp command Max Reitz
@ 2020-07-09 15:09 ` Peter Maydell
2020-07-10 9:54 ` Max Reitz
0 siblings, 1 reply; 45+ messages in thread
From: Peter Maydell @ 2020-07-09 15:09 UTC (permalink / raw)
To: Max Reitz; +Cc: Kevin Wolf, QEMU Developers, Qemu-block
On Mon, 6 Jul 2020 at 11:05, Max Reitz <mreitz@redhat.com> wrote:
>
> From: Maxim Levitsky <mlevitsk@redhat.com>
>
> blockdev-amend will be used similiar to blockdev-create
> to allow on the fly changes of the structure of the format based block devices.
>
> Current plan is to first support encryption keyslot management for luks
> based formats (raw and embedded in qcow2)
>
> Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
> Message-Id: <20200608094030.670121-12-mlevitsk@redhat.com>
> Signed-off-by: Max Reitz <mreitz@redhat.com>
Hi; Coverity reports a possible issue with this function
(CID 1430268):
> +void qmp_x_blockdev_amend(const char *job_id,
> + const char *node_name,
> + BlockdevAmendOptions *options,
> + bool has_force,
> + bool force,
> + Error **errp)
> +{
> + BlockdevAmendJob *s;
> + const char *fmt = BlockdevDriver_str(options->driver);
> + BlockDriver *drv = bdrv_find_format(fmt);
> + BlockDriverState *bs = bdrv_find_node(node_name);
bdrv_find_node() can return NULL (we check for this
in almost all callsites)...
> + if (bs->drv != drv) {
...but here we dereference it unconditionally.
> + error_setg(errp,
> + "x-blockdev-amend doesn't support changing the block driver");
> + return;
> + }
thanks
-- PMM
^ permalink raw reply [flat|nested] 45+ messages in thread* Re: [PULL 18/31] block/core: add generic infrastructure for x-blockdev-amend qmp command
2020-07-09 15:09 ` Peter Maydell
@ 2020-07-10 9:54 ` Max Reitz
0 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-10 9:54 UTC (permalink / raw)
To: Peter Maydell; +Cc: Kevin Wolf, QEMU Developers, Qemu-block
[-- Attachment #1.1: Type: text/plain, Size: 1728 bytes --]
On 09.07.20 17:09, Peter Maydell wrote:
> On Mon, 6 Jul 2020 at 11:05, Max Reitz <mreitz@redhat.com> wrote:
>>
>> From: Maxim Levitsky <mlevitsk@redhat.com>
>>
>> blockdev-amend will be used similiar to blockdev-create
>> to allow on the fly changes of the structure of the format based block devices.
>>
>> Current plan is to first support encryption keyslot management for luks
>> based formats (raw and embedded in qcow2)
>>
>> Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
>> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
>> Message-Id: <20200608094030.670121-12-mlevitsk@redhat.com>
>> Signed-off-by: Max Reitz <mreitz@redhat.com>
>
> Hi; Coverity reports a possible issue with this function
> (CID 1430268):
Thanks for the heads-up, I’ve sent a patch (“block/amend: Check whether
the node exists”).
>> +void qmp_x_blockdev_amend(const char *job_id,
>> + const char *node_name,
>> + BlockdevAmendOptions *options,
>> + bool has_force,
>> + bool force,
>> + Error **errp)
>> +{
>> + BlockdevAmendJob *s;
>> + const char *fmt = BlockdevDriver_str(options->driver);
>> + BlockDriver *drv = bdrv_find_format(fmt);
>> + BlockDriverState *bs = bdrv_find_node(node_name);
>
> bdrv_find_node() can return NULL (we check for this
> in almost all callsites)...
>
>> + if (bs->drv != drv) {
>
> ...but here we dereference it unconditionally.
>
>> + error_setg(errp,
>> + "x-blockdev-amend doesn't support changing the block driver");
>> + return;
>> + }
>
> thanks
> -- PMM
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 45+ messages in thread
* [PULL 19/31] block/crypto: implement blockdev-amend
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (17 preceding siblings ...)
2020-07-06 10:04 ` [PULL 18/31] block/core: add generic infrastructure for x-blockdev-amend qmp command Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 20/31] block/qcow2: " Max Reitz
` (13 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200608094030.670121-13-mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
qapi/block-core.json | 14 ++++++++-
block/crypto.c | 72 ++++++++++++++++++++++++++++++++------------
2 files changed, 66 insertions(+), 20 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index c22996282f..cd679ad435 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -4674,6 +4674,18 @@
'data': { 'job-id': 'str',
'options': 'BlockdevCreateOptions' } }
+##
+# @BlockdevAmendOptionsLUKS:
+#
+# Driver specific image amend options for LUKS.
+#
+# Since: 5.1
+##
+{ 'struct': 'BlockdevAmendOptionsLUKS',
+ 'base': 'QCryptoBlockAmendOptionsLUKS',
+ 'data': { }
+}
+
##
# @BlockdevAmendOptions:
#
@@ -4688,7 +4700,7 @@
'driver': 'BlockdevDriver' },
'discriminator': 'driver',
'data': {
- } }
+ 'luks': 'BlockdevAmendOptionsLUKS' } }
##
# @x-blockdev-amend:
diff --git a/block/crypto.c b/block/crypto.c
index b9c40e6922..3fbd68832b 100644
--- a/block/crypto.c
+++ b/block/crypto.c
@@ -791,32 +791,21 @@ block_crypto_get_specific_info_luks(BlockDriverState *bs, Error **errp)
}
static int
-block_crypto_amend_options_luks(BlockDriverState *bs,
- QemuOpts *opts,
- BlockDriverAmendStatusCB *status_cb,
- void *cb_opaque,
- bool force,
- Error **errp)
+block_crypto_amend_options_generic_luks(BlockDriverState *bs,
+ QCryptoBlockAmendOptions *amend_options,
+ bool force,
+ Error **errp)
{
BlockCrypto *crypto = bs->opaque;
- QDict *cryptoopts = NULL;
- QCryptoBlockAmendOptions *amend_options = NULL;
int ret;
assert(crypto);
assert(crypto->block);
- crypto->updating_keys = true;
+ /* apply for exclusive read/write permissions to the underlying file*/
+ crypto->updating_keys = true;
ret = bdrv_child_refresh_perms(bs, bs->file, errp);
- if (ret < 0) {
- goto cleanup;
- }
-
- cryptoopts = qemu_opts_to_qdict(opts, NULL);
- qdict_put_str(cryptoopts, "format", "luks");
- amend_options = block_crypto_amend_opts_init(cryptoopts, errp);
- if (!amend_options) {
- ret = -EINVAL;
+ if (ret) {
goto cleanup;
}
@@ -828,13 +817,57 @@ block_crypto_amend_options_luks(BlockDriverState *bs,
force,
errp);
cleanup:
+ /* release exclusive read/write permissions to the underlying file*/
crypto->updating_keys = false;
bdrv_child_refresh_perms(bs, bs->file, errp);
- qapi_free_QCryptoBlockAmendOptions(amend_options);
+ return ret;
+}
+
+static int
+block_crypto_amend_options_luks(BlockDriverState *bs,
+ QemuOpts *opts,
+ BlockDriverAmendStatusCB *status_cb,
+ void *cb_opaque,
+ bool force,
+ Error **errp)
+{
+ BlockCrypto *crypto = bs->opaque;
+ QDict *cryptoopts = NULL;
+ QCryptoBlockAmendOptions *amend_options = NULL;
+ int ret = -EINVAL;
+
+ assert(crypto);
+ assert(crypto->block);
+
+ cryptoopts = qemu_opts_to_qdict(opts, NULL);
+ qdict_put_str(cryptoopts, "format", "luks");
+ amend_options = block_crypto_amend_opts_init(cryptoopts, errp);
qobject_unref(cryptoopts);
+ if (!amend_options) {
+ goto cleanup;
+ }
+ ret = block_crypto_amend_options_generic_luks(bs, amend_options,
+ force, errp);
+cleanup:
+ qapi_free_QCryptoBlockAmendOptions(amend_options);
return ret;
}
+static int
+coroutine_fn block_crypto_co_amend_luks(BlockDriverState *bs,
+ BlockdevAmendOptions *opts,
+ bool force,
+ Error **errp)
+{
+ QCryptoBlockAmendOptions amend_opts;
+
+ amend_opts = (QCryptoBlockAmendOptions) {
+ .format = Q_CRYPTO_BLOCK_FORMAT_LUKS,
+ .u.luks = *qapi_BlockdevAmendOptionsLUKS_base(&opts->u.luks),
+ };
+ return block_crypto_amend_options_generic_luks(bs, &amend_opts,
+ force, errp);
+}
static void
block_crypto_child_perms(BlockDriverState *bs, BdrvChild *c,
@@ -910,6 +943,7 @@ static BlockDriver bdrv_crypto_luks = {
.bdrv_get_info = block_crypto_get_info_luks,
.bdrv_get_specific_info = block_crypto_get_specific_info_luks,
.bdrv_amend_options = block_crypto_amend_options_luks,
+ .bdrv_co_amend = block_crypto_co_amend_luks,
.is_format = true,
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 20/31] block/qcow2: implement blockdev-amend
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (18 preceding siblings ...)
2020-07-06 10:04 ` [PULL 19/31] block/crypto: implement blockdev-amend Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 21/31] iotests: add tests for blockdev-amend Max Reitz
` (12 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Maxim Levitsky <mlevitsk@redhat.com>
Currently the implementation only supports amending the encryption
options, unlike the qemu-img version
Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200608094030.670121-14-mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
qapi/block-core.json | 16 +++++++++++++++-
block/qcow2.c | 39 +++++++++++++++++++++++++++++++++++++++
2 files changed, 54 insertions(+), 1 deletion(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index cd679ad435..b20332e592 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -4686,6 +4686,19 @@
'data': { }
}
+##
+# @BlockdevAmendOptionsQcow2:
+#
+# Driver specific image amend options for qcow2.
+# For now, only encryption options can be amended
+#
+# @encrypt Encryption options to be amended
+#
+# Since: 5.1
+##
+{ 'struct': 'BlockdevAmendOptionsQcow2',
+ 'data': { '*encrypt': 'QCryptoBlockAmendOptions' } }
+
##
# @BlockdevAmendOptions:
#
@@ -4700,7 +4713,8 @@
'driver': 'BlockdevDriver' },
'discriminator': 'driver',
'data': {
- 'luks': 'BlockdevAmendOptionsLUKS' } }
+ 'luks': 'BlockdevAmendOptionsLUKS',
+ 'qcow2': 'BlockdevAmendOptionsQcow2' } }
##
# @x-blockdev-amend:
diff --git a/block/qcow2.c b/block/qcow2.c
index 750e9c4842..30f073cf2a 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -5596,6 +5596,44 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
return 0;
}
+static int coroutine_fn qcow2_co_amend(BlockDriverState *bs,
+ BlockdevAmendOptions *opts,
+ bool force,
+ Error **errp)
+{
+ BlockdevAmendOptionsQcow2 *qopts = &opts->u.qcow2;
+ BDRVQcow2State *s = bs->opaque;
+ int ret = 0;
+
+ if (qopts->has_encrypt) {
+ if (!s->crypto) {
+ error_setg(errp, "image is not encrypted, can't amend");
+ return -EOPNOTSUPP;
+ }
+
+ if (qopts->encrypt->format != Q_CRYPTO_BLOCK_FORMAT_LUKS) {
+ error_setg(errp,
+ "Amend can't be used to change the qcow2 encryption format");
+ return -EOPNOTSUPP;
+ }
+
+ if (s->crypt_method_header != QCOW_CRYPT_LUKS) {
+ error_setg(errp,
+ "Only LUKS encryption options can be amended for qcow2 with blockdev-amend");
+ return -EOPNOTSUPP;
+ }
+
+ ret = qcrypto_block_amend_options(s->crypto,
+ qcow2_crypto_hdr_read_func,
+ qcow2_crypto_hdr_write_func,
+ bs,
+ qopts->encrypt,
+ force,
+ errp);
+ }
+ return ret;
+}
+
/*
* If offset or size are negative, respectively, they will not be included in
* the BLOCK_IMAGE_CORRUPTED event emitted.
@@ -5813,6 +5851,7 @@ BlockDriver bdrv_qcow2 = {
.mutable_opts = mutable_opts,
.bdrv_co_check = qcow2_co_check,
.bdrv_amend_options = qcow2_amend_options,
+ .bdrv_co_amend = qcow2_co_amend,
.bdrv_detach_aio_context = qcow2_detach_aio_context,
.bdrv_attach_aio_context = qcow2_attach_aio_context,
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 21/31] iotests: add tests for blockdev-amend
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (19 preceding siblings ...)
2020-07-06 10:04 ` [PULL 20/31] block/qcow2: " Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 22/31] qemu-img: convert: don't use unallocated_blocks_are_zero Max Reitz
` (11 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Maxim Levitsky <mlevitsk@redhat.com>
This commit adds two tests that cover the
new blockdev-amend functionality of luks and qcow2 driver
Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
[mreitz: Let 295 verify that LUKS works; drop 295 and 296 from the auto
group]
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200625125548.870061-20-mreitz@redhat.com>
---
tests/qemu-iotests/295 | 280 +++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/295.out | 40 ++++++
tests/qemu-iotests/296 | 234 +++++++++++++++++++++++++++++++
tests/qemu-iotests/296.out | 33 +++++
tests/qemu-iotests/group | 2 +
5 files changed, 589 insertions(+)
create mode 100755 tests/qemu-iotests/295
create mode 100644 tests/qemu-iotests/295.out
create mode 100755 tests/qemu-iotests/296
create mode 100644 tests/qemu-iotests/296.out
diff --git a/tests/qemu-iotests/295 b/tests/qemu-iotests/295
new file mode 100755
index 0000000000..59e674fa85
--- /dev/null
+++ b/tests/qemu-iotests/295
@@ -0,0 +1,280 @@
+#!/usr/bin/env python3
+#
+# Test case QMP's encrypted key management
+#
+# Copyright (C) 2019 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+import iotests
+import os
+import time
+import json
+
+test_img = os.path.join(iotests.test_dir, 'test.img')
+
+class Secret:
+ def __init__(self, index):
+ self._id = "keysec" + str(index)
+ # you are not supposed to see the password...
+ self._secret = "hunter" + str(index)
+
+ def id(self):
+ return self._id
+
+ def secret(self):
+ return self._secret
+
+ def to_cmdline_object(self):
+ return [ "secret,id=" + self._id + ",data=" + self._secret]
+
+ def to_qmp_object(self):
+ return { "qom_type" : "secret", "id": self.id(),
+ "props": { "data": self.secret() } }
+
+################################################################################
+class EncryptionSetupTestCase(iotests.QMPTestCase):
+
+ # test case startup
+ def setUp(self):
+ # start the VM
+ self.vm = iotests.VM()
+ self.vm.launch()
+
+ # create the secrets and load 'em into the VM
+ self.secrets = [ Secret(i) for i in range(0, 6) ]
+ for secret in self.secrets:
+ result = self.vm.qmp("object-add", **secret.to_qmp_object())
+ self.assert_qmp(result, 'return', {})
+
+ if iotests.imgfmt == "qcow2":
+ self.pfx = "encrypt."
+ self.img_opts = [ '-o', "encrypt.format=luks" ]
+ else:
+ self.pfx = ""
+ self.img_opts = []
+
+ # test case shutdown
+ def tearDown(self):
+ # stop the VM
+ self.vm.shutdown()
+
+ ###########################################################################
+ # create the encrypted block device
+ def createImg(self, file, secret):
+
+ iotests.qemu_img(
+ 'create',
+ '--object', *secret.to_cmdline_object(),
+ '-f', iotests.imgfmt,
+ '-o', self.pfx + 'key-secret=' + secret.id(),
+ '-o', self.pfx + 'iter-time=10',
+ *self.img_opts,
+ file,
+ '1M')
+
+ ###########################################################################
+ # open an encrypted block device
+ def openImageQmp(self, id, file, secret, read_only = False):
+
+ encrypt_options = {
+ 'key-secret' : secret.id()
+ }
+
+ if iotests.imgfmt == "qcow2":
+ encrypt_options = {
+ 'encrypt': {
+ 'format':'luks',
+ **encrypt_options
+ }
+ }
+
+ result = self.vm.qmp('blockdev-add', **
+ {
+ 'driver': iotests.imgfmt,
+ 'node-name': id,
+ 'read-only': read_only,
+
+ **encrypt_options,
+
+ 'file': {
+ 'driver': 'file',
+ 'filename': test_img,
+ }
+ }
+ )
+ self.assert_qmp(result, 'return', {})
+
+ # close the encrypted block device
+ def closeImageQmp(self, id):
+ result = self.vm.qmp('blockdev-del', **{ 'node-name': id })
+ self.assert_qmp(result, 'return', {})
+
+ ###########################################################################
+ # add a key to an encrypted block device
+ def addKeyQmp(self, id, new_secret, secret = None,
+ slot = None, force = False):
+
+ crypt_options = {
+ 'state' : 'active',
+ 'new-secret' : new_secret.id(),
+ 'iter-time' : 10
+ }
+
+ if slot != None:
+ crypt_options['keyslot'] = slot
+
+
+ if secret != None:
+ crypt_options['secret'] = secret.id()
+
+ if iotests.imgfmt == "qcow2":
+ crypt_options['format'] = 'luks'
+ crypt_options = {
+ 'encrypt': crypt_options
+ }
+
+ args = {
+ 'node-name': id,
+ 'job-id' : 'job_add_key',
+ 'options' : {
+ 'driver' : iotests.imgfmt,
+ **crypt_options
+ },
+ }
+
+ if force == True:
+ args['force'] = True
+
+ #TODO: check what jobs return
+ result = self.vm.qmp('x-blockdev-amend', **args)
+ assert result['return'] == {}
+ self.vm.run_job('job_add_key')
+
+ # erase a key from an encrypted block device
+ def eraseKeyQmp(self, id, old_secret = None, slot = None, force = False):
+
+ crypt_options = {
+ 'state' : 'inactive',
+ }
+
+ if slot != None:
+ crypt_options['keyslot'] = slot
+ if old_secret != None:
+ crypt_options['old-secret'] = old_secret.id()
+
+ if iotests.imgfmt == "qcow2":
+ crypt_options['format'] = 'luks'
+ crypt_options = {
+ 'encrypt': crypt_options
+ }
+
+ args = {
+ 'node-name': id,
+ 'job-id' : 'job_erase_key',
+ 'options' : {
+ 'driver' : iotests.imgfmt,
+ **crypt_options
+ },
+ }
+
+ if force == True:
+ args['force'] = True
+
+ result = self.vm.qmp('x-blockdev-amend', **args)
+ assert result['return'] == {}
+ self.vm.run_job('job_erase_key')
+
+ ###########################################################################
+ # create image, and change its key
+ def testChangeKey(self):
+
+ # create the image with secret0 and open it
+ self.createImg(test_img, self.secrets[0]);
+ self.openImageQmp("testdev", test_img, self.secrets[0])
+
+ # add key to slot 1
+ self.addKeyQmp("testdev", new_secret = self.secrets[1])
+
+ # add key to slot 5
+ self.addKeyQmp("testdev", new_secret = self.secrets[2], slot=5)
+
+ # erase key from slot 0
+ self.eraseKeyQmp("testdev", old_secret = self.secrets[0])
+
+ #reopen the image with secret1
+ self.closeImageQmp("testdev")
+ self.openImageQmp("testdev", test_img, self.secrets[1])
+
+ # close and erase the image for good
+ self.closeImageQmp("testdev")
+ os.remove(test_img)
+
+ # test that if we erase the old password,
+ # we can still change the encryption keys using 'old-secret'
+ def testOldPassword(self):
+
+ # create the image with secret0 and open it
+ self.createImg(test_img, self.secrets[0]);
+ self.openImageQmp("testdev", test_img, self.secrets[0])
+
+ # add key to slot 1
+ self.addKeyQmp("testdev", new_secret = self.secrets[1])
+
+ # erase key from slot 0
+ self.eraseKeyQmp("testdev", old_secret = self.secrets[0])
+
+ # this will fail as the old password is no longer valid
+ self.addKeyQmp("testdev", new_secret = self.secrets[2])
+
+ # this will work
+ self.addKeyQmp("testdev", new_secret = self.secrets[2], secret = self.secrets[1])
+
+ # close and erase the image for good
+ self.closeImageQmp("testdev")
+ os.remove(test_img)
+
+ def testUseForceLuke(self):
+
+ self.createImg(test_img, self.secrets[0]);
+ self.openImageQmp("testdev", test_img, self.secrets[0])
+
+ # Add bunch of secrets
+ self.addKeyQmp("testdev", new_secret = self.secrets[1], slot=4)
+ self.addKeyQmp("testdev", new_secret = self.secrets[4], slot=2)
+
+ # overwrite an active secret
+ self.addKeyQmp("testdev", new_secret = self.secrets[5], slot=2)
+ self.addKeyQmp("testdev", new_secret = self.secrets[5], slot=2, force=True)
+
+ self.addKeyQmp("testdev", new_secret = self.secrets[0])
+
+ # Now erase all the secrets
+ self.eraseKeyQmp("testdev", old_secret = self.secrets[5])
+ self.eraseKeyQmp("testdev", slot=4)
+
+ # erase last keyslot
+ self.eraseKeyQmp("testdev", old_secret = self.secrets[0])
+ self.eraseKeyQmp("testdev", old_secret = self.secrets[0], force=True)
+
+ self.closeImageQmp("testdev")
+ os.remove(test_img)
+
+
+if __name__ == '__main__':
+ iotests.verify_working_luks()
+ # Encrypted formats support
+ iotests.activate_logging()
+ iotests.main(supported_fmts = ['qcow2', 'luks'])
diff --git a/tests/qemu-iotests/295.out b/tests/qemu-iotests/295.out
new file mode 100644
index 0000000000..ad34b2ca2c
--- /dev/null
+++ b/tests/qemu-iotests/295.out
@@ -0,0 +1,40 @@
+{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}}
+{"return": {}}
+Job failed: Invalid password, cannot unlock any keyslot
+{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
+{"return": {}}
+Job failed: Refusing to overwrite active keyslot 2 - please erase it first
+{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}}
+{"return": {}}
+Job failed: All the active keyslots match the (old) password that was given and erasing them will erase all the data in the image irreversibly - refusing operation
+{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}}
+{"return": {}}
+...
+----------------------------------------------------------------------
+Ran 3 tests
+
+OK
diff --git a/tests/qemu-iotests/296 b/tests/qemu-iotests/296
new file mode 100755
index 0000000000..ec69ec8974
--- /dev/null
+++ b/tests/qemu-iotests/296
@@ -0,0 +1,234 @@
+#!/usr/bin/env python3
+#
+# Test case for encryption key management versus image sharing
+#
+# Copyright (C) 2019 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+import iotests
+import os
+import time
+import json
+
+test_img = os.path.join(iotests.test_dir, 'test.img')
+
+class Secret:
+ def __init__(self, index):
+ self._id = "keysec" + str(index)
+ # you are not supposed to see the password...
+ self._secret = "hunter" + str(index)
+
+ def id(self):
+ return self._id
+
+ def secret(self):
+ return self._secret
+
+ def to_cmdline_object(self):
+ return [ "secret,id=" + self._id + ",data=" + self._secret]
+
+ def to_qmp_object(self):
+ return { "qom_type" : "secret", "id": self.id(),
+ "props": { "data": self.secret() } }
+
+################################################################################
+
+class EncryptionSetupTestCase(iotests.QMPTestCase):
+
+ # test case startup
+ def setUp(self):
+
+ # start the VMs
+ self.vm1 = iotests.VM(path_suffix = 'VM1')
+ self.vm2 = iotests.VM(path_suffix = 'VM2')
+ self.vm1.launch()
+ self.vm2.launch()
+
+ # create the secrets and load 'em into the VMs
+ self.secrets = [ Secret(i) for i in range(0, 4) ]
+ for secret in self.secrets:
+ result = self.vm1.qmp("object-add", **secret.to_qmp_object())
+ self.assert_qmp(result, 'return', {})
+ result = self.vm2.qmp("object-add", **secret.to_qmp_object())
+ self.assert_qmp(result, 'return', {})
+
+ # test case shutdown
+ def tearDown(self):
+ # stop the VM
+ self.vm1.shutdown()
+ self.vm2.shutdown()
+
+ ###########################################################################
+ # create the encrypted block device using qemu-img
+ def createImg(self, file, secret):
+
+ output = iotests.qemu_img_pipe(
+ 'create',
+ '--object', *secret.to_cmdline_object(),
+ '-f', iotests.imgfmt,
+ '-o', 'key-secret=' + secret.id(),
+ '-o', 'iter-time=10',
+ file,
+ '1M')
+
+ iotests.log(output, filters=[iotests.filter_test_dir])
+
+ # attempts to add a key using qemu-img
+ def addKey(self, file, secret, new_secret):
+
+ image_options = {
+ 'key-secret' : secret.id(),
+ 'driver' : iotests.imgfmt,
+ 'file' : {
+ 'driver':'file',
+ 'filename': file,
+ }
+ }
+
+ output = iotests.qemu_img_pipe(
+ 'amend',
+ '--object', *secret.to_cmdline_object(),
+ '--object', *new_secret.to_cmdline_object(),
+
+ '-o', 'state=active',
+ '-o', 'new-secret=' + new_secret.id(),
+ '-o', 'iter-time=10',
+
+ "json:" + json.dumps(image_options)
+ )
+
+ iotests.log(output, filters=[iotests.filter_test_dir])
+
+ ###########################################################################
+ # open an encrypted block device
+ def openImageQmp(self, vm, id, file, secret,
+ readOnly = False, reOpen = False):
+
+ command = 'x-blockdev-reopen' if reOpen else 'blockdev-add'
+
+ result = vm.qmp(command, **
+ {
+ 'driver': iotests.imgfmt,
+ 'node-name': id,
+ 'read-only': readOnly,
+ 'key-secret' : secret.id(),
+ 'file': {
+ 'driver': 'file',
+ 'filename': test_img,
+ }
+ }
+ )
+ self.assert_qmp(result, 'return', {})
+
+ # close the encrypted block device
+ def closeImageQmp(self, vm, id):
+ result = vm.qmp('blockdev-del', **{ 'node-name': id })
+ self.assert_qmp(result, 'return', {})
+
+ ###########################################################################
+
+ # add a key to an encrypted block device
+ def addKeyQmp(self, vm, id, new_secret):
+
+ args = {
+ 'node-name': id,
+ 'job-id' : 'job0',
+ 'options' : {
+ 'state' : 'active',
+ 'driver' : iotests.imgfmt,
+ 'new-secret': new_secret.id(),
+ 'iter-time' : 10
+ },
+ }
+
+ result = vm.qmp('x-blockdev-amend', **args)
+ assert result['return'] == {}
+ vm.run_job('job0')
+
+ # test that when the image opened by two qemu processes,
+ # neither of them can update the image
+ def test1(self):
+ self.createImg(test_img, self.secrets[0]);
+
+ # VM1 opens the image and adds a key
+ self.openImageQmp(self.vm1, "testdev", test_img, self.secrets[0])
+ self.addKeyQmp(self.vm1, "testdev", new_secret = self.secrets[1])
+
+
+ # VM2 opens the image
+ self.openImageQmp(self.vm2, "testdev", test_img, self.secrets[0])
+
+
+ # neither VMs now should be able to add a key
+ self.addKeyQmp(self.vm1, "testdev", new_secret = self.secrets[2])
+ self.addKeyQmp(self.vm2, "testdev", new_secret = self.secrets[2])
+
+
+ # VM 1 closes the image
+ self.closeImageQmp(self.vm1, "testdev")
+
+
+ # now VM2 can add the key
+ self.addKeyQmp(self.vm2, "testdev", new_secret = self.secrets[2])
+
+
+ # qemu-img should also not be able to add a key
+ self.addKey(test_img, self.secrets[0], self.secrets[2])
+
+ # cleanup
+ self.closeImageQmp(self.vm2, "testdev")
+ os.remove(test_img)
+
+
+ def test2(self):
+ self.createImg(test_img, self.secrets[0]);
+
+ # VM1 opens the image readonly
+ self.openImageQmp(self.vm1, "testdev", test_img, self.secrets[0],
+ readOnly = True)
+
+ # VM2 opens the image
+ self.openImageQmp(self.vm2, "testdev", test_img, self.secrets[0])
+
+ # VM1 can't add a key since image is readonly
+ self.addKeyQmp(self.vm1, "testdev", new_secret = self.secrets[2])
+
+ # VM2 can't add a key since VM is has the image opened
+ self.addKeyQmp(self.vm2, "testdev", new_secret = self.secrets[2])
+
+
+ #VM1 reopens the image read-write
+ self.openImageQmp(self.vm1, "testdev", test_img, self.secrets[0],
+ reOpen = True, readOnly = False)
+
+ # VM1 still can't add the key
+ self.addKeyQmp(self.vm1, "testdev", new_secret = self.secrets[2])
+
+ # VM2 gets away
+ self.closeImageQmp(self.vm2, "testdev")
+
+ # VM1 now can add the key
+ self.addKeyQmp(self.vm1, "testdev", new_secret = self.secrets[2])
+
+ self.closeImageQmp(self.vm1, "testdev")
+ os.remove(test_img)
+
+
+if __name__ == '__main__':
+ # support only raw luks since luks encrypted qcow2 is a proper
+ # format driver which doesn't allow any sharing
+ iotests.activate_logging()
+ iotests.main(supported_fmts = ['luks'])
diff --git a/tests/qemu-iotests/296.out b/tests/qemu-iotests/296.out
new file mode 100644
index 0000000000..afb6d2d09d
--- /dev/null
+++ b/tests/qemu-iotests/296.out
@@ -0,0 +1,33 @@
+Formatting 'TEST_DIR/test.img', fmt=luks size=1048576 key-secret=keysec0 iter-time=10
+
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+Job failed: Failed to get shared "consistent read" lock
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+Job failed: Failed to get shared "consistent read" lock
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+qemu-img: Failed to get shared "consistent read" lock
+Is another process using the image [TEST_DIR/test.img]?
+
+Formatting 'TEST_DIR/test.img', fmt=luks size=1048576 key-secret=keysec0 iter-time=10
+
+Job failed: Block node is read-only
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+Job failed: Failed to get shared "consistent read" lock
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+Job failed: Failed to get shared "consistent read" lock
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+..
+----------------------------------------------------------------------
+Ran 2 tests
+
+OK
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index b945dd4f20..9b07a7ed03 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -303,4 +303,6 @@
292 rw auto quick
293 rw
294 rw quick
+295 rw
+296 rw
297 meta
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 22/31] qemu-img: convert: don't use unallocated_blocks_are_zero
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (20 preceding siblings ...)
2020-07-06 10:04 ` [PULL 21/31] iotests: add tests for blockdev-amend Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 23/31] block: inline bdrv_unallocated_blocks_are_zero() Max Reitz
` (10 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
qemu-img convert wants to distinguish ZERO which comes from short
backing files. unallocated_blocks_are_zero field of bdi is unrelated:
space after EOF is always considered to be zero anyway. So, just make
post_backing_zero true in case of short backing file.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200528094405.145708-2-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
qemu-img.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/qemu-img.c b/qemu-img.c
index 7f4938a5ef..53bd32bf8f 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -1681,7 +1681,6 @@ typedef struct ImgConvertState {
BlockBackend *target;
bool has_zero_init;
bool compressed;
- bool unallocated_blocks_are_zero;
bool target_is_new;
bool target_has_backing;
int64_t target_backing_sectors; /* negative if unknown */
@@ -1726,7 +1725,7 @@ static int convert_iteration_sectors(ImgConvertState *s, int64_t sector_num)
if (s->target_backing_sectors >= 0) {
if (sector_num >= s->target_backing_sectors) {
- post_backing_zero = s->unallocated_blocks_are_zero;
+ post_backing_zero = true;
} else if (sector_num + n > s->target_backing_sectors) {
/* Split requests around target_backing_sectors (because
* starting from there, zeros are handled differently) */
@@ -2678,7 +2677,6 @@ static int img_convert(int argc, char **argv)
} else {
s.compressed = s.compressed || bdi.needs_compressed_writes;
s.cluster_sectors = bdi.cluster_size / BDRV_SECTOR_SIZE;
- s.unallocated_blocks_are_zero = bdi.unallocated_blocks_are_zero;
}
ret = convert_do_copy(&s);
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 23/31] block: inline bdrv_unallocated_blocks_are_zero()
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (21 preceding siblings ...)
2020-07-06 10:04 ` [PULL 22/31] qemu-img: convert: don't use unallocated_blocks_are_zero Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 24/31] block/vdi: return ZERO block-status when appropriate Max Reitz
` (9 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
The function has only one user: bdrv_co_block_status(). Inline it to
simplify reviewing of the following patches, which will finally drop
unallocated_blocks_are_zero field too.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200528094405.145708-3-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
include/block/block.h | 1 -
block.c | 15 ---------------
block/io.c | 11 ++++++++---
3 files changed, 8 insertions(+), 19 deletions(-)
diff --git a/include/block/block.h b/include/block/block.h
index a2414a58c5..e7934e07f2 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -489,7 +489,6 @@ int bdrv_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes);
int bdrv_co_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes);
int bdrv_has_zero_init_1(BlockDriverState *bs);
int bdrv_has_zero_init(BlockDriverState *bs);
-bool bdrv_unallocated_blocks_are_zero(BlockDriverState *bs);
bool bdrv_can_write_zeroes_with_unmap(BlockDriverState *bs);
int bdrv_block_status(BlockDriverState *bs, int64_t offset,
int64_t bytes, int64_t *pnum, int64_t *map,
diff --git a/block.c b/block.c
index 144f52e413..62e40db2f1 100644
--- a/block.c
+++ b/block.c
@@ -5408,21 +5408,6 @@ int bdrv_has_zero_init(BlockDriverState *bs)
return 0;
}
-bool bdrv_unallocated_blocks_are_zero(BlockDriverState *bs)
-{
- BlockDriverInfo bdi;
-
- if (bs->backing) {
- return false;
- }
-
- if (bdrv_get_info(bs, &bdi) == 0) {
- return bdi.unallocated_blocks_are_zero;
- }
-
- return false;
-}
-
bool bdrv_can_write_zeroes_with_unmap(BlockDriverState *bs)
{
if (!(bs->open_flags & BDRV_O_UNMAP)) {
diff --git a/block/io.c b/block/io.c
index df8f2a98d4..36d4d562a6 100644
--- a/block/io.c
+++ b/block/io.c
@@ -2407,15 +2407,20 @@ static int coroutine_fn bdrv_co_block_status(BlockDriverState *bs,
if (ret & (BDRV_BLOCK_DATA | BDRV_BLOCK_ZERO)) {
ret |= BDRV_BLOCK_ALLOCATED;
} else if (want_zero) {
- if (bdrv_unallocated_blocks_are_zero(bs)) {
- ret |= BDRV_BLOCK_ZERO;
- } else if (bs->backing) {
+ if (bs->backing) {
BlockDriverState *bs2 = bs->backing->bs;
int64_t size2 = bdrv_getlength(bs2);
if (size2 >= 0 && offset >= size2) {
ret |= BDRV_BLOCK_ZERO;
}
+ } else {
+ BlockDriverInfo bdi;
+ int ret2 = bdrv_get_info(bs, &bdi);
+
+ if (ret2 == 0 && bdi.unallocated_blocks_are_zero) {
+ ret |= BDRV_BLOCK_ZERO;
+ }
}
}
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 24/31] block/vdi: return ZERO block-status when appropriate
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (22 preceding siblings ...)
2020-07-06 10:04 ` [PULL 23/31] block: inline bdrv_unallocated_blocks_are_zero() Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 25/31] block/vpc: " Max Reitz
` (8 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
In case of !VDI_IS_ALLOCATED[], we do zero out the corresponding chunk
of qiov. So, this should be reported as ZERO.
Note that this changes visible output of "qemu-img map --output=json"
and "qemu-io -c map" commands. For qemu-img map, the change is obvious:
we just mark as zero what is really zero. For qemu-io it's less
obvious: what was unallocated now is allocated.
There is an inconsistency in understanding of unallocated regions in
Qemu: backing-supporting format-drivers return 0 block-status to report
go-to-backing logic for this area. Some protocol-drivers (iscsi) return
0 to report fs-unallocated-non-zero status (i.e., don't occupy space on
disk, read result is undefined).
BDRV_BLOCK_ALLOCATED is defined as something more close to
go-to-backing logic. Still it is calculated as ZERO | DATA, so 0 from
iscsi is treated as unallocated. It doesn't influence backing-chain
behavior, as iscsi can't have backing file. But it does influence
"qemu-io -c map".
We should solve this inconsistency at some future point. Now, let's
just make backing-not-supporting format drivers (vdi at this patch and
vpc with the following) to behave more like backing-supporting drivers
and not report 0 block-status. More over, returning ZERO status is
absolutely valid thing, and again, corresponds to how the other
format-drivers (backing-supporting) work.
After block-status update, it never reports 0, so setting
unallocated_blocks_are_zero doesn't make sense (as the only user of it
is bdrv_co_block_status and it checks unallocated_blocks_are_zero only
for unallocated areas). Drop it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200528094405.145708-4-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/vdi.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/block/vdi.c b/block/vdi.c
index 2f506a01ba..c4527a9d8c 100644
--- a/block/vdi.c
+++ b/block/vdi.c
@@ -334,7 +334,6 @@ static int vdi_get_info(BlockDriverState *bs, BlockDriverInfo *bdi)
logout("\n");
bdi->cluster_size = s->block_size;
bdi->vm_state_offset = 0;
- bdi->unallocated_blocks_are_zero = true;
return 0;
}
@@ -536,7 +535,7 @@ static int coroutine_fn vdi_co_block_status(BlockDriverState *bs,
*pnum = MIN(s->block_size - index_in_block, bytes);
result = VDI_IS_ALLOCATED(bmap_entry);
if (!result) {
- return 0;
+ return BDRV_BLOCK_ZERO;
}
*map = s->header.offset_data + (uint64_t)bmap_entry * s->block_size +
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 25/31] block/vpc: return ZERO block-status when appropriate
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (23 preceding siblings ...)
2020-07-06 10:04 ` [PULL 24/31] block/vdi: return ZERO block-status when appropriate Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 26/31] block/crypto: drop unallocated_blocks_are_zero Max Reitz
` (7 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
In case when get_image_offset() returns -1, we do zero out the
corresponding chunk of qiov. So, this should be reported as ZERO.
Note that this changes visible output of "qemu-img map --output=json"
and "qemu-io -c map" commands. For qemu-img map, the change is obvious:
we just mark as zero what is really zero. For qemu-io it's less
obvious: what was unallocated now is allocated.
There is an inconsistency in understanding of unallocated regions in
Qemu: backing-supporting format-drivers return 0 block-status to report
go-to-backing logic for this area. Some protocol-drivers (iscsi) return
0 to report fs-unallocated-non-zero status (i.e., don't occupy space on
disk, read result is undefined).
BDRV_BLOCK_ALLOCATED is defined as something more close to
go-to-backing logic. Still it is calculated as ZERO | DATA, so 0 from
iscsi is treated as unallocated. It doesn't influence backing-chain
behavior, as iscsi can't have backing file. But it does influence
"qemu-io -c map".
We should solve this inconsistency at some future point. Now, let's
just make backing-not-supporting format drivers (vdi in the previous
patch and vpc now) to behave more like backing-supporting drivers
and not report 0 block-status. More over, returning ZERO status is
absolutely valid thing, and again, corresponds to how the other
format-drivers (backing-supporting) work.
After block-status update, it never reports 0, so setting
unallocated_blocks_are_zero doesn't make sense (as the only user of it
is bdrv_co_block_status and it checks unallocated_blocks_are_zero only
for unallocated areas). Drop it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200528094405.145708-5-vsementsov@virtuozzo.com>
[mreitz: qemu-io -c map as used by iotest 146 now reports everything as
allocated; in order to make the test do something useful, we
use qemu-img map --output=json now]
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/vpc.c | 3 +-
tests/qemu-iotests/146 | 60 ++++--
tests/qemu-iotests/146.out | 405 +++++++++++++++++++++++++++++++++++--
3 files changed, 436 insertions(+), 32 deletions(-)
diff --git a/block/vpc.c b/block/vpc.c
index c055591641..01fcd37e3c 100644
--- a/block/vpc.c
+++ b/block/vpc.c
@@ -606,7 +606,6 @@ static int vpc_get_info(BlockDriverState *bs, BlockDriverInfo *bdi)
bdi->cluster_size = s->block_size;
}
- bdi->unallocated_blocks_are_zero = true;
return 0;
}
@@ -745,7 +744,7 @@ static int coroutine_fn vpc_co_block_status(BlockDriverState *bs,
image_offset = get_image_offset(bs, offset, false, NULL);
allocated = (image_offset != -1);
*pnum = 0;
- ret = 0;
+ ret = BDRV_BLOCK_ZERO;
do {
/* All sectors in a block are contiguous (without using the bitmap) */
diff --git a/tests/qemu-iotests/146 b/tests/qemu-iotests/146
index 2e43abddfc..ddc3c1fd80 100755
--- a/tests/qemu-iotests/146
+++ b/tests/qemu-iotests/146
@@ -51,19 +51,25 @@ echo === Testing VPC Autodetect ===
echo
_use_sample_img virtualpc-dynamic.vhd.bz2
-${QEMU_IO} -c "open -o driver=vpc ${TEST_IMG}" -c 'map'
+$QEMU_IMG map --output=json --image-opts \
+ "driver=vpc,file.filename=$TEST_IMG" \
+ | _filter_qemu_img_map
echo
echo === Testing VPC with current_size force ===
echo
-${QEMU_IO} -c "open -o driver=vpc,force_size_calc=current_size ${TEST_IMG}" -c 'map'
+$QEMU_IMG map --output=json --image-opts \
+ "driver=vpc,force_size_calc=current_size,file.filename=$TEST_IMG" \
+ | _filter_qemu_img_map
echo
echo === Testing VPC with chs force ===
echo
-${QEMU_IO} -c "open -o driver=vpc,force_size_calc=chs ${TEST_IMG}" -c 'map'
+$QEMU_IMG map --output=json --image-opts \
+ "driver=vpc,force_size_calc=chs,file.filename=$TEST_IMG" \
+ | _filter_qemu_img_map
_cleanup_test_img
@@ -72,19 +78,25 @@ echo === Testing Hyper-V Autodetect ===
echo
_use_sample_img hyperv2012r2-dynamic.vhd.bz2
-${QEMU_IO} -c "open -o driver=vpc ${TEST_IMG}" -c 'map'
+$QEMU_IMG map --output=json --image-opts \
+ "driver=vpc,file.filename=$TEST_IMG" \
+ | _filter_qemu_img_map
echo
echo === Testing Hyper-V with current_size force ===
echo
-${QEMU_IO} -c "open -o driver=vpc,force_size_calc=current_size ${TEST_IMG}" -c 'map'
+$QEMU_IMG map --output=json --image-opts \
+ "driver=vpc,force_size_calc=current_size,file.filename=$TEST_IMG" \
+ | _filter_qemu_img_map
echo
echo === Testing Hyper-V with chs force ===
echo
-${QEMU_IO} -c "open -o driver=vpc,force_size_calc=chs ${TEST_IMG}" -c 'map'
+$QEMU_IMG map --output=json --image-opts \
+ "driver=vpc,force_size_calc=chs,file.filename=$TEST_IMG" \
+ | _filter_qemu_img_map
_cleanup_test_img
@@ -93,19 +105,25 @@ echo === Testing d2v Autodetect ===
echo
_use_sample_img d2v-zerofilled.vhd.bz2
-${QEMU_IO} -c "open -o driver=vpc ${TEST_IMG}" -c 'map'
+$QEMU_IMG map --output=json --image-opts \
+ "driver=vpc,file.filename=$TEST_IMG" \
+ | _filter_qemu_img_map
echo
echo === Testing d2v with current_size force ===
echo
-${QEMU_IO} -c "open -o driver=vpc,force_size_calc=current_size ${TEST_IMG}" -c 'map'
+$QEMU_IMG map --output=json --image-opts \
+ "driver=vpc,force_size_calc=current_size,file.filename=$TEST_IMG" \
+ | _filter_qemu_img_map
echo
echo === Testing d2v with chs force ===
echo
-${QEMU_IO} -c "open -o driver=vpc,force_size_calc=chs ${TEST_IMG}" -c 'map'
+$QEMU_IMG map --output=json --image-opts \
+ "driver=vpc,force_size_calc=chs,file.filename=$TEST_IMG" \
+ | _filter_qemu_img_map
_cleanup_test_img
@@ -121,19 +139,25 @@ echo
echo === Read created image, default opts ====
echo
-${QEMU_IO} -c "open -o driver=vpc ${TEST_IMG}" -c 'map'
+$QEMU_IMG map --output=json --image-opts \
+ "driver=vpc,file.filename=$TEST_IMG" \
+ | _filter_qemu_img_map
echo
echo === Read created image, force_size_calc=chs ====
echo
-${QEMU_IO} -c "open -o driver=vpc,force_size_calc=chs ${TEST_IMG}" -c 'map'
+$QEMU_IMG map --output=json --image-opts \
+ "driver=vpc,force_size_calc=chs,file.filename=$TEST_IMG" \
+ | _filter_qemu_img_map
echo
echo === Read created image, force_size_calc=current_size ====
echo
-${QEMU_IO} -c "open -o driver=vpc,force_size_calc=current_size ${TEST_IMG}" -c 'map'
+$QEMU_IMG map --output=json --image-opts \
+ "driver=vpc,force_size_calc=current_size,file.filename=$TEST_IMG" \
+ | _filter_qemu_img_map
echo
echo === Testing Image create, force_size ===
@@ -145,19 +169,25 @@ echo
echo === Read created image, default opts ====
echo
-${QEMU_IO} -c "open -o driver=vpc ${TEST_IMG}" -c 'map'
+$QEMU_IMG map --output=json --image-opts \
+ "driver=vpc,file.filename=$TEST_IMG" \
+ | _filter_qemu_img_map
echo
echo === Read created image, force_size_calc=chs ====
echo
-${QEMU_IO} -c "open -o driver=vpc,force_size_calc=chs ${TEST_IMG}" -c 'map'
+$QEMU_IMG map --output=json --image-opts \
+ "driver=vpc,force_size_calc=chs,file.filename=$TEST_IMG" \
+ | _filter_qemu_img_map
echo
echo === Read created image, force_size_calc=current_size ====
echo
-${QEMU_IO} -c "open -o driver=vpc,force_size_calc=current_size ${TEST_IMG}" -c 'map'
+$QEMU_IMG map --output=json --image-opts \
+ "driver=vpc,force_size_calc=current_size,file.filename=$TEST_IMG" \
+ | _filter_qemu_img_map
echo "*** done"
rm -f $seq.full
diff --git a/tests/qemu-iotests/146.out b/tests/qemu-iotests/146.out
index 1332189d87..80513cdd06 100644
--- a/tests/qemu-iotests/146.out
+++ b/tests/qemu-iotests/146.out
@@ -2,39 +2,414 @@ QA output created by 146
=== Testing VPC Autodetect ===
-126.998 GiB (0x1fbfe04000) bytes not allocated at offset 0 bytes (0x0)
+[{ "start": 0, "length": 136363130880, "depth": 0, "zero": true, "data": false }]
=== Testing VPC with current_size force ===
-127 GiB (0x1fc0000000) bytes not allocated at offset 0 bytes (0x0)
+[{ "start": 0, "length": 136365211648, "depth": 0, "zero": true, "data": false }]
=== Testing VPC with chs force ===
-126.998 GiB (0x1fbfe04000) bytes not allocated at offset 0 bytes (0x0)
+[{ "start": 0, "length": 136363130880, "depth": 0, "zero": true, "data": false }]
=== Testing Hyper-V Autodetect ===
-127 GiB (0x1fc0000000) bytes not allocated at offset 0 bytes (0x0)
+[{ "start": 0, "length": 136365211648, "depth": 0, "zero": true, "data": false }]
=== Testing Hyper-V with current_size force ===
-127 GiB (0x1fc0000000) bytes not allocated at offset 0 bytes (0x0)
+[{ "start": 0, "length": 136365211648, "depth": 0, "zero": true, "data": false }]
=== Testing Hyper-V with chs force ===
-126.998 GiB (0x1fbfe04000) bytes not allocated at offset 0 bytes (0x0)
+[{ "start": 0, "length": 136363130880, "depth": 0, "zero": true, "data": false }]
=== Testing d2v Autodetect ===
-251.250 MiB (0xfb40000) bytes allocated at offset 0 bytes (0x0)
+[{ "start": 0, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 2097152, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 4194304, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 6291456, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 8388608, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 10485760, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 12582912, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 14680064, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 16777216, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 18874368, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 20971520, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 23068672, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 25165824, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 27262976, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 29360128, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 31457280, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 33554432, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 35651584, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 37748736, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 39845888, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 41943040, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 44040192, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 46137344, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 48234496, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 50331648, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 52428800, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 54525952, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 56623104, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 58720256, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 60817408, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 62914560, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 65011712, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 67108864, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 69206016, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 71303168, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 73400320, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 75497472, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 77594624, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 79691776, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 81788928, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 83886080, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 85983232, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 88080384, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 90177536, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 92274688, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 94371840, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 96468992, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 98566144, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 100663296, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 102760448, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 104857600, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 106954752, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 109051904, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 111149056, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 113246208, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 115343360, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 117440512, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 119537664, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 121634816, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 123731968, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 125829120, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 127926272, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 130023424, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 132120576, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 134217728, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 136314880, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 138412032, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 140509184, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 142606336, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 144703488, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 146800640, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 148897792, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 150994944, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 153092096, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 155189248, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 157286400, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 159383552, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 161480704, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 163577856, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 165675008, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 167772160, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 169869312, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 171966464, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 174063616, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 176160768, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 178257920, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 180355072, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 182452224, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 184549376, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 186646528, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 188743680, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 190840832, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 192937984, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 195035136, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 197132288, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 199229440, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 201326592, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 203423744, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 205520896, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 207618048, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 209715200, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 211812352, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 213909504, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 216006656, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 218103808, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 220200960, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 222298112, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 224395264, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 226492416, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 228589568, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 230686720, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 232783872, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 234881024, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 236978176, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 239075328, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 241172480, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 243269632, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 245366784, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 247463936, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 249561088, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 251658240, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 253755392, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 255852544, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 257949696, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 260046848, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 262144000, "length": 1310720, "depth": 0, "zero": false, "data": true, "offset": OFFSET }]
=== Testing d2v with current_size force ===
-251.250 MiB (0xfb40000) bytes allocated at offset 0 bytes (0x0)
+[{ "start": 0, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 2097152, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 4194304, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 6291456, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 8388608, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 10485760, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 12582912, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 14680064, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 16777216, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 18874368, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 20971520, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 23068672, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 25165824, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 27262976, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 29360128, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 31457280, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 33554432, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 35651584, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 37748736, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 39845888, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 41943040, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 44040192, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 46137344, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 48234496, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 50331648, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 52428800, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 54525952, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 56623104, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 58720256, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 60817408, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 62914560, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 65011712, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 67108864, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 69206016, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 71303168, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 73400320, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 75497472, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 77594624, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 79691776, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 81788928, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 83886080, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 85983232, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 88080384, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 90177536, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 92274688, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 94371840, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 96468992, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 98566144, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 100663296, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 102760448, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 104857600, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 106954752, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 109051904, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 111149056, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 113246208, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 115343360, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 117440512, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 119537664, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 121634816, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 123731968, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 125829120, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 127926272, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 130023424, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 132120576, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 134217728, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 136314880, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 138412032, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 140509184, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 142606336, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 144703488, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 146800640, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 148897792, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 150994944, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 153092096, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 155189248, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 157286400, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 159383552, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 161480704, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 163577856, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 165675008, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 167772160, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 169869312, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 171966464, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 174063616, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 176160768, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 178257920, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 180355072, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 182452224, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 184549376, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 186646528, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 188743680, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 190840832, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 192937984, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 195035136, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 197132288, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 199229440, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 201326592, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 203423744, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 205520896, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 207618048, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 209715200, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 211812352, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 213909504, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 216006656, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 218103808, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 220200960, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 222298112, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 224395264, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 226492416, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 228589568, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 230686720, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 232783872, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 234881024, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 236978176, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 239075328, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 241172480, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 243269632, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 245366784, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 247463936, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 249561088, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 251658240, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 253755392, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 255852544, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 257949696, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 260046848, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 262144000, "length": 1310720, "depth": 0, "zero": false, "data": true, "offset": OFFSET }]
=== Testing d2v with chs force ===
-251.250 MiB (0xfb40000) bytes allocated at offset 0 bytes (0x0)
+[{ "start": 0, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 2097152, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 4194304, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 6291456, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 8388608, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 10485760, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 12582912, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 14680064, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 16777216, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 18874368, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 20971520, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 23068672, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 25165824, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 27262976, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 29360128, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 31457280, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 33554432, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 35651584, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 37748736, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 39845888, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 41943040, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 44040192, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 46137344, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 48234496, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 50331648, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 52428800, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 54525952, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 56623104, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 58720256, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 60817408, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 62914560, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 65011712, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 67108864, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 69206016, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 71303168, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 73400320, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 75497472, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 77594624, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 79691776, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 81788928, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 83886080, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 85983232, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 88080384, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 90177536, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 92274688, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 94371840, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 96468992, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 98566144, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 100663296, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 102760448, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 104857600, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 106954752, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 109051904, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 111149056, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 113246208, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 115343360, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 117440512, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 119537664, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 121634816, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 123731968, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 125829120, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 127926272, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 130023424, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 132120576, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 134217728, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 136314880, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 138412032, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 140509184, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 142606336, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 144703488, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 146800640, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 148897792, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 150994944, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 153092096, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 155189248, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 157286400, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 159383552, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 161480704, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 163577856, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 165675008, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 167772160, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 169869312, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 171966464, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 174063616, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 176160768, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 178257920, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 180355072, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 182452224, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 184549376, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 186646528, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 188743680, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 190840832, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 192937984, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 195035136, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 197132288, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 199229440, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 201326592, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 203423744, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 205520896, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 207618048, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 209715200, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 211812352, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 213909504, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 216006656, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 218103808, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 220200960, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 222298112, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 224395264, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 226492416, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 228589568, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 230686720, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 232783872, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 234881024, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 236978176, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 239075328, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 241172480, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 243269632, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 245366784, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 247463936, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 249561088, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 251658240, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 253755392, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 255852544, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 257949696, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 260046848, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": OFFSET },
+{ "start": 262144000, "length": 1310720, "depth": 0, "zero": false, "data": true, "offset": OFFSET }]
=== Testing Image create, default ===
@@ -42,15 +417,15 @@ Formatting 'TEST_DIR/IMGFMT-create-test.IMGFMT', fmt=IMGFMT size=4294967296
=== Read created image, default opts ====
-4 GiB (0x10007a000) bytes not allocated at offset 0 bytes (0x0)
+[{ "start": 0, "length": 4295467008, "depth": 0, "zero": true, "data": false }]
=== Read created image, force_size_calc=chs ====
-4 GiB (0x10007a000) bytes not allocated at offset 0 bytes (0x0)
+[{ "start": 0, "length": 4295467008, "depth": 0, "zero": true, "data": false }]
=== Read created image, force_size_calc=current_size ====
-4 GiB (0x10007a000) bytes not allocated at offset 0 bytes (0x0)
+[{ "start": 0, "length": 4295467008, "depth": 0, "zero": true, "data": false }]
=== Testing Image create, force_size ===
@@ -58,13 +433,13 @@ Formatting 'TEST_DIR/IMGFMT-create-test.IMGFMT', fmt=IMGFMT size=4294967296
=== Read created image, default opts ====
-4 GiB (0x100000000) bytes not allocated at offset 0 bytes (0x0)
+[{ "start": 0, "length": 4294967296, "depth": 0, "zero": true, "data": false }]
=== Read created image, force_size_calc=chs ====
-4 GiB (0x100000000) bytes not allocated at offset 0 bytes (0x0)
+[{ "start": 0, "length": 4294967296, "depth": 0, "zero": true, "data": false }]
=== Read created image, force_size_calc=current_size ====
-4 GiB (0x100000000) bytes not allocated at offset 0 bytes (0x0)
+[{ "start": 0, "length": 4294967296, "depth": 0, "zero": true, "data": false }]
*** done
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 26/31] block/crypto: drop unallocated_blocks_are_zero
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (24 preceding siblings ...)
2020-07-06 10:04 ` [PULL 25/31] block/vpc: " Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 27/31] block/iscsi: " Max Reitz
` (6 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
It's false by default, no needs to set it. We are going to drop this
variable at all, so drop it now here, it doesn't hurt.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200528094405.145708-6-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/crypto.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/block/crypto.c b/block/crypto.c
index 3fbd68832b..2636e959ae 100644
--- a/block/crypto.c
+++ b/block/crypto.c
@@ -758,7 +758,6 @@ static int block_crypto_get_info_luks(BlockDriverState *bs,
return ret;
}
- bdi->unallocated_blocks_are_zero = false;
bdi->cluster_size = subbdi.cluster_size;
return 0;
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 27/31] block/iscsi: drop unallocated_blocks_are_zero
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (25 preceding siblings ...)
2020-07-06 10:04 ` [PULL 26/31] block/crypto: drop unallocated_blocks_are_zero Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 28/31] block/file-posix: " Max Reitz
` (5 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
We set bdi->unallocated_blocks_are_zero = iscsilun->lbprz, but
iscsi_co_block_status doesn't return 0 in case of iscsilun->lbprz, it
returns ZERO when appropriate. So actually unallocated_blocks_are_zero
is useless (it doesn't affect the only user of the field:
bdrv_co_block_status()). Drop it now.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200528094405.145708-7-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/iscsi.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/block/iscsi.c b/block/iscsi.c
index a8b76979d8..767e3e75fd 100644
--- a/block/iscsi.c
+++ b/block/iscsi.c
@@ -2163,7 +2163,6 @@ static int coroutine_fn iscsi_co_truncate(BlockDriverState *bs, int64_t offset,
static int iscsi_get_info(BlockDriverState *bs, BlockDriverInfo *bdi)
{
IscsiLun *iscsilun = bs->opaque;
- bdi->unallocated_blocks_are_zero = iscsilun->lbprz;
bdi->cluster_size = iscsilun->cluster_size;
return 0;
}
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 28/31] block/file-posix: drop unallocated_blocks_are_zero
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (26 preceding siblings ...)
2020-07-06 10:04 ` [PULL 27/31] block/iscsi: " Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 29/31] block/vhdx: " Max Reitz
` (4 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
raw_co_block_status() in block/file-posix.c never returns 0, so
unallocated_blocks_are_zero is useless (it doesn't affect the only user
of the field: bdrv_co_block_status()). Drop it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200528094405.145708-8-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/file-posix.c | 3 ---
1 file changed, 3 deletions(-)
diff --git a/block/file-posix.c b/block/file-posix.c
index 3ab8f5a0fa..d86ea57769 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -2878,9 +2878,6 @@ static int coroutine_fn raw_co_pwrite_zeroes(
static int raw_get_info(BlockDriverState *bs, BlockDriverInfo *bdi)
{
- BDRVRawState *s = bs->opaque;
-
- bdi->unallocated_blocks_are_zero = s->discard_zeroes;
return 0;
}
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 29/31] block/vhdx: drop unallocated_blocks_are_zero
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (27 preceding siblings ...)
2020-07-06 10:04 ` [PULL 28/31] block/file-posix: " Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 30/31] block: " Max Reitz
` (3 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
vhdx doesn't have .bdrv_co_block_status handler, so DATA|ALLOCATED is
always assumed for it in bdrv_co_block_status().
unallocated_blocks_are_zero is useless (it doesn't affect the only user
of the field: bdrv_co_block_status()), drop it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200528094405.145708-9-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/vhdx.c | 3 ---
1 file changed, 3 deletions(-)
diff --git a/block/vhdx.c b/block/vhdx.c
index fa9e544a5e..645dc4b4f4 100644
--- a/block/vhdx.c
+++ b/block/vhdx.c
@@ -1164,9 +1164,6 @@ static int vhdx_get_info(BlockDriverState *bs, BlockDriverInfo *bdi)
bdi->cluster_size = s->block_size;
- bdi->unallocated_blocks_are_zero =
- (s->params.data_bits & VHDX_PARAMS_HAS_PARENT) == 0;
-
return 0;
}
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 30/31] block: drop unallocated_blocks_are_zero
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (28 preceding siblings ...)
2020-07-06 10:04 ` [PULL 29/31] block/vhdx: " Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:04 ` [PULL 31/31] qed: Simplify backing reads Max Reitz
` (2 subsequent siblings)
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Currently this field only set by qed and qcow2. But in fact, all
backing-supporting formats (parallels, qcow, qcow2, qed, vmdk) share
these semantics: on unallocated blocks, if there is no backing file they
just memset the buffer with zeroes.
So, document this behavior for .supports_backing and drop
.unallocated_blocks_are_zero
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200528094405.145708-10-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
include/block/block.h | 5 -----
include/block/block_int.h | 12 +++++++++++-
block/io.c | 9 ++-------
block/qcow2.c | 1 -
block/qed.c | 1 -
5 files changed, 13 insertions(+), 15 deletions(-)
diff --git a/include/block/block.h b/include/block/block.h
index e7934e07f2..bca3bb831c 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -21,11 +21,6 @@ typedef struct BlockDriverInfo {
/* offset at which the VM state can be saved (0 if not possible) */
int64_t vm_state_offset;
bool is_dirty;
- /*
- * True if unallocated blocks read back as zeroes. This is equivalent
- * to the LBPRZ flag in the SCSI logical block provisioning page.
- */
- bool unallocated_blocks_are_zero;
/*
* True if this block driver only supports compressed writes
*/
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 1b86b59af1..3d6cf88592 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -123,7 +123,17 @@ struct BlockDriver {
*/
bool bdrv_needs_filename;
- /* Set if a driver can support backing files */
+ /*
+ * Set if a driver can support backing files. This also implies the
+ * following semantics:
+ *
+ * - Return status 0 of .bdrv_co_block_status means that corresponding
+ * blocks are not allocated in this layer of backing-chain
+ * - For such (unallocated) blocks, read will:
+ * - fill buffer with zeros if there is no backing file
+ * - read from the backing file otherwise, where the block layer
+ * takes care of reading zeros beyond EOF if backing file is short
+ */
bool supports_backing;
/* For handling image reopen for split or non-split files */
diff --git a/block/io.c b/block/io.c
index 36d4d562a6..b6564e34c5 100644
--- a/block/io.c
+++ b/block/io.c
@@ -2406,7 +2406,7 @@ static int coroutine_fn bdrv_co_block_status(BlockDriverState *bs,
if (ret & (BDRV_BLOCK_DATA | BDRV_BLOCK_ZERO)) {
ret |= BDRV_BLOCK_ALLOCATED;
- } else if (want_zero) {
+ } else if (want_zero && bs->drv->supports_backing) {
if (bs->backing) {
BlockDriverState *bs2 = bs->backing->bs;
int64_t size2 = bdrv_getlength(bs2);
@@ -2415,12 +2415,7 @@ static int coroutine_fn bdrv_co_block_status(BlockDriverState *bs,
ret |= BDRV_BLOCK_ZERO;
}
} else {
- BlockDriverInfo bdi;
- int ret2 = bdrv_get_info(bs, &bdi);
-
- if (ret2 == 0 && bdi.unallocated_blocks_are_zero) {
- ret |= BDRV_BLOCK_ZERO;
- }
+ ret |= BDRV_BLOCK_ZERO;
}
}
diff --git a/block/qcow2.c b/block/qcow2.c
index 30f073cf2a..38198b4e75 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -4987,7 +4987,6 @@ err:
static int qcow2_get_info(BlockDriverState *bs, BlockDriverInfo *bdi)
{
BDRVQcow2State *s = bs->opaque;
- bdi->unallocated_blocks_are_zero = true;
bdi->cluster_size = s->cluster_size;
bdi->vm_state_offset = qcow2_vm_state_offset(s);
return 0;
diff --git a/block/qed.c b/block/qed.c
index c0c65015c7..a2dd952699 100644
--- a/block/qed.c
+++ b/block/qed.c
@@ -1514,7 +1514,6 @@ static int bdrv_qed_get_info(BlockDriverState *bs, BlockDriverInfo *bdi)
memset(bdi, 0, sizeof(*bdi));
bdi->cluster_size = s->header.cluster_size;
bdi->is_dirty = s->header.features & QED_F_NEED_CHECK;
- bdi->unallocated_blocks_are_zero = true;
return 0;
}
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PULL 31/31] qed: Simplify backing reads
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (29 preceding siblings ...)
2020-07-06 10:04 ` [PULL 30/31] block: " Max Reitz
@ 2020-07-06 10:04 ` Max Reitz
2020-07-06 10:31 ` [PULL 00/31] Block patches no-reply
2020-07-07 20:40 ` Peter Maydell
32 siblings, 0 replies; 45+ messages in thread
From: Max Reitz @ 2020-07-06 10:04 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel, Max Reitz
From: Eric Blake <eblake@redhat.com>
The other four drivers that support backing files (qcow, qcow2,
parallels, vmdk) all rely on the block layer to populate zeroes when
reading beyond EOF of a short backing file. We can simplify the qed
code by doing likewise.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200528094405.145708-11-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/qed.h | 1 -
block/qed.c | 64 +++++------------------------------------------------
2 files changed, 6 insertions(+), 59 deletions(-)
diff --git a/block/qed.h b/block/qed.h
index 42c115d822..3d12bf78d4 100644
--- a/block/qed.h
+++ b/block/qed.h
@@ -140,7 +140,6 @@ typedef struct QEDAIOCB {
/* Current cluster scatter-gather list */
QEMUIOVector cur_qiov;
- QEMUIOVector *backing_qiov;
uint64_t cur_pos; /* position on block device, in bytes */
uint64_t cur_cluster; /* cluster offset in image file */
unsigned int cur_nclusters; /* number of clusters being accessed */
diff --git a/block/qed.c b/block/qed.c
index a2dd952699..ece8b9bb60 100644
--- a/block/qed.c
+++ b/block/qed.c
@@ -849,56 +849,18 @@ static BDRVQEDState *acb_to_s(QEDAIOCB *acb)
* @s: QED state
* @pos: Byte position in device
* @qiov: Destination I/O vector
- * @backing_qiov: Possibly shortened copy of qiov, to be allocated here
- * @cb: Completion function
- * @opaque: User data for completion function
*
* This function reads qiov->size bytes starting at pos from the backing file.
* If there is no backing file then zeroes are read.
*/
static int coroutine_fn qed_read_backing_file(BDRVQEDState *s, uint64_t pos,
- QEMUIOVector *qiov,
- QEMUIOVector **backing_qiov)
+ QEMUIOVector *qiov)
{
- uint64_t backing_length = 0;
- size_t size;
- int ret;
-
- /* If there is a backing file, get its length. Treat the absence of a
- * backing file like a zero length backing file.
- */
if (s->bs->backing) {
- int64_t l = bdrv_getlength(s->bs->backing->bs);
- if (l < 0) {
- return l;
- }
- backing_length = l;
- }
-
- /* Zero all sectors if reading beyond the end of the backing file */
- if (pos >= backing_length ||
- pos + qiov->size > backing_length) {
- qemu_iovec_memset(qiov, 0, 0, qiov->size);
- }
-
- /* Complete now if there are no backing file sectors to read */
- if (pos >= backing_length) {
- return 0;
- }
-
- /* If the read straddles the end of the backing file, shorten it */
- size = MIN((uint64_t)backing_length - pos, qiov->size);
-
- assert(*backing_qiov == NULL);
- *backing_qiov = g_new(QEMUIOVector, 1);
- qemu_iovec_init(*backing_qiov, qiov->niov);
- qemu_iovec_concat(*backing_qiov, qiov, 0, size);
-
- BLKDBG_EVENT(s->bs->file, BLKDBG_READ_BACKING_AIO);
- ret = bdrv_co_preadv(s->bs->backing, pos, size, *backing_qiov, 0);
- if (ret < 0) {
- return ret;
+ BLKDBG_EVENT(s->bs->file, BLKDBG_READ_BACKING_AIO);
+ return bdrv_co_preadv(s->bs->backing, pos, qiov->size, qiov, 0);
}
+ qemu_iovec_memset(qiov, 0, 0, qiov->size);
return 0;
}
@@ -915,7 +877,6 @@ static int coroutine_fn qed_copy_from_backing_file(BDRVQEDState *s,
uint64_t offset)
{
QEMUIOVector qiov;
- QEMUIOVector *backing_qiov = NULL;
int ret;
/* Skip copy entirely if there is no work to do */
@@ -925,13 +886,7 @@ static int coroutine_fn qed_copy_from_backing_file(BDRVQEDState *s,
qemu_iovec_init_buf(&qiov, qemu_blockalign(s->bs, len), len);
- ret = qed_read_backing_file(s, pos, &qiov, &backing_qiov);
-
- if (backing_qiov) {
- qemu_iovec_destroy(backing_qiov);
- g_free(backing_qiov);
- backing_qiov = NULL;
- }
+ ret = qed_read_backing_file(s, pos, &qiov);
if (ret) {
goto out;
@@ -1339,8 +1294,7 @@ static int coroutine_fn qed_aio_read_data(void *opaque, int ret,
qemu_iovec_memset(&acb->cur_qiov, 0, 0, acb->cur_qiov.size);
r = 0;
} else if (ret != QED_CLUSTER_FOUND) {
- r = qed_read_backing_file(s, acb->cur_pos, &acb->cur_qiov,
- &acb->backing_qiov);
+ r = qed_read_backing_file(s, acb->cur_pos, &acb->cur_qiov);
} else {
BLKDBG_EVENT(bs->file, BLKDBG_READ_AIO);
r = bdrv_co_preadv(bs->file, offset, acb->cur_qiov.size,
@@ -1365,12 +1319,6 @@ static int coroutine_fn qed_aio_next_io(QEDAIOCB *acb)
while (1) {
trace_qed_aio_next_io(s, acb, 0, acb->cur_pos + acb->cur_qiov.size);
- if (acb->backing_qiov) {
- qemu_iovec_destroy(acb->backing_qiov);
- g_free(acb->backing_qiov);
- acb->backing_qiov = NULL;
- }
-
acb->qiov_offset += acb->cur_qiov.size;
acb->cur_pos += acb->cur_qiov.size;
qemu_iovec_reset(&acb->cur_qiov);
--
2.26.2
^ permalink raw reply related [flat|nested] 45+ messages in thread* Re: [PULL 00/31] Block patches
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (30 preceding siblings ...)
2020-07-06 10:04 ` [PULL 31/31] qed: Simplify backing reads Max Reitz
@ 2020-07-06 10:31 ` no-reply
2020-07-07 20:40 ` Peter Maydell
32 siblings, 0 replies; 45+ messages in thread
From: no-reply @ 2020-07-06 10:31 UTC (permalink / raw)
To: mreitz; +Cc: kwolf, peter.maydell, qemu-devel, qemu-block, mreitz
Patchew URL: https://patchew.org/QEMU/20200706100432.2301919-1-mreitz@redhat.com/
Hi,
This series seems to have some coding style problems. See output below for
more information:
Subject: [PULL 00/31] Block patches
Type: series
Message-id: 20200706100432.2301919-1-mreitz@redhat.com
=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===
From https://github.com/patchew-project/qemu
* [new tag] patchew/20200706100432.2301919-1-mreitz@redhat.com -> patchew/20200706100432.2301919-1-mreitz@redhat.com
Switched to a new branch 'test'
61a6805 qed: Simplify backing reads
8160191 block: drop unallocated_blocks_are_zero
d59b973 block/vhdx: drop unallocated_blocks_are_zero
fc425f8 block/file-posix: drop unallocated_blocks_are_zero
ee4e5f6 block/iscsi: drop unallocated_blocks_are_zero
9aacd22 block/crypto: drop unallocated_blocks_are_zero
4d6c6ad block/vpc: return ZERO block-status when appropriate
42d9c04 block/vdi: return ZERO block-status when appropriate
f73a13e block: inline bdrv_unallocated_blocks_are_zero()
d5d6d75 qemu-img: convert: don't use unallocated_blocks_are_zero
ecc887e iotests: add tests for blockdev-amend
ee66938 block/qcow2: implement blockdev-amend
9254299 block/crypto: implement blockdev-amend
f905f40 block/core: add generic infrastructure for x-blockdev-amend qmp command
5737d94 iotests: qemu-img tests for luks key management
1981eea block/qcow2: extend qemu-img amend interface with crypto options
75ebd25 block/crypto: implement the encryption key management
381affe block/crypto: rename two functions
37d5645 block/amend: refactor qcow2 amend options
2057114 block/amend: separate amend and create options for qemu-img
01ead3a block/amend: add 'force' option
23c67be qcrypto/luks: implement encryption key management
255ed50 qcrypto/core: add generic infrastructure for crypto options amendment
2267d63 iotests: Check whether luks works
a28c683 iotests.py: Add (verify|has)_working_luks()
e4821d5 iotests.py: Add qemu_img_pipe_and_status()
e9500aa iotests/common.rc: Add _require_working_luks
b3b46b5 iotests: filter few more luks specific create options
f872aeb iotests: Make _filter_img_create more active
d327a4c qcow2: Fix preallocation on images with unaligned sizes
9e289b8 block/block-copy: block_copy_dirty_clusters: fix failure check
=== OUTPUT BEGIN ===
1/31 Checking commit 9e289b8e0d77 (block/block-copy: block_copy_dirty_clusters: fix failure check)
2/31 Checking commit d327a4c43167 (qcow2: Fix preallocation on images with unaligned sizes)
3/31 Checking commit f872aebcf148 (iotests: Make _filter_img_create more active)
4/31 Checking commit b3b46b52bfec (iotests: filter few more luks specific create options)
5/31 Checking commit e9500aa71b58 (iotests/common.rc: Add _require_working_luks)
6/31 Checking commit e4821d5b92da (iotests.py: Add qemu_img_pipe_and_status())
7/31 Checking commit a28c683849bf (iotests.py: Add (verify|has)_working_luks())
8/31 Checking commit 2267d63fda82 (iotests: Check whether luks works)
9/31 Checking commit 255ed5095c89 (qcrypto/core: add generic infrastructure for crypto options amendment)
10/31 Checking commit 23c67bedd6e4 (qcrypto/luks: implement encryption key management)
11/31 Checking commit 01ead3a1b6b0 (block/amend: add 'force' option)
12/31 Checking commit 2057114b7a98 (block/amend: separate amend and create options for qemu-img)
ERROR: Macros with multiple statements should be enclosed in a do - while loop
#34: FILE: block/qcow2.c:5663:
+#define QCOW_COMMON_OPTIONS \
+ { \
+ .name = BLOCK_OPT_SIZE, \
+ .type = QEMU_OPT_SIZE, \
+ .help = "Virtual disk size" \
+ }, \
+ { \
+ .name = BLOCK_OPT_COMPAT_LEVEL, \
+ .type = QEMU_OPT_STRING, \
+ .help = "Compatibility level (v2 [0.10] or v3 [1.1])" \
+ }, \
+ { \
+ .name = BLOCK_OPT_BACKING_FILE, \
+ .type = QEMU_OPT_STRING, \
+ .help = "File name of a base image" \
+ }, \
+ { \
+ .name = BLOCK_OPT_BACKING_FMT, \
+ .type = QEMU_OPT_STRING, \
+ .help = "Image format of the base image" \
+ }, \
+ { \
+ .name = BLOCK_OPT_DATA_FILE, \
+ .type = QEMU_OPT_STRING, \
+ .help = "File name of an external data file" \
+ }, \
+ { \
+ .name = BLOCK_OPT_DATA_FILE_RAW, \
+ .type = QEMU_OPT_BOOL, \
+ .help = "The external data file must stay valid " \
+ "as a raw image" \
+ }, \
+ { \
+ .name = BLOCK_OPT_ENCRYPT, \
+ .type = QEMU_OPT_BOOL, \
+ .help = "Encrypt the image with format 'aes'. (Deprecated " \
+ "in favor of " BLOCK_OPT_ENCRYPT_FORMAT "=aes)", \
+ }, \
+ { \
+ .name = BLOCK_OPT_ENCRYPT_FORMAT, \
+ .type = QEMU_OPT_STRING, \
+ .help = "Encrypt the image, format choices: 'aes', 'luks'", \
+ }, \
+ BLOCK_CRYPTO_OPT_DEF_KEY_SECRET("encrypt.", \
+ "ID of secret providing qcow AES key or LUKS passphrase"), \
+ BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_ALG("encrypt."), \
+ BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_MODE("encrypt."), \
+ BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_ALG("encrypt."), \
+ BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG("encrypt."), \
+ BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG("encrypt."), \
+ BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME("encrypt."), \
+ { \
+ .name = BLOCK_OPT_CLUSTER_SIZE, \
+ .type = QEMU_OPT_SIZE, \
+ .help = "qcow2 cluster size", \
+ .def_value_str = stringify(DEFAULT_CLUSTER_SIZE) \
+ }, \
+ { \
+ .name = BLOCK_OPT_PREALLOC, \
+ .type = QEMU_OPT_STRING, \
+ .help = "Preallocation mode (allowed values: off, " \
+ "metadata, falloc, full)" \
+ }, \
+ { \
+ .name = BLOCK_OPT_LAZY_REFCOUNTS, \
+ .type = QEMU_OPT_BOOL, \
+ .help = "Postpone refcount updates", \
+ .def_value_str = "off" \
+ }, \
+ { \
+ .name = BLOCK_OPT_REFCOUNT_BITS, \
+ .type = QEMU_OPT_NUMBER, \
+ .help = "Width of a reference count entry in bits", \
+ .def_value_str = "16" \
+ }, \
+ { \
+ .name = BLOCK_OPT_COMPRESSION_TYPE, \
+ .type = QEMU_OPT_STRING, \
+ .help = "Compression method used for image cluster " \
+ "compression", \
+ .def_value_str = "zlib" \
+ }
total: 1 errors, 0 warnings, 244 lines checked
Patch 12/31 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
13/31 Checking commit 37d5645c2d89 (block/amend: refactor qcow2 amend options)
14/31 Checking commit 381affeb1932 (block/crypto: rename two functions)
15/31 Checking commit 75ebd2518718 (block/crypto: implement the encryption key management)
16/31 Checking commit 1981eea00495 (block/qcow2: extend qemu-img amend interface with crypto options)
17/31 Checking commit 5737d9411ca1 (iotests: qemu-img tests for luks key management)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#19:
new file mode 100755
total: 0 errors, 1 warnings, 433 lines checked
Patch 17/31 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
18/31 Checking commit f905f403e917 (block/core: add generic infrastructure for x-blockdev-amend qmp command)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#33:
new file mode 100644
total: 0 errors, 1 warnings, 221 lines checked
Patch 18/31 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
19/31 Checking commit 925429913f58 (block/crypto: implement blockdev-amend)
20/31 Checking commit ee66938d1193 (block/qcow2: implement blockdev-amend)
21/31 Checking commit ecc887e8c0a4 (iotests: add tests for blockdev-amend)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#19:
new file mode 100755
total: 0 errors, 1 warnings, 593 lines checked
Patch 21/31 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
22/31 Checking commit d5d6d75d306c (qemu-img: convert: don't use unallocated_blocks_are_zero)
23/31 Checking commit f73a13e6c086 (block: inline bdrv_unallocated_blocks_are_zero())
24/31 Checking commit 42d9c04825ab (block/vdi: return ZERO block-status when appropriate)
25/31 Checking commit 4d6c6ad7b389 (block/vpc: return ZERO block-status when appropriate)
26/31 Checking commit 9aacd22b62a4 (block/crypto: drop unallocated_blocks_are_zero)
27/31 Checking commit ee4e5f6a6804 (block/iscsi: drop unallocated_blocks_are_zero)
28/31 Checking commit fc425f8700a0 (block/file-posix: drop unallocated_blocks_are_zero)
29/31 Checking commit d59b973ebbed (block/vhdx: drop unallocated_blocks_are_zero)
30/31 Checking commit 816019134241 (block: drop unallocated_blocks_are_zero)
31/31 Checking commit 61a68058fcb1 (qed: Simplify backing reads)
=== OUTPUT END ===
Test command exited with code: 1
The full log is available at
http://patchew.org/logs/20200706100432.2301919-1-mreitz@redhat.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 45+ messages in thread* Re: [PULL 00/31] Block patches
2020-07-06 10:04 [PULL 00/31] Block patches Max Reitz
` (31 preceding siblings ...)
2020-07-06 10:31 ` [PULL 00/31] Block patches no-reply
@ 2020-07-07 20:40 ` Peter Maydell
2020-07-08 15:11 ` Maxim Levitsky
32 siblings, 1 reply; 45+ messages in thread
From: Peter Maydell @ 2020-07-07 20:40 UTC (permalink / raw)
To: Max Reitz; +Cc: Kevin Wolf, QEMU Developers, Qemu-block
On Mon, 6 Jul 2020 at 11:04, Max Reitz <mreitz@redhat.com> wrote:
>
> The following changes since commit eb6490f544388dd24c0d054a96dd304bc7284450:
>
> Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20200703' into staging (2020-07-04 16:08:41 +0100)
>
> are available in the Git repository at:
>
> https://github.com/XanClic/qemu.git tags/pull-block-2020-07-06
>
> for you to fetch changes up to 365fed5111b06d31c1632af63c7528dfe49d62a2:
>
> qed: Simplify backing reads (2020-07-06 10:34:14 +0200)
>
> ----------------------------------------------------------------
> Block patches for 5.1:
> - LUKS keyslot amendment
> (+ patches to make the iotests pass on non-Linux systems, and to keep
> the tests passing for qcow v1, and to skip LUKS tests (including
> qcow2 LUKS) when the built qemu does not support it)
> - Refactoring in the block layer: Drop the basically unnecessary
> unallocated_blocks_are_zero field from BlockDriverInfo
> - Fix qcow2 preallocation when the image size is not a multiple of the
> cluster size
> - Fix in block-copy code
>
Applied, thanks.
Please update the changelog at https://wiki.qemu.org/ChangeLog/5.1
for any user-visible changes.
-- PMM
^ permalink raw reply [flat|nested] 45+ messages in thread* Re: [PULL 00/31] Block patches
2020-07-07 20:40 ` Peter Maydell
@ 2020-07-08 15:11 ` Maxim Levitsky
2020-07-08 15:45 ` Kevin Wolf
2020-07-08 15:50 ` Maxim Levitsky
0 siblings, 2 replies; 45+ messages in thread
From: Maxim Levitsky @ 2020-07-08 15:11 UTC (permalink / raw)
To: Peter Maydell, Max Reitz; +Cc: Kevin Wolf, QEMU Developers, Qemu-block
On Tue, 2020-07-07 at 21:40 +0100, Peter Maydell wrote:
> On Mon, 6 Jul 2020 at 11:04, Max Reitz <mreitz@redhat.com> wrote:
> > The following changes since commit eb6490f544388dd24c0d054a96dd304bc7284450:
> >
> > Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20200703' into staging (2020-07-04 16:08:41 +0100)
> >
> > are available in the Git repository at:
> >
> > https://github.com/XanClic/qemu.git tags/pull-block-2020-07-06
> >
> > for you to fetch changes up to 365fed5111b06d31c1632af63c7528dfe49d62a2:
> >
> > qed: Simplify backing reads (2020-07-06 10:34:14 +0200)
> >
> > ----------------------------------------------------------------
> > Block patches for 5.1:
> > - LUKS keyslot amendment
> > (+ patches to make the iotests pass on non-Linux systems, and to keep
> > the tests passing for qcow v1, and to skip LUKS tests (including
> > qcow2 LUKS) when the built qemu does not support it)
> > - Refactoring in the block layer: Drop the basically unnecessary
> > unallocated_blocks_are_zero field from BlockDriverInfo
> > - Fix qcow2 preallocation when the image size is not a multiple of the
> > cluster size
> > - Fix in block-copy code
> >
>
>
> Applied, thanks.
>
> Please update the changelog at https://wiki.qemu.org/ChangeLog/5.1
> for any user-visible changes.
Since this includes my luks patches, I guess I need to add them to this changelog.
How do I get an account on the wiki?
Best regards,
Maxim Levitsky
>
> -- PMM
>
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PULL 00/31] Block patches
2020-07-08 15:11 ` Maxim Levitsky
@ 2020-07-08 15:45 ` Kevin Wolf
2020-07-08 15:50 ` Maxim Levitsky
1 sibling, 0 replies; 45+ messages in thread
From: Kevin Wolf @ 2020-07-08 15:45 UTC (permalink / raw)
To: Maxim Levitsky; +Cc: Peter Maydell, QEMU Developers, Qemu-block, Max Reitz
Am 08.07.2020 um 17:11 hat Maxim Levitsky geschrieben:
> On Tue, 2020-07-07 at 21:40 +0100, Peter Maydell wrote:
> > Applied, thanks.
> >
> > Please update the changelog at https://wiki.qemu.org/ChangeLog/5.1
> > for any user-visible changes.
>
> Since this includes my luks patches, I guess I need to add them to this changelog.
> How do I get an account on the wiki?
For the record, I contacted Maxim on IRC and created an account for him.
Kevin
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PULL 00/31] Block patches
2020-07-08 15:11 ` Maxim Levitsky
2020-07-08 15:45 ` Kevin Wolf
@ 2020-07-08 15:50 ` Maxim Levitsky
1 sibling, 0 replies; 45+ messages in thread
From: Maxim Levitsky @ 2020-07-08 15:50 UTC (permalink / raw)
To: Peter Maydell, Max Reitz; +Cc: Kevin Wolf, QEMU Developers, Qemu-block
On Wed, 2020-07-08 at 18:11 +0300, Maxim Levitsky wrote:
> On Tue, 2020-07-07 at 21:40 +0100, Peter Maydell wrote:
> > On Mon, 6 Jul 2020 at 11:04, Max Reitz <mreitz@redhat.com> wrote:
> > > The following changes since commit eb6490f544388dd24c0d054a96dd304bc7284450:
> > >
> > > Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20200703' into staging (2020-07-04 16:08:41 +0100)
> > >
> > > are available in the Git repository at:
> > >
> > > https://github.com/XanClic/qemu.git tags/pull-block-2020-07-06
> > >
> > > for you to fetch changes up to 365fed5111b06d31c1632af63c7528dfe49d62a2:
> > >
> > > qed: Simplify backing reads (2020-07-06 10:34:14 +0200)
> > >
> > > ----------------------------------------------------------------
> > > Block patches for 5.1:
> > > - LUKS keyslot amendment
> > > (+ patches to make the iotests pass on non-Linux systems, and to keep
> > > the tests passing for qcow v1, and to skip LUKS tests (including
> > > qcow2 LUKS) when the built qemu does not support it)
> > > - Refactoring in the block layer: Drop the basically unnecessary
> > > unallocated_blocks_are_zero field from BlockDriverInfo
> > > - Fix qcow2 preallocation when the image size is not a multiple of the
> > > cluster size
> > > - Fix in block-copy code
> > >
> >
> > Applied, thanks.
> >
> > Please update the changelog at https://wiki.qemu.org/ChangeLog/5.1
> > for any user-visible changes.
>
> Since this includes my luks patches, I guess I need to add them to this changelog.
> How do I get an account on the wiki?
Kevin Wolf gave me an account, so I'll write today a short description in the changelog.
Thanks for everyone that helped me get this merged upstream!
Best regards,
Maxim Levitsky
>
> Best regards,
> Maxim Levitsky
>
> > -- PMM
> >
^ permalink raw reply [flat|nested] 45+ messages in thread