Re: [Qemu-devel] [PATCH] iotests: Test non-self-referential qcow2 refblocks

[Eric Blake <eblake@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 3665 bytes --]

On 12/05/2014 09:53 AM, Max Reitz wrote:
> It is easy to create only self-referential refblocks, but there are
> cases where that is impossible. This adds a test for two of those cases
> (combined in a single test case).
> 
> Suggested-by: Eric Blake <eblake@redhat.com>
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> This patch depends on version 4 (or hopefully any later version) of my
> "qcow2: Support refcount orders != 4" series.
> ---
>  tests/qemu-iotests/115     | 95 ++++++++++++++++++++++++++++++++++++++++++++++
>  tests/qemu-iotests/115.out |  8 ++++
>  tests/qemu-iotests/group   |  1 +
>  3 files changed, 104 insertions(+)
>  create mode 100755 tests/qemu-iotests/115
>  create mode 100644 tests/qemu-iotests/115.out

> +
> +# One refblock can describe (with cluster_size=512 and refcount_bits=64)
> +# 512/8 = 64 clusters, therefore the L1 table should cover 128 clusters, which
> +# equals 128 * (512/8) = 8192 entries (actually, 8192 - 512/8 = 8129 would

This math is slightly off; you really only need 127 consecutive clusters
to guarantee that you have an aligned 64 clusters somewhere in the mix.
 Also, 8192 - 512/8 = 8128 (you are missing the +1 L2 entry that is
necessary to ensure the rollover to 128 clusters).  The real minimum of
L2 entries to provoke the situation we are after is thus 127 * (512/8) -
512/8 + 1 = 8065...

> +# suffice, but it does not really matter). 8192 L2 tables can in turn describe

...at any rate, your conclusion that it does not really matter is
correct; and rounding up to the actual power of 2 is both easier to
explain and more likely to happen in practice (I'm not going to go out
of my way to create a 255.9M guest image, after all).

So, whether or not you want to tweak the comment wording, the test
itself is correct.

Reviewed-by: Eric Blake <eblake@redhat.com>

> +# 8192 * 512/8 = 524,288 clusters which cover a space of 256 MB.
> +
> +# Since with refcount_bits=64 every refcount block entry is 64 bits wide (just
> +# like the L2 table entries), the same calculation applies to the refcount table
> +# as well; the difference is that while for the L1 table the guest disk size is
> +# concerned, for the refcount table it is the image length that has to be at
> +# least 256 MB. We can achieve that by using preallocation=metadata for an image
> +# which has a guest disk size of 256 MB.
> +
> +IMGOPTS="$IMGOPTS,refcount_bits=64,cluster_size=512,preallocation=metadata" \
> +    _make_test_img 256M

Wow - that's MUCH faster than my earlier message that omitted
preallocation then used qemu-io as a followup (around 8 seconds on my
machine); but ALSO much more fragmented (the resulting image, per
qemu-img map, of my multi-minute approach had guest data runs of  mainly
0x8000 bytes at a time, with a few 0x7e00 and 0x8200 sprinkled in, while
your preallocation has lots of runs of 0x200 (1 cluster), 0x400 and so
on, a few runs of 0x1000 or larger, but no run larger than 0x1a00).  But
I did confirm with 'qemu-img map -f raw' that the file is non-sparse, so
you indeed covered both L1 and reftables.

[and seeing how fragmented things can be makes me wonder what sort of
performance gains we could get, if any, by writing a qcow2 defragmenter
- but that's a project for another day, if ever]

> +++ b/tests/qemu-iotests/group
> @@ -115,3 +115,4 @@
>  111 rw auto quick
>  112 rw auto
>  114 rw auto quick
> +115 rw auto

8 seconds is still pretty slow, so I agree with not marking this 'quick'.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 539 bytes --]