Re: qcow2: Zero-initialization of external data files

[Max Reitz <mreitz@redhat.com>]

[-- Attachment #1.1: Type: text/plain, Size: 2468 bytes --]

On 07.04.20 00:22, Eric Blake wrote:
> On 2/17/20 10:56 AM, Max Reitz wrote:
>> Hi,
>>
>> AFAIU, external data files with data_file_raw=on are supposed to return
>> the same data as the qcow2 file when read.  But we still use the qcow2
>> metadata structures (which are by default initialized to “everything
>> unallocated”), even though we never ensure that the external data file
>> is zero, too, so this can happen:
>>
>> $ dd if=/dev/urandom of=foo.raw 64M
>> [...]
>>
>> $ sudo losetup -f --show foo.raw
>> /dev/loop0
>>
>> $ sudo ./qemu-img create -f qcow2 -o \
>>      data_file=/dev/loop0,data_file_raw=on foo.qcow2 64M
>> [...]
>>
>> $ sudo ./qemu-io -c 'read -P 0 0 64M' foo.qcow2
>> read 67108864/67108864 bytes at offset 0
>> 64 MiB, 1 ops; 00.00 sec (25.036 GiB/sec and 400.5751 ops/sec)
> 
> This looks like a bug (and we should fix it for 5.0 if possible)

It seems a bit difficult for 5.0 now.  (But I don’t think it’d be a
regression, so that shouldn’t be too bad.)

> read
> of a data_file_raw=on should not treat unallocated clusters as reading
> 0, but rather as reading whatever the raw data contains.
> 
>>
>> $ sudo ./qemu-io -c 'read -P 0 0 64M' -f raw foo.raw
>> Pattern verification failed at offset 0, 67108864 bytes
>> read 67108864/67108864 bytes at offset 0
>> 64 MiB, 1 ops; 00.01 sec (5.547 GiB/sec and 88.7484 ops/sec)
>>
>> I suppose this behavior is fine for blockdev-create because I guess it’s
>> the user’s responsibility to ensure that the external data file is zero.
>>   But maybe it isn’t, so that’s my first question: Is it really the
>> user’s responsibility or should we always ensure it’s zero?
> 
> I'd argue that requiring the user to pre-zero the raw data file is
> undesirable; and that we should instead fix our code to not report the
> image as reading all zeroes when creating with data_file_raw=on.

OK.  I think that could be achieved by just enforcing @preallocation to
be at least “metadata” whenever @data-file-raw is set.  Would that make
sense?

Max

>> My second question is: If we decide that this is fine for
>> blockdev-create, should at least qcow2_co_create_opts() ensure the data
>> file it just created is zero?
> 
> Having an option to make qemu force-zero the raw image during
> qcow2_co_create_opts seems reasonable, but for performance reasons, I
> don't think the flag should be on by default.
> 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]