Re: Clarification regarding new qemu-img convert --target-is-zero flag

[Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>]

10.06.2020 15:19, Sam Eiderman wrote:
> Thanks David,
> 
> Yes, I imaging the following use case:
> 
> disk.vmdk is a 50 GB disk that contains 12 MB binary of zeroes in its beginning.
> /dev/sda is a raw disk containing garbage
> 
> I invoke:
> qemu-img convert disk.vmdk -O raw /dev/sda
> 
> Required output:
> The first 12 MB of /dev/sda contain zeros, the rest garbage, qemu-img
> finishes fast.
> 
> Kevin, from what I understood from you, this is the default behavior.
> 
> So if my VMDK is causing trouble (all virtual size is being written)
> this is probably since all the grains in the VMDK are zero allocated
> right?
> 
> Thanks!

I'm not sure that skipping unallocated clusters in qcow2/vmdk is default. As I see,
briefly looking at the code, unallocated clusters are skipped with -B option. But
it assuming using some backing file, which is not your case.

Let's check:
]# ./qemu-img create -f raw b 1M
Formatting 'b', fmt=raw size=1048576
]# ./qemu-img create -f qcow2 a 1M
Formatting 'a', fmt=qcow2 size=1048576 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
]# ./qemu-io -c 'write -P 0xff 0 1M' -f raw b
wrote 1048576/1048576 bytes at offset 0
1 MiB, 1 ops; 00.05 sec (21.646 MiB/sec and 21.6457 ops/sec)
]# xxd b | head
00000000: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000010: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000020: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000030: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000040: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000050: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000060: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000070: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000080: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000090: ffff ffff ffff ffff ffff ffff ffff ffff  ................
]# ./qemu-img convert -f qcow2 -O raw a b
]# xxd b | head
00000000: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000020: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000030: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000040: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000050: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000060: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000070: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000080: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000090: 0000 0000 0000 0000 0000 0000 0000 0000  ................
]# ./qemu-io -c 'write -P 0xff 0 1M' -f raw b
wrote 1048576/1048576 bytes at offset 0
1 MiB, 1 ops; 00.05 sec (20.648 MiB/sec and 20.6478 ops/sec)
]# ./qemu-img create -f qcow2 base 1M
Formatting 'base', fmt=qcow2 size=1048576 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
]# ./qemu-img convert -f qcow2 -O raw -B base a b
qemu-img: Backing file not supported for file format 'raw'


So you see, in a newly created qcow2 file all cllusters are unallocated. Still by default qemu-img convert writes all zeroes. And we can't use -B with raw tartget.

> 
> On Wed, Jun 10, 2020 at 2:56 PM David Edmondson <dme@dme.org> wrote:
>>
>> On Wednesday, 2020-06-10 at 08:28:29 +03, Sam Eiderman wrote:
>>
>>> Hi,
>>>
>>> 168468fe19c8 ("qemu-img: Add --target-is-zero to convert") has added a
>>> nice functionality for cloud scenarios:
>>>
>>> * Create a virtual disk
>>> * Convert a sparse image (qcow2, vmdk) to the virtual disk using
>>> --target-is-zero
>>> * Use the virtual disk
>>>
>>> This saves many unnecessary writes - a qcow2 with 1MB of allocated
>>> data but with 100GB virtual size will be converted efficiently.
>>>
>>> However, does this pose a problem if the virtual disk is not zero initialized?
>>
>> As Vladimir indicated, the intent of the flag is supposed to be clear
>> from the name :-) If your storage doesn't read zeroes absent any earlier
>> writes, you probably don't want to be using it.
>>
>>> Theoretically - if all unallocated blocks contain garbage - this
>>> shouldn't matter, however what about allocated blocks of zero? Will
>>> convert skip copying allocated zero blocks in the source image to the
>>> target since it assumes that the target is zeroed out first thing?
>>
>> So something like a "--no-need-to-zero" flag would do what you want,
>> presuming that it would write known zeroes but no longer clean the
>> device before use?
>>
>> dme.
>> --
>> You can't hide from the flipside.


-- 
Best regards,
Vladimir