[Qemu-devel] using "qemu-img convert -O qcow2" to convert qcow v1 to v2 creates a qcow v3 file?

[Qemu-devel] using "qemu-img convert -O qcow2" to convert qcow v1 to v2 creates a qcow v3 file?

[Peter Maydell]

I have a qcow v1 file which I created by mistake (forgetting that
you need to tell qemu-img create 'qcow2' and not just 'qcow'),
which I want to convert to a v2 file so I can put snapshots into
it. But when I try to do this with qemu-img convert it creates a
v3 file instead:

$ file hda-old.qcow
hda-old.qcow: QEMU QCOW Image (v1), 5368709120 bytes
$ build/x86/qemu-img convert -O qcow2 hda-old.qcow hda.qcow2
$ file hda.qcow2
hda.qcow2: QEMU QCOW Image (v3), 5368709120 bytes

and if you then try to use that in QEMU it complains:

qemu-system-aarch64: -drive if=none,file=hda.qcow2,format=qcow,id=hd:
Unsupported qcow version 3

What am I missing here?

thanks
-- PMM

Re: [Qemu-devel] using "qemu-img convert -O qcow2" to convert qcow v1 to v2 creates a qcow v3 file?

[Peter Maydell]

On 13 November 2017 at 17:14, Peter Maydell <peter.maydell@linaro.org> wrote:
> I have a qcow v1 file which I created by mistake (forgetting that
> you need to tell qemu-img create 'qcow2' and not just 'qcow'),
> which I want to convert to a v2 file so I can put snapshots into
> it. But when I try to do this with qemu-img convert it creates a
> v3 file instead:
>
> $ file hda-old.qcow
> hda-old.qcow: QEMU QCOW Image (v1), 5368709120 bytes
> $ build/x86/qemu-img convert -O qcow2 hda-old.qcow hda.qcow2
> $ file hda.qcow2
> hda.qcow2: QEMU QCOW Image (v3), 5368709120 bytes
>
> and if you then try to use that in QEMU it complains:
>
> qemu-system-aarch64: -drive if=none,file=hda.qcow2,format=qcow,id=hd:
> Unsupported qcow version 3

ah, this means it wants "format=qcow2".

This is pretty confusing, especially the error message, the
output of "file", and the fact that "format=qcow" can't just
DTRT if it gets a qcow version 3 (2?), since it can clearly
identify what it's got.

thanks
-- PMM

Re: [Qemu-devel] using "qemu-img convert -O qcow2" to convert qcow v1 to v2 creates a qcow v3 file?

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1442 bytes --]

On 11/13/2017 11:29 AM, Peter Maydell wrote:
> On 13 November 2017 at 17:14, Peter Maydell <peter.maydell@linaro.org> wrote:
>> I have a qcow v1 file which I created by mistake (forgetting that
>> you need to tell qemu-img create 'qcow2' and not just 'qcow'),
>> which I want to convert to a v2 file so I can put snapshots into
>> it. But when I try to do this with qemu-img convert it creates a
>> v3 file instead:
>>
>> $ file hda-old.qcow
>> hda-old.qcow: QEMU QCOW Image (v1), 5368709120 bytes
>> $ build/x86/qemu-img convert -O qcow2 hda-old.qcow hda.qcow2
>> $ file hda.qcow2
>> hda.qcow2: QEMU QCOW Image (v3), 5368709120 bytes
>>
>> and if you then try to use that in QEMU it complains:
>>
>> qemu-system-aarch64: -drive if=none,file=hda.qcow2,format=qcow,id=hd:
>> Unsupported qcow version 3
> 
> ah, this means it wants "format=qcow2".

Oh, I should have read this followup before writing my other reply.

> 
> This is pretty confusing, especially the error message, the
> output of "file", and the fact that "format=qcow" can't just
> DTRT if it gets a qcow version 3 (2?), since it can clearly
> identify what it's got.

Indeed, making the qcow driver smart enough to reopen with the qcow2
driver (for both v2 and v3 images) might be an interesting ease-of-use hack.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: [Qemu-devel] using "qemu-img convert -O qcow2" to convert qcow v1 to v2 creates a qcow v3 file?

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1654 bytes --]

On 11/13/2017 11:58 AM, Eric Blake wrote:

>>> qemu-system-aarch64: -drive if=none,file=hda.qcow2,format=qcow,id=hd:
>>> Unsupported qcow version 3
>>
>> ah, this means it wants "format=qcow2".
> 
> Oh, I should have read this followup before writing my other reply.
> 
>>
>> This is pretty confusing, especially the error message, the
>> output of "file", and the fact that "format=qcow" can't just
>> DTRT if it gets a qcow version 3 (2?), since it can clearly
>> identify what it's got.
> 
> Indeed, making the qcow driver smart enough to reopen with the qcow2
> driver (for both v2 and v3 images) might be an interesting ease-of-use hack.

And having the qcow2 driverautomatically be able to open a qcow v1 image
read-only might also be nice - although v1 is lacking so many features
that we'd be insane to let a read-write request of the qcow2 driver
downgrade to qcow.

Another observation: is there any official documentation for the qcow
(v1) format?  I know it's an outdated format that we no longer recommend
for new image creation, but it's sad when I have to resort to a google
search to find old posts like this:

https://people.gnome.org/~markmc/qcow-image-format-version-1.html

rather than being able to see the documentation of v1 alongside v2 in
the qemu.git repository.

At any rate, since qcow and qcow2 share the same 4-byte magic number and
version field, it explains why both drivers are able to identify (but
fail to open) the files of the other version number.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: [Qemu-devel] using "qemu-img convert -O qcow2" to convert qcow v1 to v2 creates a qcow v3 file?

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 1943 bytes --]

On 2017-11-13 19:08, Eric Blake wrote:
> On 11/13/2017 11:58 AM, Eric Blake wrote:
> 
>>>> qemu-system-aarch64: -drive if=none,file=hda.qcow2,format=qcow,id=hd:
>>>> Unsupported qcow version 3
>>>
>>> ah, this means it wants "format=qcow2".
>>
>> Oh, I should have read this followup before writing my other reply.
>>
>>>
>>> This is pretty confusing, especially the error message, the
>>> output of "file", and the fact that "format=qcow" can't just
>>> DTRT if it gets a qcow version 3 (2?), since it can clearly
>>> identify what it's got.
>>
>> Indeed, making the qcow driver smart enough to reopen with the qcow2
>> driver (for both v2 and v3 images) might be an interesting ease-of-use hack.
> 
> And having the qcow2 driverautomatically be able to open a qcow v1 image
> read-only might also be nice - although v1 is lacking so many features
> that we'd be insane to let a read-write request of the qcow2 driver
> downgrade to qcow.

Both seem to be hacks to me, though, which needlessly complicate everything.

Just issuing a deprecation warning when creating qcow v1 images and
improving the warning when opening qcow2 (v2/v3) images with the qcow
driver should be enough.

> Another observation: is there any official documentation for the qcow
> (v1) format?  I know it's an outdated format that we no longer recommend
> for new image creation, but it's sad when I have to resort to a google
> search to find old posts like this:
> 
> https://people.gnome.org/~markmc/qcow-image-format-version-1.html
> 
> rather than being able to see the documentation of v1 alongside v2 in
> the qemu.git repository.

Well, do you want to document it?  I'd rather deprecate it altogether.

Max

> At any rate, since qcow and qcow2 share the same 4-byte magic number and
> version field, it explains why both drivers are able to identify (but
> fail to open) the files of the other version number.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: [Qemu-devel] using "qemu-img convert -O qcow2" to convert qcow v1 to v2 creates a qcow v3 file?

[Thomas Huth]

[-- Attachment #1: Type: text/plain, Size: 315 bytes --]

On 14.11.2017 14:32, Max Reitz wrote:
[...]
> Well, do you want to document it?  I'd rather deprecate it altogether.

Maybe a first step could be to change qemu-img so that it refuses to
create new qcow1 images (but still can convert them into other formats).
So basically make qcow1 read-only?

 Thomas


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [Qemu-devel] using "qemu-img convert -O qcow2" to convert qcow v1 to v2 creates a qcow v3 file?

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 511 bytes --]

On 2017-11-14 19:45, Thomas Huth wrote:
> On 14.11.2017 14:32, Max Reitz wrote:
> [...]
>> Well, do you want to document it?  I'd rather deprecate it altogether.
> 
> Maybe a first step could be to change qemu-img so that it refuses to
> create new qcow1 images (but still can convert them into other formats).
> So basically make qcow1 read-only?

Yep, and the actual first step to that is to make it issue a deprecation
warning when creating qcow v1 images (which is what I proposed). :-)

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: [Qemu-devel] using "qemu-img convert -O qcow2" to convert qcow v1 to v2 creates a qcow v3 file?

[John Snow]

On 11/14/2017 01:46 PM, Max Reitz wrote:
> On 2017-11-14 19:45, Thomas Huth wrote:
>> On 14.11.2017 14:32, Max Reitz wrote:
>> [...]
>>> Well, do you want to document it?  I'd rather deprecate it altogether.
>>
>> Maybe a first step could be to change qemu-img so that it refuses to
>> create new qcow1 images (but still can convert them into other formats).
>> So basically make qcow1 read-only?
> 
> Yep, and the actual first step to that is to make it issue a deprecation
> warning when creating qcow v1 images (which is what I proposed). :-)
> 
> Max
> 

Deprecation warning is good.

In future versions you can shimmy it behind a
--no-really-I-want-this-old-format option, I think we ought to support
creating the images for as long as is technologically convenient.

Re: [Qemu-devel] using "qemu-img convert -O qcow2" to convert qcow v1 to v2 creates a qcow v3 file?

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 1593 bytes --]

On 2017-11-14 21:30, John Snow wrote:
> 
> 
> On 11/14/2017 01:46 PM, Max Reitz wrote:
>> On 2017-11-14 19:45, Thomas Huth wrote:
>>> On 14.11.2017 14:32, Max Reitz wrote:
>>> [...]
>>>> Well, do you want to document it?  I'd rather deprecate it altogether.
>>>
>>> Maybe a first step could be to change qemu-img so that it refuses to
>>> create new qcow1 images (but still can convert them into other formats).
>>> So basically make qcow1 read-only?
>>
>> Yep, and the actual first step to that is to make it issue a deprecation
>> warning when creating qcow v1 images (which is what I proposed). :-)
>>
>> Max
>>
> 
> Deprecation warning is good.
> 
> In future versions you can shimmy it behind a
> --no-really-I-want-this-old-format option, I think we ought to support
> creating the images for as long as is technologically convenient.

Well, at some point you can also demand from users to just dig out some
old version of qemu-img to convert their qcow v1 images to qcow2.  It's
not like they are going to miss out on anything.

(If you deprecate emulated hardware, users may complain that they don't
get the newest qemu features/bugfixes/... while continuing to use that
hardware, so I can see that it's a tough decision whether to deprecate
that.  But it's not like you are going to lose any features or anything
if you convert your dusty images to qcow2.  On the contrary, we're
helping you to get more performance out of them.  Maybe qemu should just
silently convert qcow v1 images to qcow2 without asking the user, like
Apple did...)

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: [Qemu-devel] using "qemu-img convert -O qcow2" to convert qcow v1 to v2 creates a qcow v3 file?

[John Snow]

On 11/14/2017 03:35 PM, Max Reitz wrote:
> On 2017-11-14 21:30, John Snow wrote:
>>
>>
>> On 11/14/2017 01:46 PM, Max Reitz wrote:
>>> On 2017-11-14 19:45, Thomas Huth wrote:
>>>> On 14.11.2017 14:32, Max Reitz wrote:
>>>> [...]
>>>>> Well, do you want to document it?  I'd rather deprecate it altogether.
>>>>
>>>> Maybe a first step could be to change qemu-img so that it refuses to
>>>> create new qcow1 images (but still can convert them into other formats).
>>>> So basically make qcow1 read-only?
>>>
>>> Yep, and the actual first step to that is to make it issue a deprecation
>>> warning when creating qcow v1 images (which is what I proposed). :-)
>>>
>>> Max
>>>
>>
>> Deprecation warning is good.
>>
>> In future versions you can shimmy it behind a
>> --no-really-I-want-this-old-format option, I think we ought to support
>> creating the images for as long as is technologically convenient.
> 
> Well, at some point you can also demand from users to just dig out some
> old version of qemu-img to convert their qcow v1 images to qcow2.  It's
> not like they are going to miss out on anything.
> 

As long is convenient. I won't throw a fit that it needs to be around
forever, but as long as it's sufficiently guarded from use and isn't
hard to keep around I'd prefer to do that.

I suppose it's just a weak preference.

> (If you deprecate emulated hardware, users may complain that they don't
> get the newest qemu features/bugfixes/... while continuing to use that
> hardware, so I can see that it's a tough decision whether to deprecate
> that.  But it's not like you are going to lose any features or anything
> if you convert your dusty images to qcow2.  On the contrary, we're
> helping you to get more performance out of them.  Maybe qemu should just
> silently convert qcow v1 images to qcow2 without asking the user, like
> Apple did...)
> 
> Max
> 

"Like Apple did" seems sufficient justification to never do that, but
maybe that's just my own opinion.

Re: [Qemu-devel] using "qemu-img convert -O qcow2" to convert qcow v1 to v2 creates a qcow v3 file?

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 3010 bytes --]

On 2017-11-14 21:38, John Snow wrote:
> 
> 
> On 11/14/2017 03:35 PM, Max Reitz wrote:
>> On 2017-11-14 21:30, John Snow wrote:
>>>
>>>
>>> On 11/14/2017 01:46 PM, Max Reitz wrote:
>>>> On 2017-11-14 19:45, Thomas Huth wrote:
>>>>> On 14.11.2017 14:32, Max Reitz wrote:
>>>>> [...]
>>>>>> Well, do you want to document it?  I'd rather deprecate it altogether.
>>>>>
>>>>> Maybe a first step could be to change qemu-img so that it refuses to
>>>>> create new qcow1 images (but still can convert them into other formats).
>>>>> So basically make qcow1 read-only?
>>>>
>>>> Yep, and the actual first step to that is to make it issue a deprecation
>>>> warning when creating qcow v1 images (which is what I proposed). :-)
>>>>
>>>> Max
>>>>
>>>
>>> Deprecation warning is good.
>>>
>>> In future versions you can shimmy it behind a
>>> --no-really-I-want-this-old-format option, I think we ought to support
>>> creating the images for as long as is technologically convenient.
>>
>> Well, at some point you can also demand from users to just dig out some
>> old version of qemu-img to convert their qcow v1 images to qcow2.  It's
>> not like they are going to miss out on anything.
>>
> 
> As long is convenient. I won't throw a fit that it needs to be around
> forever, but as long as it's sufficiently guarded from use and isn't
> hard to keep around I'd prefer to do that.
> 
> I suppose it's just a weak preference.

I agree that sufficiently guarding it (albeit our definitions on what is
sufficient may differ) serves all the purpose I need, that is, make
users aware of the fact that they are doing something for which I can
see no reason.

But on the other hand, I want those guards to make it dead code,
effectively.  And if something is dead code... There is no reason to
keep it around.

>> (If you deprecate emulated hardware, users may complain that they don't
>> get the newest qemu features/bugfixes/... while continuing to use that
>> hardware, so I can see that it's a tough decision whether to deprecate
>> that.  But it's not like you are going to lose any features or anything
>> if you convert your dusty images to qcow2.  On the contrary, we're
>> helping you to get more performance out of them.  Maybe qemu should just
>> silently convert qcow v1 images to qcow2 without asking the user, like
>> Apple did...)
>>
>> Max
>>
> 
> "Like Apple did" seems sufficient justification to never do that, but
> maybe that's just my own opinion.

Sorry, forgot the emoticon. :-)

Yes, that was meant to be a joke.  Although adding a qemu-img amend for
amending qcow v1 to v2/v3 images is probably mostly an issue of creating
the right internal interface for cross-format amendments.

(Silently storing qcow v1 as v2/v3 images is actually something where
users could have a problem because maybe they have some tool that only
works on qcow v1 images; and then they can't use that together with an
auto-amending qemu...)

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: [Qemu-devel] using "qemu-img convert -O qcow2" to convert qcow v1 to v2 creates a qcow v3 file?

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1247 bytes --]

On 11/13/2017 11:14 AM, Peter Maydell wrote:> I have a qcow v1 file
which I created by mistake (forgetting that
> you need to tell qemu-img create 'qcow2' and not just 'qcow'),
> which I want to convert to a v2 file so I can put snapshots into
> it. But when I try to do this with qemu-img convert it creates a
> v3 file instead:
>
> $ file hda-old.qcow
> hda-old.qcow: QEMU QCOW Image (v1), 5368709120 bytes
> $ build/x86/qemu-img convert -O qcow2 hda-old.qcow hda.qcow2
> $ file hda.qcow2
> hda.qcow2: QEMU QCOW Image (v3), 5368709120 bytes
>
> and if you then try to use that in QEMU it complains:
>
> qemu-system-aarch64: -drive if=none,file=hda.qcow2,format=qcow,id=hd:
> Unsupported qcow version 3
>
> What am I missing here?
qcow2 has two versions; v2 (aka compat=0.10), and v3 (aka compat=1.1).
We changed qemu-img to create v3 by default several years ago, but there
are older qemu binaries (hello, CentOS 6) that still don't understand
v3.  Your qemu-system-aarch64 appears to be one of these older binaries.

Try:
qemu-img convert -O qcow2 -o compat=0.10 hda-old.qcow hda.qcow2

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]