[Qemu-devel] [PATCH] docs: clarify that qcow2 file size is not always a cluster multiple

[Qemu-devel] [PATCH] docs: clarify that qcow2 file size is not always a cluster multiple

[Stefan Hajnoczi]

Normally one would expect that qcow2 image file lengths are multiples of
the cluster size.  This is not true in all cases and the spec should
document this so implementers remember to accept such files.

$ qemu-img create -f qcow2 foo.qcow2 2G
Formatting 'foo.qcow2', fmt=qcow2 size=2147483648 encryption=off cluster_size=65536 lazy_refcounts=off
$ ls -l foo.qcow2
-rw-r--r-- 1 stefanha stefanha 197120 May 22 11:40 foo.qcow2
$ bc -q
3 * (64 * 1024) + 512
197120

The extra sector are the 4 L1 table entries that a 2 GB disk with 64 KB
cluster size needs.  The rest of the L1 table is omitted from the file
but allocation will continue at the next cluster boundary.

Reported-by: Maria Kustova <maxa@catit.be>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 docs/specs/qcow2.txt | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/docs/specs/qcow2.txt b/docs/specs/qcow2.txt
index f19536a..a46ee57 100644
--- a/docs/specs/qcow2.txt
+++ b/docs/specs/qcow2.txt
@@ -4,6 +4,10 @@ A qcow2 image file is organized in units of constant size, which are called
 (host) clusters. A cluster is the unit in which all allocations are done,
 both for actual guest data and for image metadata.
 
+If the end of the image file is not on a cluster boundary, the missing data is
+treated as zeroes.  This layout can occur when an L1 table or refcount table is
+the last thing in the file and not all entries in the table are used.
+
 Likewise, the virtual disk as seen by the guest is divided into (guest)
 clusters of the same size.
 
-- 
1.9.0

Re: [Qemu-devel] [PATCH] docs: clarify that qcow2 file size is not always a cluster multiple

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1100 bytes --]

On 05/22/2014 03:42 AM, Stefan Hajnoczi wrote:
> Normally one would expect that qcow2 image file lengths are multiples of
> the cluster size.  This is not true in all cases and the spec should
> document this so implementers remember to accept such files.
> 
> $ qemu-img create -f qcow2 foo.qcow2 2G
> Formatting 'foo.qcow2', fmt=qcow2 size=2147483648 encryption=off cluster_size=65536 lazy_refcounts=off
> $ ls -l foo.qcow2
> -rw-r--r-- 1 stefanha stefanha 197120 May 22 11:40 foo.qcow2
> $ bc -q
> 3 * (64 * 1024) + 512
> 197120
> 
> The extra sector are the 4 L1 table entries that a 2 GB disk with 64 KB
> cluster size needs.  The rest of the L1 table is omitted from the file
> but allocation will continue at the next cluster boundary.
> 
> Reported-by: Maria Kustova <maxa@catit.be>
> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
> ---
>  docs/specs/qcow2.txt | 4 ++++
>  1 file changed, 4 insertions(+)

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH] docs: clarify that qcow2 file size is not always a cluster multiple

[Kevin Wolf]

Am 22.05.2014 um 11:42 hat Stefan Hajnoczi geschrieben:
> Normally one would expect that qcow2 image file lengths are multiples of
> the cluster size.  This is not true in all cases and the spec should
> document this so implementers remember to accept such files.
> 
> $ qemu-img create -f qcow2 foo.qcow2 2G
> Formatting 'foo.qcow2', fmt=qcow2 size=2147483648 encryption=off cluster_size=65536 lazy_refcounts=off
> $ ls -l foo.qcow2
> -rw-r--r-- 1 stefanha stefanha 197120 May 22 11:40 foo.qcow2
> $ bc -q
> 3 * (64 * 1024) + 512
> 197120
> 
> The extra sector are the 4 L1 table entries that a 2 GB disk with 64 KB
> cluster size needs.  The rest of the L1 table is omitted from the file
> but allocation will continue at the next cluster boundary.
> 
> Reported-by: Maria Kustova <maxa@catit.be>
> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
> ---
>  docs/specs/qcow2.txt | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/docs/specs/qcow2.txt b/docs/specs/qcow2.txt
> index f19536a..a46ee57 100644
> --- a/docs/specs/qcow2.txt
> +++ b/docs/specs/qcow2.txt
> @@ -4,6 +4,10 @@ A qcow2 image file is organized in units of constant size, which are called
>  (host) clusters. A cluster is the unit in which all allocations are done,
>  both for actual guest data and for image metadata.
>  
> +If the end of the image file is not on a cluster boundary, the missing data is
> +treated as zeroes.  This layout can occur when an L1 table or refcount table is
> +the last thing in the file and not all entries in the table are used.
> +
>  Likewise, the virtual disk as seen by the guest is divided into (guest)
>  clusters of the same size.

Why don't we specify this for _any_ data after EOF, as we discussed on
IRC, instead of just for a partial last cluster?

If we restrict it to the last cluster, specifying the data as zero
doesn't make a whole lot of sense because then the data there wouldn't
be supposed to be interpreted anyway.

Kevin

Re: [Qemu-devel] [PATCH] docs: clarify that qcow2 file size is not always a cluster multiple

[Stefan Hajnoczi]

On Fri, May 23, 2014 at 01:18:50PM +0200, Kevin Wolf wrote:
> Am 22.05.2014 um 11:42 hat Stefan Hajnoczi geschrieben:
> > diff --git a/docs/specs/qcow2.txt b/docs/specs/qcow2.txt
> > index f19536a..a46ee57 100644
> > --- a/docs/specs/qcow2.txt
> > +++ b/docs/specs/qcow2.txt
> > @@ -4,6 +4,10 @@ A qcow2 image file is organized in units of constant size, which are called
> >  (host) clusters. A cluster is the unit in which all allocations are done,
> >  both for actual guest data and for image metadata.
> >  
> > +If the end of the image file is not on a cluster boundary, the missing data is
> > +treated as zeroes.  This layout can occur when an L1 table or refcount table is
> > +the last thing in the file and not all entries in the table are used.
> > +
> >  Likewise, the virtual disk as seen by the guest is divided into (guest)
> >  clusters of the same size.
> 
> Why don't we specify this for _any_ data after EOF, as we discussed on
> IRC, instead of just for a partial last cluster?
> 
> If we restrict it to the last cluster, specifying the data as zero
> doesn't make a whole lot of sense because then the data there wouldn't
> be supposed to be interpreted anyway.

That's too general.  Specifying it that way means almost all offsets
become valid because you're just supposed to read zeroes!  That will
definitely cause trouble, let's be stricter here.

Stefan