[Qemu-devel] [6907] Introducing qcow2 extensions (Uri Lublin)

[Qemu-devel] [6907] Introducing qcow2 extensions (Uri Lublin)

[Anthony Liguori]

Revision: 6907
          http://svn.sv.gnu.org/viewvc/?view=rev&root=qemu&revision=6907
Author:   aliguori
Date:     2009-03-28 17:55:06 +0000 (Sat, 28 Mar 2009)
Log Message:
-----------
Introducing qcow2 extensions (Uri Lublin)

Qcow2 extensions are build of magic (id) len (in bytes) and data.
They reside right after the qcow2 header.
If a backing filename exists it follows the qcow2 extension (if exist)

Qcow2 extensions are read upon image open.
Qcow2 extensions are identified by their magic.
Unknown qcow2 extensions (unknown magic) are skipped.
A Special magic of 0 means end-of-qcow2-extensions.

In this patchset, to be used to keep backing file format.

Based on a work done by Shahar Frank <sfrank@redhat.com>.

Signed-off-by: Uri Lublin <uril@redhat.com>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>

Modified Paths:
--------------
    trunk/block-qcow2.c

Modified: trunk/block-qcow2.c
===================================================================
--- trunk/block-qcow2.c	2009-03-28 17:51:40 UTC (rev 6906)
+++ trunk/block-qcow2.c	2009-03-28 17:55:06 UTC (rev 6907)
@@ -45,6 +45,7 @@
 
 //#define DEBUG_ALLOC
 //#define DEBUG_ALLOC2
+//#define DEBUG_EXT
 
 #define QCOW_MAGIC (('Q' << 24) | ('F' << 16) | ('I' << 8) | 0xfb)
 #define QCOW_VERSION 2
@@ -77,6 +78,14 @@
     uint64_t snapshots_offset;
 } QCowHeader;
 
+
+typedef struct {
+    uint32_t magic;
+    uint32_t len;
+} QCowExtension;
+#define  QCOW_EXT_MAGIC_END 0
+
+
 typedef struct __attribute__((packed)) QCowSnapshotHeader {
     /* header is 8 byte aligned */
     uint64_t l1_table_offset;
@@ -183,11 +192,66 @@
         return 0;
 }
 
+
+/* 
+ * read qcow2 extension and fill bs
+ * start reading from start_offset
+ * finish reading upon magic of value 0 or when end_offset reached
+ * unknown magic is skipped (future extension this version knows nothing about)
+ * return 0 upon success, non-0 otherwise
+ */
+static int qcow_read_extensions(BlockDriverState *bs, uint64_t start_offset,
+                                uint64_t end_offset)
+{
+    BDRVQcowState *s = bs->opaque;
+    QCowExtension ext;
+    uint64_t offset;
+
+#ifdef DEBUG_EXT
+    printf("qcow_read_extensions: start=%ld end=%ld\n", start_offset, end_offset);
+#endif
+    offset = start_offset;
+    while (offset < end_offset) {
+
+#ifdef DEBUG_EXT
+        /* Sanity check */
+        if (offset > s->cluster_size)
+            printf("qcow_handle_extension: suspicious offset %lu\n", offset);
+
+        printf("attemting to read extended header in offset %lu\n", offset);
+#endif
+
+        if (bdrv_pread(s->hd, offset, &ext, sizeof(ext)) != sizeof(ext)) {
+            fprintf(stderr, "qcow_handle_extension: ERROR: pread fail from offset %lu\n",
+                    offset);
+            return 1;
+        }
+        be32_to_cpus(&ext.magic);
+        be32_to_cpus(&ext.len);
+        offset += sizeof(ext);
+#ifdef DEBUG_EXT
+        printf("ext.magic = 0x%x\n", ext.magic);
+#endif
+        switch (ext.magic) {
+        case QCOW_EXT_MAGIC_END:
+            return 0;
+        default:
+            /* unknown magic -- just skip it */
+            offset += ((ext.len + 7) & ~7);
+            break;
+        }
+    }
+
+    return 0;
+}
+
+
 static int qcow_open(BlockDriverState *bs, const char *filename, int flags)
 {
     BDRVQcowState *s = bs->opaque;
     int len, i, shift, ret;
     QCowHeader header;
+    uint64_t ext_end;
 
     /* Performance is terrible right now with cache=writethrough due mainly
      * to reference count updates.  If the user does not explicitly specify
@@ -270,6 +334,14 @@
     if (refcount_init(bs) < 0)
         goto fail;
 
+    /* read qcow2 extensions */
+    if (header.backing_file_offset)
+        ext_end = header.backing_file_offset;
+    else
+        ext_end = s->cluster_size;
+    if (qcow_read_extensions(bs, sizeof(header), ext_end))
+        goto fail;
+
     /* read the backing file name */
     if (header.backing_file_offset != 0) {
         len = header.backing_file_size;

Re: [Qemu-devel] [6907] Introducing qcow2 extensions (Uri Lublin)

[Avi Kivity]

Anthony Liguori wrote:
> Revision: 6907
>           http://svn.sv.gnu.org/viewvc/?view=rev&root=qemu&revision=6907
> Author:   aliguori
> Date:     2009-03-28 17:55:06 +0000 (Sat, 28 Mar 2009)
> Log Message:
> -----------
> Introducing qcow2 extensions (Uri Lublin)
>
> Qcow2 extensions are build of magic (id) len (in bytes) and data.
> They reside right after the qcow2 header.
> If a backing filename exists it follows the qcow2 extension (if exist)
>
> Qcow2 extensions are read upon image open.
> Qcow2 extensions are identified by their magic.
> Unknown qcow2 extensions (unknown magic) are skipped.
> A Special magic of 0 means end-of-qcow2-extensions.
>
>   

We should introduce a notion of compatible vs. incompatible extensions.

A compatible extension my be ignored by the qcow2 code if it does not 
understand the magic number.  An incompatible extension causes an 
abort.  This allows both more flexibility in how we can change the file 
format.  I believe ext* does the same thing.


-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: [Qemu-devel] [6907] Introducing qcow2 extensions (Uri Lublin)

[Anthony Liguori]

Avi Kivity wrote: 
>
> We should introduce a notion of compatible vs. incompatible extensions.
>
> A compatible extension my be ignored by the qcow2 code if it does not 
> understand the magic number.  An incompatible extension causes an 
> abort.  This allows both more flexibility in how we can change the 
> file format.  I believe ext* does the same thing.

I was assuming that all extensions would be compatible.  I don't like 
the idea of having qcow2 files floating around that require specific 
version of QEMU.

For that, we should just bump to qcow3 (that's what versioning is for, 
right? :-).

Regards,

Anthony Liguori

Re: [Qemu-devel] [6907] Introducing qcow2 extensions (Uri Lublin)

[Avi Kivity]

Anthony Liguori wrote:
>> We should introduce a notion of compatible vs. incompatible extensions.
>>
>> A compatible extension my be ignored by the qcow2 code if it does not 
>> understand the magic number.  An incompatible extension causes an 
>> abort.  This allows both more flexibility in how we can change the 
>> file format.  I believe ext* does the same thing.
>
> I was assuming that all extensions would be compatible.  I don't like 
> the idea of having qcow2 files floating around that require specific 
> version of QEMU.
>
> For that, we should just bump to qcow3 (that's what versioning is for, 
> right? :-).

The problem is that you would need to bump the version each time you 
made an incompatible change.  We'd end up with qcow4351 very quickly.

One option is to declare qcow3 as qcow2 + all incompatible extensions 
just before 0.11.  Another is to require explicit user action to enable 
an incompatible option.  If you did that, you could expect only to 
upgrade qemu, not degrade, but that's a reasonable assumption in many 
scenarios.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: [Qemu-devel] [6907] Introducing qcow2 extensions (Uri Lublin)

[Anthony Liguori]

Avi Kivity wrote:
> Anthony Liguori wrote:
>>> We should introduce a notion of compatible vs. incompatible extensions.
>>>
>>> A compatible extension my be ignored by the qcow2 code if it does 
>>> not understand the magic number.  An incompatible extension causes 
>>> an abort.  This allows both more flexibility in how we can change 
>>> the file format.  I believe ext* does the same thing.
>>
>> I was assuming that all extensions would be compatible.  I don't like 
>> the idea of having qcow2 files floating around that require specific 
>> version of QEMU.
>>
>> For that, we should just bump to qcow3 (that's what versioning is 
>> for, right? :-).
>
> The problem is that you would need to bump the version each time you 
> made an incompatible change.  We'd end up with qcow4351 very quickly.
>
> One option is to declare qcow3 as qcow2 + all incompatible extensions 
> just before 0.11.  Another is to require explicit user action to 
> enable an incompatible option.  If you did that, you could expect only 
> to upgrade qemu, not degrade, but that's a reasonable assumption in 
> many scenarios.

Well what sort of incompatible extensions are we talking about?  Are 
there things being kicked around?

Regards,

Anthony Liguori