[Qemu-devel] [PATCH v3 0/2] qemu-img: Add --backing-chain option to info command

[Qemu-devel] [PATCH v3 0/2] qemu-img: Add --backing-chain option to info command

[Stefan Hajnoczi]

This series adds the --backing-chain option for enumerating the backing file
chain.  Given the topmost image it will print qemu-img info information for
each image file in the chain.

Special care needs to be taken when image files form an infinite loop.  This is
very unusual, most like due to malicious image files.  Nevertheless, qemu-img
must be robust against invalid inputs so we explicit check for this.

v3:
 * Use ImageInfoList to avoid manually outputting JSON and to make the code
   more QAPI-friendly.
 * Solve the output vs error messages issues by printing a detailed error
   message but no image info.  To dig into a broken image chain, rerun without
   --backing-chain.  This is different from what I initially tried but is much
   cleaner because it doesn't produce confusing output.
 * rm -f $TEST_IMG.[123].base [Kevin]
 * Include --output=json in test case [Kevin]
 * Rename test case to free 043 number [Eric]

Stefan Hajnoczi (2):
  qemu-img: Add --backing-chain option to info command
  qemu-iotests: Add 043 backing file chain infinite loop test

 qemu-img.c                   | 167 +++++++++++++++++++++++++++++++++++++++----
 tests/qemu-iotests/043       |  94 ++++++++++++++++++++++++
 tests/qemu-iotests/043.out   |  66 +++++++++++++++++
 tests/qemu-iotests/common.rc |  10 +++
 tests/qemu-iotests/group     |   1 +
 5 files changed, 324 insertions(+), 14 deletions(-)
 create mode 100755 tests/qemu-iotests/043
 create mode 100644 tests/qemu-iotests/043.out

-- 
1.7.11.7

[Qemu-devel] [PATCH v3 1/2] qemu-img: Add --backing-chain option to info command

[Stefan Hajnoczi]

The qemu-img info --backing-chain option enumerates the backing file
chain.  For example, for base.qcow2 <- snap1.qcow2 <- snap2.qcow2 the
output becomes:

  $ qemu-img info --backing-chain snap2.qcow2
  image: snap2.qcow2
  file format: qcow2
  virtual size: 100M (104857600 bytes)
  disk size: 196K
  cluster_size: 65536
  backing file: snap1.qcow2
  backing file format: qcow2

  image: snap1.qcow2
  file format: qcow2
  virtual size: 100M (104857600 bytes)
  disk size: 196K
  cluster_size: 65536
  backing file: base.qcow2
  backing file format: qcow2

  image: base.qcow2
  file format: qcow2
  virtual size: 100M (104857600 bytes)
  disk size: 136K
  cluster_size: 65536

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 qemu-img.c | 167 +++++++++++++++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 153 insertions(+), 14 deletions(-)

diff --git a/qemu-img.c b/qemu-img.c
index f17f187..7261be9 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -1108,6 +1108,23 @@ static void dump_snapshots(BlockDriverState *bs)
     g_free(sn_tab);
 }
 
+static void dump_json_image_info_list(ImageInfoList *list)
+{
+    Error *errp = NULL;
+    QString *str;
+    QmpOutputVisitor *ov = qmp_output_visitor_new();
+    QObject *obj;
+    visit_type_ImageInfoList(qmp_output_get_visitor(ov),
+                             &list, NULL, &errp);
+    obj = qmp_output_get_qobject(ov);
+    str = qobject_to_json_pretty(obj);
+    assert(str != NULL);
+    printf("%s\n", qstring_get_str(str));
+    qobject_decref(obj);
+    qmp_output_visitor_cleanup(ov);
+    QDECREF(str);
+}
+
 static void collect_snapshots(BlockDriverState *bs , ImageInfo *info)
 {
     int i, sn_count;
@@ -1247,9 +1264,129 @@ static void dump_human_image_info(ImageInfo *info)
             printf("backing file format: %s\n", info->backing_filename_format);
         }
     }
+
+    if (info->has_snapshots) {
+        SnapshotInfoList *elem;
+        char buf[256];
+
+        printf("Snapshot list:\n");
+        printf("%s\n", bdrv_snapshot_dump(buf, sizeof(buf), NULL));
+
+        /* Ideally bdrv_snapshot_dump() would operate on SnapshotInfoList but
+         * we convert to the block layer's native QEMUSnapshotInfo for now.
+         */
+        for (elem = info->snapshots; elem; elem = elem->next) {
+            QEMUSnapshotInfo sn = {
+                .vm_state_size = elem->value->vm_state_size,
+                .date_sec = elem->value->date_sec,
+                .date_nsec = elem->value->date_nsec,
+                .vm_clock_nsec = elem->value->vm_clock_sec * 1000000000ULL +
+                                 elem->value->vm_clock_nsec,
+            };
+
+            pstrcpy(sn.id_str, sizeof(sn.id_str), elem->value->id);
+            pstrcpy(sn.name, sizeof(sn.name), elem->value->name);
+            printf("%s\n", bdrv_snapshot_dump(buf, sizeof(buf), &sn));
+        }
+    }
+}
+
+static void dump_human_image_info_list(ImageInfoList *list)
+{
+    ImageInfoList *elem;
+    bool delim = false;
+
+    for (elem = list; elem; elem = elem->next) {
+        if (delim) {
+            printf("\n");
+        }
+        delim = true;
+
+        dump_human_image_info(elem->value);
+    }
 }
 
-enum {OPTION_OUTPUT = 256};
+static gboolean str_equal_func(gconstpointer a, gconstpointer b)
+{
+    return strcmp(a, b) == 0;
+}
+
+/**
+ * Open an image file chain and return an ImageInfoList
+ *
+ * @filename: topmost image filename
+ * @fmt: topmost image format (may be NULL to autodetect)
+ * @chain: true  - enumerate entire backing file chain
+ *         false - only topmost image file
+ *
+ * Returns a list of ImageInfo objects or NULL if there was an error opening an
+ * image file.  If there was an error a message will have been printed to
+ * stderr.
+ */
+static ImageInfoList *collect_image_info_list(const char *filename,
+                                              const char *fmt,
+                                              bool chain)
+{
+    ImageInfoList *head = NULL;
+    ImageInfoList **last = &head;
+    GHashTable *filenames;
+
+    filenames = g_hash_table_new_full(g_str_hash, str_equal_func, NULL, NULL);
+
+    while (filename) {
+        BlockDriverState *bs;
+        ImageInfo *info;
+        ImageInfoList *elem;
+
+        if (g_hash_table_lookup_extended(filenames, filename, NULL, NULL)) {
+            error_report("Backing file '%s' creates an infinite loop.",
+                         filename);
+            goto err;
+        }
+        g_hash_table_insert(filenames, (gpointer)filename, NULL);
+
+        bs = bdrv_new_open(filename, fmt, BDRV_O_FLAGS | BDRV_O_NO_BACKING,
+                           false);
+        if (!bs) {
+            goto err;
+        }
+
+        info = g_new0(ImageInfo, 1);
+        collect_image_info(bs, info, filename, fmt);
+        collect_snapshots(bs, info);
+
+        elem = g_new0(ImageInfoList, 1);
+        elem->value = info;
+        *last = elem;
+        last = &elem->next;
+
+        bdrv_delete(bs);
+
+        filename = fmt = NULL;
+        if (chain) {
+            if (info->has_full_backing_filename) {
+                filename = info->full_backing_filename;
+            } else if (info->has_backing_filename) {
+                filename = info->backing_filename;
+            }
+            if (info->has_backing_filename_format) {
+                fmt = info->backing_filename_format;
+            }
+        }
+    }
+    g_hash_table_destroy(filenames);
+    return head;
+
+err:
+    qapi_free_ImageInfoList(head);
+    g_hash_table_destroy(filenames);
+    return NULL;
+}
+
+enum {
+    OPTION_OUTPUT = 256,
+    OPTION_BACKING_CHAIN = 257,
+};
 
 typedef enum OutputFormat {
     OFORMAT_JSON,
@@ -1260,9 +1397,9 @@ static int img_info(int argc, char **argv)
 {
     int c;
     OutputFormat output_format = OFORMAT_HUMAN;
+    bool chain = false;
     const char *filename, *fmt, *output;
-    BlockDriverState *bs;
-    ImageInfo *info;
+    ImageInfoList *list;
 
     fmt = NULL;
     output = NULL;
@@ -1272,6 +1409,7 @@ static int img_info(int argc, char **argv)
             {"help", no_argument, 0, 'h'},
             {"format", required_argument, 0, 'f'},
             {"output", required_argument, 0, OPTION_OUTPUT},
+            {"backing-chain", no_argument, 0, OPTION_BACKING_CHAIN},
             {0, 0, 0, 0}
         };
         c = getopt_long(argc, argv, "f:h",
@@ -1290,6 +1428,9 @@ static int img_info(int argc, char **argv)
         case OPTION_OUTPUT:
             output = optarg;
             break;
+        case OPTION_BACKING_CHAIN:
+            chain = true;
+            break;
         }
     }
     if (optind >= argc) {
@@ -1306,27 +1447,25 @@ static int img_info(int argc, char **argv)
         return 1;
     }
 
-    bs = bdrv_new_open(filename, fmt, BDRV_O_FLAGS | BDRV_O_NO_BACKING, false);
-    if (!bs) {
+    list = collect_image_info_list(filename, fmt, chain);
+    if (!list) {
         return 1;
     }
 
-    info = g_new0(ImageInfo, 1);
-    collect_image_info(bs, info, filename, fmt);
-
     switch (output_format) {
     case OFORMAT_HUMAN:
-        dump_human_image_info(info);
-        dump_snapshots(bs);
+        dump_human_image_info_list(list);
         break;
     case OFORMAT_JSON:
-        collect_snapshots(bs, info);
-        dump_json_image_info(info);
+        if (chain) {
+            dump_json_image_info_list(list);
+        } else {
+            dump_json_image_info(list->value);
+        }
         break;
     }
 
-    qapi_free_ImageInfo(info);
-    bdrv_delete(bs);
+    qapi_free_ImageInfoList(list);
     return 0;
 }
 
-- 
1.7.11.7

[Qemu-devel] [PATCH v3 2/2] qemu-iotests: Add 043 backing file chain infinite loop test

[Stefan Hajnoczi]

This new test verifies that qemu-img info --backing-chain safely aborts
when an image file has a backing file infinite loop.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 tests/qemu-iotests/043       | 94 ++++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/043.out   | 66 +++++++++++++++++++++++++++++++
 tests/qemu-iotests/common.rc | 10 +++++
 tests/qemu-iotests/group     |  1 +
 4 files changed, 171 insertions(+)
 create mode 100755 tests/qemu-iotests/043
 create mode 100644 tests/qemu-iotests/043.out

diff --git a/tests/qemu-iotests/043 b/tests/qemu-iotests/043
new file mode 100755
index 0000000..bf390ac
--- /dev/null
+++ b/tests/qemu-iotests/043
@@ -0,0 +1,94 @@
+#!/bin/bash
+#
+# Test that qemu-img info --backing-chain detects infinite loops
+#
+# Copyright (C) 2012 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+# creator
+owner=stefanha@redhat.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+
+_cleanup()
+{
+	_cleanup_test_img
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+# Any format supporting backing files
+_supported_fmt qcow qcow2 vmdk qed
+_supported_proto generic
+_supported_os Linux
+
+
+size=128M
+_make_test_img $size
+$QEMU_IMG rebase -u -b $TEST_IMG $TEST_IMG
+
+echo
+echo "== backing file references self =="
+_img_info --backing-chain
+
+_make_test_img $size
+mv $TEST_IMG $TEST_IMG.base
+_make_test_img -b $TEST_IMG.base $size
+$QEMU_IMG rebase -u -b $TEST_IMG $TEST_IMG.base
+
+echo
+echo "== parent references self =="
+_img_info --backing-chain
+
+_make_test_img $size
+mv $TEST_IMG $TEST_IMG.1.base
+_make_test_img -b $TEST_IMG.1.base $size
+mv $TEST_IMG $TEST_IMG.2.base
+_make_test_img -b $TEST_IMG.2.base $size
+mv $TEST_IMG $TEST_IMG.3.base
+_make_test_img -b $TEST_IMG.3.base $size
+$QEMU_IMG rebase -u -b $TEST_IMG.2.base $TEST_IMG.1.base
+
+echo
+echo "== ancestor references another ancestor =="
+_img_info --backing-chain
+
+_make_test_img $size
+mv $TEST_IMG $TEST_IMG.1.base
+_make_test_img -b $TEST_IMG.1.base $size
+mv $TEST_IMG $TEST_IMG.2.base
+_make_test_img -b $TEST_IMG.2.base $size
+
+echo
+echo "== finite chain of length 3 (human) =="
+_img_info --backing-chain
+
+echo
+echo "== finite chain of length 3 (json) =="
+_img_info --backing-chain --output=json
+
+# success, all done
+echo "*** done"
+rm -f $seq.full $TEST_IMG.[123].base
+status=0
diff --git a/tests/qemu-iotests/043.out b/tests/qemu-iotests/043.out
new file mode 100644
index 0000000..ad23337
--- /dev/null
+++ b/tests/qemu-iotests/043.out
@@ -0,0 +1,66 @@
+QA output created by 043
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 
+
+== backing file references self ==
+qemu-img: Backing file 'TEST_DIR/t.IMGFMT' creates an infinite loop.
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file='TEST_DIR/t.IMGFMT.base' 
+
+== parent references self ==
+qemu-img: Backing file 'TEST_DIR/t.IMGFMT' creates an infinite loop.
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file='TEST_DIR/t.IMGFMT.1.base' 
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file='TEST_DIR/t.IMGFMT.2.base' 
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file='TEST_DIR/t.IMGFMT.3.base' 
+
+== ancestor references another ancestor ==
+qemu-img: Backing file 'TEST_DIR/t.IMGFMT.2.base' creates an infinite loop.
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file='TEST_DIR/t.IMGFMT.1.base' 
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file='TEST_DIR/t.IMGFMT.2.base' 
+
+== finite chain of length 3 (human) ==
+image: TEST_DIR/t.IMGFMT
+file format: IMGFMT
+virtual size: 128M (134217728 bytes)
+cluster_size: 65536
+backing file: TEST_DIR/t.IMGFMT.2.base
+
+image: TEST_DIR/t.IMGFMT.2.base
+file format: IMGFMT
+virtual size: 128M (134217728 bytes)
+cluster_size: 65536
+backing file: TEST_DIR/t.IMGFMT.1.base
+
+image: TEST_DIR/t.IMGFMT.1.base
+file format: IMGFMT
+virtual size: 128M (134217728 bytes)
+cluster_size: 65536
+
+== finite chain of length 3 (json) ==
+[
+    {
+        "virtual-size": 134217728, 
+        "filename": "TEST_DIR/t.IMGFMT", 
+        "cluster-size": 65536, 
+        "format": "IMGFMT", 
+        "backing-filename": "TEST_DIR/t.IMGFMT.2.base", 
+        "dirty-flag": false
+    }, 
+    {
+        "virtual-size": 134217728, 
+        "filename": "TEST_DIR/t.IMGFMT.2.base", 
+        "cluster-size": 65536, 
+        "format": "IMGFMT", 
+        "backing-filename": "TEST_DIR/t.IMGFMT.1.base", 
+        "dirty-flag": false
+    }, 
+    {
+        "virtual-size": 134217728, 
+        "filename": "TEST_DIR/t.IMGFMT.1.base", 
+        "cluster-size": 65536, 
+        "format": "IMGFMT", 
+        "dirty-flag": false
+    }
+]
+*** done
diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index d534e94..334534f 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -145,6 +145,16 @@ _check_test_img()
     	sed -e 's/qemu-img\: This image format does not support checks/No errors were found on the image./'
 }
 
+_img_info()
+{
+    $QEMU_IMG info "$@" $TEST_IMG 2>&1 | \
+        sed -e "s#$IMGPROTO:$TEST_DIR#TEST_DIR#g" \
+            -e "s#$TEST_DIR#TEST_DIR#g" \
+            -e "s#$IMGFMT#IMGFMT#g" \
+            -e "/^disk size:/ D" \
+            -e "/actual-size/ D"
+}
+
 _get_pids_by_name()
 {
     if [ $# -ne 1 ]
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 66d2ba9..a1b8917 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -47,3 +47,4 @@
 038 rw auto backing
 039 rw auto
 040 rw auto
+043 rw auto backing
-- 
1.7.11.7

Re: [Qemu-devel] [PATCH v3 0/2] qemu-img: Add --backing-chain option to info command

[Kevin Wolf]

Am 17.10.2012 14:02, schrieb Stefan Hajnoczi:
> This series adds the --backing-chain option for enumerating the backing file
> chain.  Given the topmost image it will print qemu-img info information for
> each image file in the chain.
> 
> Special care needs to be taken when image files form an infinite loop.  This is
> very unusual, most like due to malicious image files.  Nevertheless, qemu-img
> must be robust against invalid inputs so we explicit check for this.
> 
> v3:
>  * Use ImageInfoList to avoid manually outputting JSON and to make the code
>    more QAPI-friendly.
>  * Solve the output vs error messages issues by printing a detailed error
>    message but no image info.  To dig into a broken image chain, rerun without
>    --backing-chain.  This is different from what I initially tried but is much
>    cleaner because it doesn't produce confusing output.
>  * rm -f $TEST_IMG.[123].base [Kevin]
>  * Include --output=json in test case [Kevin]
>  * Rename test case to free 043 number [Eric]
> 
> Stefan Hajnoczi (2):
>   qemu-img: Add --backing-chain option to info command
>   qemu-iotests: Add 043 backing file chain infinite loop test
> 
>  qemu-img.c                   | 167 +++++++++++++++++++++++++++++++++++++++----
>  tests/qemu-iotests/043       |  94 ++++++++++++++++++++++++
>  tests/qemu-iotests/043.out   |  66 +++++++++++++++++
>  tests/qemu-iotests/common.rc |  10 +++
>  tests/qemu-iotests/group     |   1 +
>  5 files changed, 324 insertions(+), 14 deletions(-)
>  create mode 100755 tests/qemu-iotests/043
>  create mode 100644 tests/qemu-iotests/043.out

Thanks applied both to the block branch.

Kevin

Re: [Qemu-devel] [PATCH v3 0/2] qemu-img: Add --backing-chain option to info command

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 558 bytes --]

On 10/17/2012 08:47 AM, Kevin Wolf wrote:
> Am 17.10.2012 14:02, schrieb Stefan Hajnoczi:
>> This series adds the --backing-chain option for enumerating the backing file
>> chain.  Given the topmost image it will print qemu-img info information for
>> each image file in the chain.
>>
> 
> Thanks applied both to the block branch.

Now we also need the documentation:
https://lists.gnu.org/archive/html/qemu-devel/2012-10/msg02360.html

-- 
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 617 bytes --]