[Qemu-devel] [PATCH V7 0/2] Add JSON output to qemu-img info

[Qemu-devel] [PATCH V7 0/2] Add JSON output to qemu-img info

[Benoît Canet]

This patchset add a JSON output mode to the qemu-img info command.
It's a rewrite from scratch of the original patchset by Wenchao Xia
following Anthony Liguori advices on JSON formating.

the --output=(json|human) option is now mandatory on the command line.

Benoît Canet (3):
  qapi: Add SnapshotInfo.
  qapi: Add ImageInfo.
  qemu-img: Add json output option to the info command.

in v2:

eblake: make some field optionals
        squash the two qapi patchs together
        fix a typo on vm_clock_nsec

bcanet: fix a potential memory leak

in v3: 

lcapitulino: 
       remove unneeded test
       put '\n' at the end of json in printf statement
       drop the uneeded head pointer in collect_snapshots

in v4:

Wenchao Xia && Kevin Wolf: -Refactor to separate rate ImageInfo
                           collection from human printing.

Kevin Wolf: -Use --output=(json|human).
            -make the two choice exclusive and print a message
            if none is specified.
            -cosmetic '=' alignement in collect snapshots.

Benoît Canet: -add full-backing-filename to the ImageInfo structure
              (needed for human printing)
              -make ImageInfo->actual_size optional depending on the
              context.
in v5:

Eric Blake: -use a constant for getopt parsing to avoid future
	     short options collision.
            -make the command default to --output=human.
            -fix spurious whitespace change.
            -split vm-clock-nsec in two fields vm-clock-sec and
             vm-clock-nsec.
            -declare JSON structure as "Since 1.3" 
  
in v6:

Blue Swirl: -Add missing const in getopt structure declaration.

Eric Blake: -Remove spurious undef.
            -Use an enum instead of two boolean.

in v7:

Kevin Wolf: -Add missing documentation.
            -Change bogus comment about chained lists support in qapi.
            -Remove collect_backing_file_format and use bs->backing_format instead.
	    -Remove uneeded if.
            -Change bogus printing logic for backing file format.
            -Rename Format enum to OutputFormat.
            -Use OFORMAT_HUMAN by default and get rid of uneeded else branch
            -Use error_report insted of fprintf.
            -Use a switch instead of ifs.


Benoît Canet: Rename FORMAT_HUMAN and FORMAT_JSON to OFORMAT_JSON and OFORMAT_HUMAN

Benoît Canet (2):
  qapi: Add SnapshotInfo and ImageInfo.
  qemu-img: Add json output option to the info command.

 Makefile         |    3 +-
 qapi-schema.json |   64 +++++++++++++++
 qemu-img-cmds.hx |    4 +-
 qemu-img.c       |  232 +++++++++++++++++++++++++++++++++++++++++++-----------
 qemu-img.texi    |    5 +-
 5 files changed, 259 insertions(+), 49 deletions(-)

-- 
1.7.9.5

[Qemu-devel] [PATCH V7 1/2] qapi: Add SnapshotInfo and ImageInfo.

[Benoît Canet]

Signed-off-by: Benoit Canet <benoit@irqsave.net>
---
 qapi-schema.json |   64 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)

diff --git a/qapi-schema.json b/qapi-schema.json
index bd8ad74..8723795 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -156,6 +156,70 @@
             'running', 'save-vm', 'shutdown', 'suspended', 'watchdog' ] }
 
 ##
+# @SnapshotInfo
+#
+# @id: unique snapshot id
+#
+# @name: user chosen name
+#
+# @vm-state-size: size of the VM state
+#
+# @date-sec: UTC date of the snapshot in seconds
+#
+# @date-nsec: fractional part in nano seconds to be used with date-sec
+#
+# @vm-clock-sec: VM clock relative to boot in seconds
+#
+# @vm-clock-nsec: fractional part in nano seconds to be used with vm-clock-sec
+#
+# Since: 1.3
+#
+##
+
+{ 'type': 'SnapshotInfo',
+  'data': { 'id': 'str', 'name': 'str', 'vm-state-size': 'int',
+            'date-sec': 'int', 'date-nsec': 'int',
+            'vm-clock-sec': 'int', 'vm-clock-nsec': 'int' } }
+
+##
+# @ImageInfo:
+#
+# Information about a QEMU image file
+#
+# @filename: name of the image file
+#
+# @format: format of the image file
+#
+# @virtual-size: maximum capacity in bytes of the image
+#
+# @actual-size: #optional actual size on disk in bytes of the image
+#
+# @dirty-flag: #optional true if image is not cleanly closed
+#
+# @cluster-size: #optional size of a cluster in bytes
+#
+# @encrypted: #optional true if the image is encrypted
+#
+# @backing-filename: #optional name of the backing file
+#
+# @full-backing-filename: #optional full path of the backing file
+#
+# @backing-filename-format: #optional the format of the backing file
+#
+# @snapshots: #optional list of VM snapshots
+#
+# Since: 1.3
+#
+##
+
+{ 'type': 'ImageInfo',
+  'data': {'filename': 'str', 'format': 'str', '*dirty-flag': 'bool',
+           '*actual-size': 'int', 'virtual-size': 'int',
+           '*cluster-size': 'int', '*encrypted': 'bool',
+           '*backing-filename': 'str', '*full-backing-filename': 'str',
+           '*backing-filename-format': 'str', '*snapshots': ['SnapshotInfo'] } }
+
+##
 # @StatusInfo:
 #
 # Information about VCPU run state
-- 
1.7.9.5

[Qemu-devel] [PATCH V7 2/2] qemu-img: Add json output option to the info command.

[Benoît Canet]

This option --output=[human|json] make qemu-img info output on
human or JSON representation at the choice of the user.

example:
{
    "snapshots": [
        {
            "vm-clock-nsec": 637102488,
            "name": "vm-20120821145509",
            "date-sec": 1345553709,
            "date-nsec": 220289000,
            "vm-clock-sec": 20,
            "id": "1",
            "vm-state-size": 96522745
        },
        {
            "vm-clock-nsec": 28210866,
            "name": "vm-20120821154059",
            "date-sec": 1345556459,
            "date-nsec": 171392000,
            "vm-clock-sec": 46,
            "id": "2",
            "vm-state-size": 101208714
        }
    ],
    "virtual-size": 1073741824,
    "filename": "snap.qcow2",
    "cluster-size": 65536,
    "format": "qcow2",
    "actual-size": 985587712,
    "dirty-flag": false
}

Signed-off-by: Benoit Canet <benoit@irqsave.net>
---
 Makefile         |    3 +-
 qemu-img-cmds.hx |    4 +-
 qemu-img.c       |  232 +++++++++++++++++++++++++++++++++++++++++++-----------
 qemu-img.texi    |    5 +-
 4 files changed, 195 insertions(+), 49 deletions(-)

diff --git a/Makefile b/Makefile
index 1cd5bc8..971e92f 100644
--- a/Makefile
+++ b/Makefile
@@ -157,7 +157,8 @@ tools-obj-y = $(oslib-obj-y) $(trace-obj-y) qemu-tool.o qemu-timer.o \
 	iohandler.o cutils.o iov.o async.o
 tools-obj-$(CONFIG_POSIX) += compatfd.o
 
-qemu-img$(EXESUF): qemu-img.o $(tools-obj-y) $(block-obj-y)
+qemu-img$(EXESUF): qemu-img.o $(tools-obj-y) $(block-obj-y) $(qapi-obj-y) \
+                              qapi-visit.o qapi-types.o
 qemu-nbd$(EXESUF): qemu-nbd.o $(tools-obj-y) $(block-obj-y)
 qemu-io$(EXESUF): qemu-io.o cmd.o $(tools-obj-y) $(block-obj-y)
 
diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
index 39419a0..0ef82e9 100644
--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -34,9 +34,9 @@ STEXI
 ETEXI
 
 DEF("info", img_info,
-    "info [-f fmt] filename")
+    "info [-f fmt] [--output=ofmt] filename")
 STEXI
-@item info [-f @var{fmt}] @var{filename}
+@item info [-f @var{fmt}] [--output=@var{ofmt}] @var{filename}
 ETEXI
 
 DEF("snapshot", img_snapshot,
diff --git a/qemu-img.c b/qemu-img.c
index b41e670..399f384 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -21,12 +21,16 @@
  * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
  * THE SOFTWARE.
  */
+#include "qapi-visit.h"
+#include "qapi/qmp-output-visitor.h"
+#include "qjson.h"
 #include "qemu-common.h"
 #include "qemu-option.h"
 #include "qemu-error.h"
 #include "osdep.h"
 #include "sysemu.h"
 #include "block_int.h"
+#include <getopt.h>
 #include <stdio.h>
 
 #ifdef _WIN32
@@ -84,6 +88,7 @@ static void help(void)
            "  '-p' show progress of command (only certain commands)\n"
            "  '-S' indicates the consecutive number of bytes that must contain only zeros\n"
            "       for qemu-img to create a sparse image during conversion\n"
+           "  '--output' takes the format in which the output must be done (human or json)\n"
            "\n"
            "Parameters to check subcommand:\n"
            "  '-r' tries to repair any inconsistencies that are found during the check.\n"
@@ -1102,21 +1107,169 @@ static void dump_snapshots(BlockDriverState *bs)
     g_free(sn_tab);
 }
 
-static int img_info(int argc, char **argv)
+static void collect_snapshots(BlockDriverState *bs , ImageInfo *info)
+{
+    int i, sn_count;
+    QEMUSnapshotInfo *sn_tab = NULL;
+    SnapshotInfoList *info_list, *cur_item = NULL;
+    sn_count = bdrv_snapshot_list(bs, &sn_tab);
+
+    for (i = 0; i < sn_count; i++) {
+        info->has_snapshots = true;
+        info_list = g_new0(SnapshotInfoList, 1);
+
+        info_list->value                = g_new0(SnapshotInfo, 1);
+        info_list->value->id            = g_strdup(sn_tab[i].id_str);
+        info_list->value->name          = g_strdup(sn_tab[i].name);
+        info_list->value->vm_state_size = sn_tab[i].vm_state_size;
+        info_list->value->date_sec      = sn_tab[i].date_sec;
+        info_list->value->date_nsec     = sn_tab[i].date_nsec;
+        info_list->value->vm_clock_sec  = sn_tab[i].vm_clock_nsec / 1000000000;
+        info_list->value->vm_clock_nsec = sn_tab[i].vm_clock_nsec % 1000000000;
+
+        /* XXX: waiting for the qapi to support qemu-queue.h types */
+        if (!cur_item) {
+            info->snapshots = cur_item = info_list;
+        } else {
+            cur_item->next = info_list;
+            cur_item = info_list;
+        }
+
+    }
+
+    g_free(sn_tab);
+}
+
+static void dump_json_image_info(ImageInfo *info)
+{
+    Error *errp = NULL;
+    QString *str;
+    QmpOutputVisitor *ov = qmp_output_visitor_new();
+    QObject *obj;
+    visit_type_ImageInfo(qmp_output_get_visitor(ov),
+                         &info, NULL, &errp);
+    obj = qmp_output_get_qobject(ov);
+    str = qobject_to_json_pretty(obj);
+    assert(str != NULL);
+    printf("%s\n", qstring_get_str(str));
+    qobject_decref(obj);
+    qmp_output_visitor_cleanup(ov);
+    QDECREF(str);
+}
+
+static void collect_image_info(BlockDriverState *bs,
+                   ImageInfo *info,
+                   const char *filename,
+                   const char *fmt)
 {
-    int c;
-    const char *filename, *fmt;
-    BlockDriverState *bs;
-    char size_buf[128], dsize_buf[128];
     uint64_t total_sectors;
-    int64_t allocated_size;
     char backing_filename[1024];
     char backing_filename2[1024];
     BlockDriverInfo bdi;
 
+    bdrv_get_geometry(bs, &total_sectors);
+
+    info->filename        = g_strdup(filename);
+    info->format          = g_strdup(bdrv_get_format_name(bs));
+    info->virtual_size    = total_sectors * 512;
+    info->actual_size     = bdrv_get_allocated_file_size(bs);
+    info->has_actual_size = info->actual_size >= 0;
+    if (bdrv_is_encrypted(bs)) {
+        info->encrypted = true;
+        info->has_encrypted = true;
+    }
+    if (bdrv_get_info(bs, &bdi) >= 0) {
+        if (bdi.cluster_size != 0) {
+            info->cluster_size = bdi.cluster_size;
+            info->has_cluster_size = true;
+        }
+        info->dirty_flag = bdi.is_dirty;
+        info->has_dirty_flag = true;
+    }
+    bdrv_get_backing_filename(bs, backing_filename, sizeof(backing_filename));
+    if (backing_filename[0] != '\0') {
+        info->backing_filename = g_strdup(backing_filename);
+        info->has_backing_filename = true;
+        bdrv_get_full_backing_filename(bs, backing_filename2,
+                                       sizeof(backing_filename2));
+
+        if (strcmp(backing_filename, backing_filename2) != 0) {
+            info->full_backing_filename =
+                        g_strdup(backing_filename2);
+             info->has_full_backing_filename = true;
+        }
+
+        info->backing_filename_format = bs->backing_format;
+        info->has_backing_filename_format = true;
+    }
+}
+
+static void dump_human_image_info(ImageInfo *info)
+{
+    char size_buf[128], dsize_buf[128];
+    if (!info->has_actual_size) {
+        snprintf(dsize_buf, sizeof(dsize_buf), "unavailable");
+    } else {
+        get_human_readable_size(dsize_buf, sizeof(dsize_buf),
+                                info->actual_size);
+    }
+    get_human_readable_size(size_buf, sizeof(size_buf), info->virtual_size);
+    printf("image: %s\n"
+           "file format: %s\n"
+           "virtual size: %s (%" PRId64 " bytes)\n"
+           "disk size: %s\n",
+           info->filename, info->format, size_buf,
+           info->virtual_size,
+           dsize_buf);
+
+    if (info->has_encrypted && info->encrypted) {
+        printf("encrypted: yes\n");
+    }
+
+    if (info->has_cluster_size) {
+        printf("cluster_size: %" PRId64 "\n", info->cluster_size);
+    }
+
+    if (info->has_dirty_flag && info->dirty_flag) {
+        printf("cleanly shut down: no\n");
+    }
+
+    if (info->has_backing_filename) {
+        printf("backing file: %s", info->backing_filename);
+        if (info->has_full_backing_filename) {
+            printf(" (actual path: %s)", info->full_backing_filename);
+        }
+        putchar('\n');
+    }
+}
+
+enum {OPTION_OUTPUT = 256};
+
+typedef enum OutputFormat {
+    OFORMAT_JSON,
+    OFORMAT_HUMAN,
+} OutputFormat;
+
+static int img_info(int argc, char **argv)
+{
+    int c;
+    OutputFormat output_format = OFORMAT_HUMAN;
+    const char *filename, *fmt, *output;
+    BlockDriverState *bs;
+    ImageInfo *info;
+
     fmt = NULL;
+    output = NULL;
     for(;;) {
-        c = getopt(argc, argv, "f:h");
+        int option_index = 0;
+        static const struct option long_options[] = {
+            {"help", no_argument, 0, 'h'},
+            {"format", required_argument, 0, 'f'},
+            {"output", required_argument, 0, OPTION_OUTPUT},
+            {0, 0, 0, 0}
+        };
+        c = getopt_long(argc, argv, "f:h",
+                        long_options, &option_index);
         if (c == -1) {
             break;
         }
@@ -1128,6 +1281,9 @@ static int img_info(int argc, char **argv)
         case 'f':
             fmt = optarg;
             break;
+        case OPTION_OUTPUT:
+            output = optarg;
+            break;
         }
     }
     if (optind >= argc) {
@@ -1135,48 +1291,36 @@ static int img_info(int argc, char **argv)
     }
     filename = argv[optind++];
 
+    if (output && !strcmp(output, "json")) {
+        output_format = OFORMAT_JSON;
+    } else if (output && !strcmp(output, "human")) {
+        output_format = OFORMAT_HUMAN;
+    } else if (output) {
+        error_report("--output must be used with human or json as argument.");
+        return 1;
+    }
+
+    info = g_new0(ImageInfo, 1);
     bs = bdrv_new_open(filename, fmt, BDRV_O_FLAGS | BDRV_O_NO_BACKING);
     if (!bs) {
+        g_free(info);
         return 1;
     }
-    bdrv_get_geometry(bs, &total_sectors);
-    get_human_readable_size(size_buf, sizeof(size_buf), total_sectors * 512);
-    allocated_size = bdrv_get_allocated_file_size(bs);
-    if (allocated_size < 0) {
-        snprintf(dsize_buf, sizeof(dsize_buf), "unavailable");
-    } else {
-        get_human_readable_size(dsize_buf, sizeof(dsize_buf),
-                                allocated_size);
-    }
-    printf("image: %s\n"
-           "file format: %s\n"
-           "virtual size: %s (%" PRId64 " bytes)\n"
-           "disk size: %s\n",
-           filename, bdrv_get_format_name(bs), size_buf,
-           (total_sectors * 512),
-           dsize_buf);
-    if (bdrv_is_encrypted(bs)) {
-        printf("encrypted: yes\n");
-    }
-    if (bdrv_get_info(bs, &bdi) >= 0) {
-        if (bdi.cluster_size != 0) {
-            printf("cluster_size: %d\n", bdi.cluster_size);
-        }
-        if (bdi.is_dirty) {
-            printf("cleanly shut down: no\n");
-        }
-    }
-    bdrv_get_backing_filename(bs, backing_filename, sizeof(backing_filename));
-    if (backing_filename[0] != '\0') {
-        bdrv_get_full_backing_filename(bs, backing_filename2,
-                                       sizeof(backing_filename2));
-        printf("backing file: %s", backing_filename);
-        if (strcmp(backing_filename, backing_filename2) != 0) {
-            printf(" (actual path: %s)", backing_filename2);
-        }
-        putchar('\n');
+
+    collect_image_info(bs, info, filename, fmt);
+
+    switch (output_format) {
+    case OFORMAT_HUMAN:
+        dump_human_image_info(info);
+        dump_snapshots(bs);
+        break;
+    case OFORMAT_JSON:
+        collect_snapshots(bs, info);
+        dump_json_image_info(info);
+        break;
     }
-    dump_snapshots(bs);
+
+    qapi_free_ImageInfo(info);
     bdrv_delete(bs);
     return 0;
 }
diff --git a/qemu-img.texi b/qemu-img.texi
index 6b42e35..75a7c8b 100644
--- a/qemu-img.texi
+++ b/qemu-img.texi
@@ -129,12 +129,13 @@ created as a copy on write image of the specified base image; the
 @var{backing_file} should have the same content as the input's base image,
 however the path, image format, etc may differ.
 
-@item info [-f @var{fmt}] @var{filename}
+@item info [-f @var{fmt}] [--output=@var{ofmt}] @var{filename}
 
 Give information about the disk image @var{filename}. Use it in
 particular to know the size reserved on disk which can be different
 from the displayed size. If VM snapshots are stored in the disk image,
-they are displayed too.
+they are displayed too. The command can output in the format @var{ofmt}
+which is either human or json.
 
 @item snapshot [-l | -a @var{snapshot} | -c @var{snapshot} | -d @var{snapshot} ] @var{filename}
 
-- 
1.7.9.5

Re: [Qemu-devel] [PATCH V7 2/2] qemu-img: Add json output option to the info command.

[Kevin Wolf]

Am 05.09.2012 11:25, schrieb Benoît Canet:
> This option --output=[human|json] make qemu-img info output on
> human or JSON representation at the choice of the user.
> 
> example:
> {
>     "snapshots": [
>         {
>             "vm-clock-nsec": 637102488,
>             "name": "vm-20120821145509",
>             "date-sec": 1345553709,
>             "date-nsec": 220289000,
>             "vm-clock-sec": 20,
>             "id": "1",
>             "vm-state-size": 96522745
>         },
>         {
>             "vm-clock-nsec": 28210866,
>             "name": "vm-20120821154059",
>             "date-sec": 1345556459,
>             "date-nsec": 171392000,
>             "vm-clock-sec": 46,
>             "id": "2",
>             "vm-state-size": 101208714
>         }
>     ],
>     "virtual-size": 1073741824,
>     "filename": "snap.qcow2",
>     "cluster-size": 65536,
>     "format": "qcow2",
>     "actual-size": 985587712,
>     "dirty-flag": false
> }
> 
> Signed-off-by: Benoit Canet <benoit@irqsave.net>

Looks much better, but I still have a few comments:

> +    bdrv_get_backing_filename(bs, backing_filename, sizeof(backing_filename));
> +    if (backing_filename[0] != '\0') {
> +        info->backing_filename = g_strdup(backing_filename);
> +        info->has_backing_filename = true;
> +        bdrv_get_full_backing_filename(bs, backing_filename2,
> +                                       sizeof(backing_filename2));
> +
> +        if (strcmp(backing_filename, backing_filename2) != 0) {
> +            info->full_backing_filename =
> +                        g_strdup(backing_filename2);
> +             info->has_full_backing_filename = true;
> +        }
> +
> +        info->backing_filename_format = bs->backing_format;
> +        info->has_backing_filename_format = true;

I believe we should check bs->backing_format[0] first. If it's empty,
don't include the format.

> +    }
> +}
> +
> +static void dump_human_image_info(ImageInfo *info)
> +{
> +    char size_buf[128], dsize_buf[128];
> +    if (!info->has_actual_size) {
> +        snprintf(dsize_buf, sizeof(dsize_buf), "unavailable");
> +    } else {
> +        get_human_readable_size(dsize_buf, sizeof(dsize_buf),
> +                                info->actual_size);
> +    }
> +    get_human_readable_size(size_buf, sizeof(size_buf), info->virtual_size);
> +    printf("image: %s\n"
> +           "file format: %s\n"
> +           "virtual size: %s (%" PRId64 " bytes)\n"
> +           "disk size: %s\n",
> +           info->filename, info->format, size_buf,
> +           info->virtual_size,
> +           dsize_buf);
> +
> +    if (info->has_encrypted && info->encrypted) {
> +        printf("encrypted: yes\n");
> +    }
> +
> +    if (info->has_cluster_size) {
> +        printf("cluster_size: %" PRId64 "\n", info->cluster_size);
> +    }
> +
> +    if (info->has_dirty_flag && info->dirty_flag) {
> +        printf("cleanly shut down: no\n");
> +    }
> +
> +    if (info->has_backing_filename) {
> +        printf("backing file: %s", info->backing_filename);
> +        if (info->has_full_backing_filename) {
> +            printf(" (actual path: %s)", info->full_backing_filename);
> +        }

Maybe add the backing file format here if it's available?

> @@ -1135,48 +1291,36 @@ static int img_info(int argc, char **argv)
>      }
>      filename = argv[optind++];
>  
> +    if (output && !strcmp(output, "json")) {
> +        output_format = OFORMAT_JSON;
> +    } else if (output && !strcmp(output, "human")) {
> +        output_format = OFORMAT_HUMAN;
> +    } else if (output) {
> +        error_report("--output must be used with human or json as argument.");
> +        return 1;
> +    }
> +
> +    info = g_new0(ImageInfo, 1);
>      bs = bdrv_new_open(filename, fmt, BDRV_O_FLAGS | BDRV_O_NO_BACKING);
>      if (!bs) {
> +        g_free(info);
>          return 1;
>      }

If you move thee g_new0 below, you don't have to free here.

> diff --git a/qemu-img.texi b/qemu-img.texi
> index 6b42e35..75a7c8b 100644
> --- a/qemu-img.texi
> +++ b/qemu-img.texi
> @@ -129,12 +129,13 @@ created as a copy on write image of the specified base image; the
>  @var{backing_file} should have the same content as the input's base image,
>  however the path, image format, etc may differ.
>  
> -@item info [-f @var{fmt}] @var{filename}
> +@item info [-f @var{fmt}] [--output=@var{ofmt}] @var{filename}
>  
>  Give information about the disk image @var{filename}. Use it in
>  particular to know the size reserved on disk which can be different
>  from the displayed size. If VM snapshots are stored in the disk image,
> -they are displayed too.
> +they are displayed too. The command can output in the format @var{ofmt}
> +which is either human or json.

"...@code{human} or @code{json}" is better, I think.

Kevin