[Qemu-devel] [PATCH 0/4 V5] char: Add CirMemCharDriver and provide QMP interface

[Qemu-devel] [PATCH 0/4 V5] char: Add CirMemCharDriver and provide QMP interface

[Lei Li]

This patch series attempts to add new char backend CirMemCharDriver with
a circular buffer and expose it to users by introducing QMP interface
memchar-write and memchar-read and via the command line like the other
CharDriverStates.

Serial ports in qemu always use CharDriverStates as there backends,
Right now, all of our backends always try to write the data from the
guest to a socket or file. The concern from OpenStack is that this could
lead to unbounded disk space usage since they log the serial output.
For more detail of the background info:
https://bugs.launchpad.net/nova/+bug/832507

So we want to use a circular buffer in QEMU instead, and then OpenStack
can periodically read the buffer in QEMU and log it.

The QMP commands introduced like:

{ 'command': 'memchar-write',
  'data': {'chardev': 'str', 'size': 'int', 'data': 'str',
           'format': 'str' } }

{ 'command': 'memchar-read',
  'data': {'chardev': 'str', 'size': 'int', 'format': 'str' },
  'returns': 'str' }

Expose CirMemCharDriver via the command line like:

qemu -chardev memchr,id=foo,maxcapacity=640k -serial chardev:foo

Introduce HMP command 'console' like:

(qemu) console foo
foo: Input data

Note:
Now all of the feature were implemented, and the pervious comments
are fixed up too. Please comment and let me know if there is anything
else need to be improved. Your suggestion would be very appreciated!

Changes since v4:
  - Get rid of all CongestionControl bits, and assume a dropping behavior
    based on Luiz's suggestion for now. Will add it when we add async
    support to QMP.
  - Squashed the patches about CirMemCharDriver in one.
  - Other fixups from Luiz.

Changes since v3:
  - Improve the algorithm of circular buffer based on Anthony's
    suggestion.
  - Some changes suggested by Luiz and Blue.
  - And other fixups.

Changes since v2:
  - Add congestion mechanism. For the 'block' option as sync command,
    will support it later when we gain the necessary infrastructure
    enhancement.
  - Add HMP 'console' command so that can interact with multiple
    chardevs via a single monitor socket.
  - Make the circular buffer backend and the current MemCharDriver
    live in parallel, expose a new char backend with circular buffer
    CirMemCharDriver suggested by Luiz.
  - Other fixs from Eric and Markus.

Changes since v1:
  - Exposing the MemCharDriver via command line.
  - Support base64 data format suggested by Anthony and Eric.
  - Follow the new rule for the name of qmp command from Eric.


Lei Li (4):
  qemu-char: Add new char backend CirMemCharDriver
  QAPI: Introduce memchar-write QMP command
  QAPI: Introduce memchar-read QMP command
  HMP: Introduce console command

 hmp-commands.hx  |   72 +++++++++++++++++++
 hmp.c            |   99 +++++++++++++++++++++++
 hmp.h            |    3 +
 monitor.c        |   15 ++++
 monitor.h        |    3 +
 qapi-schema.json |   96 +++++++++++++++++++++++++
 qemu-char.c      |  217 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 qemu-config.c    |    3 +
 qemu-options.hx  |   10 +++
 qmp-commands.hx  |   89 +++++++++++++++++++++
 10 files changed, 607 insertions(+), 0 deletions(-)

[Qemu-devel] [PATCH 1/4] qemu-char: Add new char backend CirMemCharDriver

[Lei Li]

Signed-off-by: Lei Li <lilei@linux.vnet.ibm.com>
---
 qemu-char.c     |  136 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 qemu-config.c   |    3 +
 qemu-options.hx |   10 ++++
 3 files changed, 149 insertions(+), 0 deletions(-)

diff --git a/qemu-char.c b/qemu-char.c
index b082bae..45d2a86 100644
--- a/qemu-char.c
+++ b/qemu-char.c
@@ -99,6 +99,7 @@
 #include "ui/qemu-spice.h"
 
 #define READ_BUF_LEN 4096
+#define CBUFF_SIZE 65536
 
 /***********************************************************/
 /* character device */
@@ -2588,6 +2589,130 @@ size_t qemu_chr_mem_osize(const CharDriverState *chr)
     return d->outbuf_size;
 }
 
+/*********************************************************/
+/*CircularMemoryr chardev*/
+
+typedef struct {
+    size_t size;
+    size_t producer;
+    size_t consumer;
+    uint8_t *cbuf;
+} CirMemCharDriver;
+
+static bool cirmem_chr_is_empty(const CharDriverState *chr)
+{
+    const CirMemCharDriver *d = chr->opaque;
+
+    return d->producer == d->consumer;
+}
+
+static bool cirmem_chr_is_full(const CharDriverState *chr)
+{
+    const CirMemCharDriver *d = chr->opaque;
+
+    return (d->producer - d->consumer) >= d->size;
+}
+
+static bool cirmem_chr_unlimit(const CharDriverState *chr)
+{
+    const CirMemCharDriver *d = chr->opaque;
+
+    return d->producer >= d->consumer;
+}
+
+static int cirmem_chr_write(CharDriverState *chr, const uint8_t *buf, int len)
+{
+    CirMemCharDriver *d = chr->opaque;
+    int i;
+
+    if (!buf || (len < 0)) {
+        return -1;
+    }
+
+    for (i = 0; i < len; i++ ) {
+        if (cirmem_chr_unlimit(chr)) {
+            d->cbuf[d->producer % d->size] = buf[i];
+            d->producer++;
+        } else {
+            return -1;
+        }
+    }
+
+    return 0;
+}
+
+static int cirmem_chr_read(CharDriverState *chr, uint8_t *buf, int len)
+{
+    CirMemCharDriver *d = chr->opaque;
+    int i;
+
+    if (cirmem_chr_is_empty(chr) || len < 0) {
+        return -1;
+    }
+
+    if (len > d->size) {
+        len = d->size;
+    }
+
+    for (i = 0; i < len; i++) {
+        if (cirmem_chr_unlimit(chr)) {
+            buf[i] = d->cbuf[d->consumer % d->size];
+            d->consumer++;
+
+            if (cirmem_chr_is_empty(chr)) {
+                break;
+            }
+        } else {
+            return -1;
+        }
+    }
+
+    return 0;
+}
+
+static void cirmem_chr_close(struct CharDriverState *chr)
+{
+    CirMemCharDriver *d = chr->opaque;
+
+    g_free(d->cbuf);
+    g_free(d);
+    chr->opaque = NULL;
+}
+
+static CharDriverState *qemu_chr_open_cirmemchr(QemuOpts *opts)
+{
+    CharDriverState *chr;
+    CirMemCharDriver *d;
+
+    chr = g_malloc0(sizeof(CharDriverState));
+    d = g_malloc(sizeof(*d));
+
+    d->size = qemu_opt_get_number(opts, "maxcapacity", 0);
+    if (d->size == 0) {
+        d->size = CBUFF_SIZE;
+    }
+
+    /* The size must be power of 2 */
+    if (d->size & (d->size - 1)) {
+        goto fail;
+    }
+
+    d->producer = 0;
+    d->consumer = 0;
+    d->cbuf = g_malloc0(d->size);
+
+    chr->opaque = d;
+    chr->chr_write = cirmem_chr_write;
+    chr->chr_close = cirmem_chr_close;
+
+    return chr;
+
+fail:
+    g_free(d);
+    g_free(chr);
+    return NULL;
+}
+
 QemuOpts *qemu_chr_parse_compat(const char *label, const char *filename)
 {
     char host[65], port[33], width[8], height[8];
@@ -2652,6 +2777,16 @@ QemuOpts *qemu_chr_parse_compat(const char *label, const char *filename)
         qemu_opt_set(opts, "path", p);
         return opts;
     }
+    if (strstart(filename, "memchr", &p)) {
+        qemu_opt_set(opts, "backend", "memchr");
+        qemu_opt_set(opts, "maxcapacity", p);
+        return opts;
+    }
+    if (strstart(filename, "memchr", &p)) {
+        qemu_opt_set(opts, "backend", "memchr");
+        qemu_opt_set(opts, "maxcapacity", p);
+        return opts;
+    }
     if (strstart(filename, "tcp:", &p) ||
         strstart(filename, "telnet:", &p)) {
         if (sscanf(p, "%64[^:]:%32[^,]%n", host, port, &pos) < 2) {
@@ -2725,6 +2860,7 @@ static const struct {
     { .name = "udp",       .open = qemu_chr_open_udp },
     { .name = "msmouse",   .open = qemu_chr_open_msmouse },
     { .name = "vc",        .open = text_console_init },
+    { .name = "memchr",    .open = qemu_chr_open_cirmemchr },
 #ifdef _WIN32
     { .name = "file",      .open = qemu_chr_open_win_file_out },
     { .name = "pipe",      .open = qemu_chr_open_win_pipe },
diff --git a/qemu-config.c b/qemu-config.c
index cd1ec21..5553bb6 100644
--- a/qemu-config.c
+++ b/qemu-config.c
@@ -213,6 +213,9 @@ static QemuOptsList qemu_chardev_opts = {
         },{
             .name = "debug",
             .type = QEMU_OPT_NUMBER,
+        },{
+            .name = "maxcapacity",
+            .type = QEMU_OPT_NUMBER,
         },
         { /* end of list */ }
     },
diff --git a/qemu-options.hx b/qemu-options.hx
index 46f0539..9ccf930 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -1680,6 +1680,7 @@ DEF("chardev", HAS_ARG, QEMU_OPTION_chardev,
     "-chardev msmouse,id=id[,mux=on|off]\n"
     "-chardev vc,id=id[[,width=width][,height=height]][[,cols=cols][,rows=rows]]\n"
     "         [,mux=on|off]\n"
+    "-chardev memchr,id=id,maxcapacity=maxcapacity\n"
     "-chardev file,id=id,path=path[,mux=on|off]\n"
     "-chardev pipe,id=id,path=path[,mux=on|off]\n"
 #ifdef _WIN32
@@ -1718,6 +1719,7 @@ Backend is one of:
 @option{udp},
 @option{msmouse},
 @option{vc},
+@option{memchr},
 @option{file},
 @option{pipe},
 @option{console},
@@ -1824,6 +1826,14 @@ the console, in pixels.
 @option{cols} and @option{rows} specify that the console be sized to fit a text
 console with the given dimensions.
 
+@item -chardev memchr ,id=@var{id} ,maxcapacity=@var{maxcapacity}
+
+Create a circular buffer with fixed size indicated by optionally @option{maxcapacity}
+which will be default 64K if it is not given.
+
+@option{maxcapacity} specify the max capacity of the size of circular buffer
+want to create. Should be power of 2.
+
 @item -chardev file ,id=@var{id} ,path=@var{path}
 
 Log all traffic received from the guest to a file.
-- 
1.7.7.6

[Qemu-devel] [PATCH 2/4] QAPI: Introduce memchar-write QMP command

[Lei Li]

Signed-off-by: Lei Li <lilei@linux.vnet.ibm.com>
---
 hmp-commands.hx  |   17 +++++++++++++++++
 hmp.c            |   15 +++++++++++++++
 hmp.h            |    1 +
 qapi-schema.json |   47 +++++++++++++++++++++++++++++++++++++++++++++++
 qemu-char.c      |   44 ++++++++++++++++++++++++++++++++++++++++++++
 qmp-commands.hx  |   34 ++++++++++++++++++++++++++++++++++
 6 files changed, 158 insertions(+), 0 deletions(-)

diff --git a/hmp-commands.hx b/hmp-commands.hx
index e0b537d..a37b8e9 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -825,6 +825,23 @@ Inject an NMI on the given CPU (x86 only).
 ETEXI
 
     {
+        .name       = "memchar_write",
+        .args_type  = "chardev:s,data:s",
+        .params     = "chardev data",
+        .mhandler.cmd = hmp_memchar_write,
+    },
+
+STEXI
+@item memchar_write @var{chardev} @var{data}
+@findex memchar_write
+Provide writing interface for CirMemCharDriver. Write @var{data}
+to cirmemchr char device. Note that we  will add 'control' options
+for read and write command that specifies behavior when the queue
+is full/empty, for now just assume a drop behaver in these two commands.
+
+ETEXI
+
+    {
         .name       = "migrate",
         .args_type  = "detach:-d,blk:-b,inc:-i,uri:s",
         .params     = "[-d] [-b] [-i] uri",
diff --git a/hmp.c b/hmp.c
index 2b97982..082985b 100644
--- a/hmp.c
+++ b/hmp.c
@@ -683,6 +683,21 @@ void hmp_pmemsave(Monitor *mon, const QDict *qdict)
     hmp_handle_error(mon, &errp);
 }
 
+void hmp_memchar_write(Monitor *mon, const QDict *qdict)
+{
+    uint32_t size;
+    const char *chardev = qdict_get_str(qdict, "chardev");
+    const char *data = qdict_get_str(qdict, "data");
+    enum DataFormat format;
+    Error *errp = NULL;
+
+    size = strlen(data);
+    format = DATA_FORMAT_UTF8;
+    qmp_memchar_write(chardev, size, data, true, format, &errp);
+
+    hmp_handle_error(mon, &errp);
+}
+
 static void hmp_cont_cb(void *opaque, int err)
 {
     if (!err) {
diff --git a/hmp.h b/hmp.h
index 71ea384..406ebb1 100644
--- a/hmp.h
+++ b/hmp.h
@@ -43,6 +43,7 @@ void hmp_system_powerdown(Monitor *mon, const QDict *qdict);
 void hmp_cpu(Monitor *mon, const QDict *qdict);
 void hmp_memsave(Monitor *mon, const QDict *qdict);
 void hmp_pmemsave(Monitor *mon, const QDict *qdict);
+void hmp_memchar_write(Monitor *mon, const QDict *qdict);
 void hmp_cont(Monitor *mon, const QDict *qdict);
 void hmp_system_wakeup(Monitor *mon, const QDict *qdict);
 void hmp_inject_nmi(Monitor *mon, const QDict *qdict);
diff --git a/qapi-schema.json b/qapi-schema.json
index c615ee2..43ef6bc 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -325,6 +325,53 @@
 { 'command': 'query-chardev', 'returns': ['ChardevInfo'] }
 
 ##
+# @DataFormat:
+#
+# An enumeration of data format. The default value would
+# be utf8.
+#
+# @utf8: The data format is 'utf8'.
+#
+# @base64: The data format is 'base64'.
+#
+# Note: The data format start with 'utf8' and 'base64',
+#       will support other data format as well.
+#
+# Since: 1.3
+##
+{ 'enum': 'DataFormat'
+  'data': [ 'utf8', 'base64' ] }
+
+##
+# @memchar-write:
+#
+# Provide writing interface for memchardev. Write data to memchar
+# char device.
+#
+# @chardev: the name of the memchar char device.
+#
+# @size: the size to write in bytes. Should be power of 2.
+#
+# @data: the source data write to memchar.
+#
+# @format: #optional the format of the data write to memchardev, by
+#          default is 'utf8'.
+#
+# Returns: Nothing on success
+#          If @chardev is not a valid memchr device, DeviceNotFound
+#
+# Notes: The option 'block' is not supported now due to the miss
+#        feature in qmp. Will add it later when we gain the necessary
+#        infrastructure enhancement. For now just assume 'drop' behaver
+#        for this command.
+#
+# Since: 1.3
+##
+{ 'command': 'memchar-write',
+  'data': {'chardev': 'str', 'size': 'int', 'data': 'str',
+           '*format': 'DataFormat'} }
+
+##
 # @CommandInfo:
 #
 # Information about a QMP command
diff --git a/qemu-char.c b/qemu-char.c
index 45d2a86..78fc634 100644
--- a/qemu-char.c
+++ b/qemu-char.c
@@ -2713,6 +2713,50 @@ fail:
     return NULL;
 }
 
+void qmp_memchar_write(const char *chardev, int64_t size,
+                       const char *data, bool has_format,
+                       enum DataFormat format,
+                       Error **errp)
+{
+    CharDriverState *chr;
+    guchar *write_data;
+    int ret;
+    gsize write_count;
+
+    chr = qemu_chr_find(chardev);
+    if (!chr) {
+        error_set(errp, QERR_DEVICE_NOT_FOUND, chardev);
+        return;
+    }
+
+    if (strcmp(chr->filename, "memchr") != 0) {
+        error_setg(errp,"%s is not memory char device\n", chardev);
+        return;
+    }
+
+    /* XXX: For the sync command as 'block', waiting for the qmp
+     * to support necessary feature. Now just act as 'drop' */
+    if (cirmem_chr_is_full(chr)) {
+        error_setg(errp, "Failed to write to memchr %s", chardev);
+        return;
+    }
+
+    write_count = (gsize)size;
+
+    if (has_format && (format == DATA_FORMAT_BASE64)) {
+        write_data = g_base64_decode(data, &write_count);
+    } else {
+        write_data = (uint8_t *)data;
+    }
+
+    ret = cirmem_chr_write(chr, write_data, write_count);
+
+    if (ret < 0) {
+        error_setg(errp, "Failed to write to memchr %s", chardev);
+        return;
+    }
+}
+
 QemuOpts *qemu_chr_parse_compat(const char *label, const char *filename)
 {
     char host[65], port[33], width[8], height[8];
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 5ba8c48..7548b9b 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -466,6 +466,40 @@ Note: inject-nmi fails when the guest doesn't support injecting.
 EQMP
 
     {
+        .name       = "memchar-write",
+        .args_type  = "chardev:s,size:i,data:s,format:s?,control:s?",
+        .help       = "write size of data to memchar chardev",
+        .mhandler.cmd_new = qmp_marshal_input_memchar_write,
+    },
+
+SQMP
+memchar-write
+-------------
+
+Provide writing interface for memchardev. Write data to memchar
+char device.
+
+Arguments:
+
+- "chardev": the name of the char device, must be unique (json-string)
+- "size": the memory size, in bytes, should be power of 2 (json-int)
+- "data": the source data writed to memchar (json-string)
+- "format": the data format write to memchardev, default is
+            utf8. (json-string, optional)
+          - Possible values: "utf8", "base64"
+
+Example:
+
+-> { "execute": "memchar-write",
+                "arguments": { "chardev": foo,
+                               "size": 8,
+                               "data": "abcdefgh",
+                               "format": "utf8" } }
+<- { "return": {} }
+
+EQMP
+
+    {
         .name       = "xen-save-devices-state",
         .args_type  = "filename:F",
     .mhandler.cmd_new = qmp_marshal_input_xen_save_devices_state,
-- 
1.7.7.6

[Qemu-devel] [PATCH 3/4] QAPI: Introduce memchar-read QMP command

[Lei Li]

Signed-off-by: Lei Li <lilei@linux.vnet.ibm.com>
---
 hmp-commands.hx  |   19 ++++++++++++++++++
 hmp.c            |   19 ++++++++++++++++++
 hmp.h            |    1 +
 qapi-schema.json |   27 ++++++++++++++++++++++++++
 qemu-char.c      |   55 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 qmp-commands.hx  |   40 +++++++++++++++++++++++++++++++++++++++
 6 files changed, 161 insertions(+), 0 deletions(-)

diff --git a/hmp-commands.hx b/hmp-commands.hx
index a37b8e9..df294eb 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -842,6 +842,25 @@ is full/empty, for now just assume a drop behaver in these two commands.
 ETEXI
 
     {
+        .name       = "memchar_read",
+        .args_type  = "chardev:s,size:i",
+        .params     = "chardev size",
+        .mhandler.cmd = hmp_memchar_read,
+    },
+
+STEXI
+@item memchar_read @var{chardev}
+@findex memchar_read
+Provide read interface for CirMemCharDriver. Read from cirmemchr
+char device and return @var{size} of the data.
+
+@var{size} is the size of data want to read from. Refer to unencoded
+size of the raw data, would adjust to the init size of the memchar
+if the requested size is larger than it.
+
+ETEXI
+
+    {
         .name       = "migrate",
         .args_type  = "detach:-d,blk:-b,inc:-i,uri:s",
         .params     = "[-d] [-b] [-i] uri",
diff --git a/hmp.c b/hmp.c
index 082985b..ef85736 100644
--- a/hmp.c
+++ b/hmp.c
@@ -698,6 +698,25 @@ void hmp_memchar_write(Monitor *mon, const QDict *qdict)
     hmp_handle_error(mon, &errp);
 }
 
+void hmp_memchar_read(Monitor *mon, const QDict *qdict)
+{
+    uint32_t size = qdict_get_int(qdict, "size");
+    const char *chardev = qdict_get_str(qdict, "chardev");
+    char *data;
+    enum DataFormat format;
+    Error *errp = NULL;
+
+    format = DATA_FORMAT_UTF8;
+    data = qmp_memchar_read(chardev, size, true, format, &errp);
+    if (errp) {
+        monitor_printf(mon, "%s\n", error_get_pretty(errp));
+        error_free(errp);
+        return;
+    }
+
+    monitor_printf(mon, "%s\n", data);
+}
+
 static void hmp_cont_cb(void *opaque, int err)
 {
     if (!err) {
diff --git a/hmp.h b/hmp.h
index 406ebb1..a5a0cfe 100644
--- a/hmp.h
+++ b/hmp.h
@@ -44,6 +44,7 @@ void hmp_cpu(Monitor *mon, const QDict *qdict);
 void hmp_memsave(Monitor *mon, const QDict *qdict);
 void hmp_pmemsave(Monitor *mon, const QDict *qdict);
 void hmp_memchar_write(Monitor *mon, const QDict *qdict);
+void hmp_memchar_read(Monitor *mon, const QDict *qdict);
 void hmp_cont(Monitor *mon, const QDict *qdict);
 void hmp_system_wakeup(Monitor *mon, const QDict *qdict);
 void hmp_inject_nmi(Monitor *mon, const QDict *qdict);
diff --git a/qapi-schema.json b/qapi-schema.json
index 43ef6bc..a8c9430 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -372,6 +372,33 @@
            '*format': 'DataFormat'} }
 
 ##
+# @memchar-read:
+#
+# Provide read interface for memchardev. Read from memchar
+# char device and return the data.
+#
+# @chardev: the name of the memchar char device.
+#
+# @size: the size to read in bytes.
+#
+# @format: #optional the format of the data want to read from
+#          memchardev, by default is 'utf8'.
+#
+# Returns: The data read from memchar as string
+#          If @chardev is not a valid memchr device, DeviceNotFound
+#
+# Notes: The option 'block' is not supported now due to the miss
+#        feature in qmp. Will add it later when we gain the necessary
+#        infrastructure enhancement. For now just assume 'drop' behaver
+#        for this command.
+#
+# Since: 1.3
+##
+{ 'command': 'memchar-read',
+  'data': {'chardev': 'str', 'size': 'int', '*format': 'DataFormat'},
+  'returns': 'str' }
+
+##
 # @CommandInfo:
 #
 # Information about a QMP command
diff --git a/qemu-char.c b/qemu-char.c
index 78fc634..a7c5ea2 100644
--- a/qemu-char.c
+++ b/qemu-char.c
@@ -2757,6 +2757,61 @@ void qmp_memchar_write(const char *chardev, int64_t size,
     }
 }
 
+char *qmp_memchar_read(const char *chardev, int64_t size,
+                       bool has_format, enum DataFormat format,
+                       Error **errp)
+{
+    CharDriverState *chr;
+    guchar *read_data;
+    char *data = NULL;
+    int ret;
+
+    read_data = g_malloc0(size);
+
+    chr = qemu_chr_find(chardev);
+    if (!chr) {
+        error_set(errp, QERR_DEVICE_NOT_FOUND, chardev);
+        goto fail;
+    }
+
+    if (strcmp(chr->filename, "memchr") != 0) {
+        error_setg(errp,"The %s is not memory char device\n", chardev);
+        goto fail;
+    }
+
+    /* XXX: For the sync command as 'block', waiting for the qmp
+     * to support necessary feature. Now just act as 'drop'. */
+    if (cirmem_chr_is_empty(chr)) {
+        error_setg(errp, "Failed to read from memchr %s", chardev);
+        goto fail;
+    }
+
+    if (size == 0) {
+        size = CBUFF_SIZE;
+    }
+
+    ret = cirmem_chr_read(chr, read_data, size);
+
+    if (ret < 0) {
+        error_setg(errp, "Failed to read from memchr %s", chardev);
+        goto fail;
+    }
+
+    if (has_format && (format == DATA_FORMAT_BASE64)) {
+       if (read_data) {
+           data = g_base64_encode(read_data, (size_t)size);
+       }
+    } else {
+        data = (char *)read_data;
+    }
+
+    return data;
+
+fail:
+    g_free(read_data);
+    return NULL;
+}
+
 QemuOpts *qemu_chr_parse_compat(const char *label, const char *filename)
 {
     char host[65], port[33], width[8], height[8];
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 7548b9b..7729fb0 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -500,6 +500,46 @@ Example:
 EQMP
 
     {
+        .name       = "memchar-read",
+        .args_type  = "chardev:s,size:i,format:s?",
+        .help       = "return the size of data from memchar chardev",
+        .mhandler.cmd_new = qmp_marshal_input_memchar_read,
+    },
+
+SQMP
+memchar-read
+-------------
+
+Provide read interface for memchardev. Read from memchar
+char device and return the data.
+
+Arguments:
+
+- "chardev": the name of the char device, must be unique (json-string)
+- "size": the memory size wanted to read in bytes(refer to unencoded
+          size of the raw data), would adjust to the init size of the
+          memchar if the requested size is larger than it. (json-int)
+- "format": the data format write to memchardev, default is
+            utf8. (json-string, optional)
+          - Possible values: "utf8", "base64"
+
+Example:
+
+-> { "execute": "memchar-read",
+                "arguments": { "chardev": foo,
+                               "size": 1000,
+                               "format": "utf8" } }
+<- { "return": "data string..." }
+
+Notes:
+
+We will add 'control' options for read and write command that specifies
+behavior when the queue is full/empty, for now just assume a drop
+behaver in these two commands.
+
+EQMP
+
+    {
         .name       = "xen-save-devices-state",
         .args_type  = "filename:F",
     .mhandler.cmd_new = qmp_marshal_input_xen_save_devices_state,
-- 
1.7.7.6

[Qemu-devel] [PATCH 4/4] HMP: Introduce console command

[Lei Li]

Signed-off-by: Lei Li <lilei@linux.vnet.ibm.com>
---
 hmp-commands.hx |   25 +++++++++++++++++++++++++
 hmp.c           |   52 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 hmp.h           |    1 +
 monitor.c       |   15 +++++++++++++++
 monitor.h       |    3 +++
 5 files changed, 96 insertions(+), 0 deletions(-)

diff --git a/hmp-commands.hx b/hmp-commands.hx
index df294eb..7cba42c 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -861,6 +861,31 @@ if the requested size is larger than it.
 ETEXI
 
     {
+        .name       = "console",
+        .args_type  = "chardev:s",
+        .params     = "chardev",
+        .help       = "Connect to the serial console from within the"
+                      "monitor, allow to write data to memchardev"
+                      "'chardev'. Exit from the console and return back"
+                      "to monitor by typing 'ctrl-]'",
+        .mhandler.cmd = hmp_console,
+    },
+
+STEXI
+@item console @var{device}
+@findex console
+
+Connect to the serial console from within the monitor, allow to write data
+to memchardev @var{chardev}. Exit from the console and return back to
+monitor by typing 'ctrl-]'.
+@example
+(qemu) console foo
+foo: data string...
+@end example
+
+ETEXI
+
+    {
         .name       = "migrate",
         .args_type  = "detach:-d,blk:-b,inc:-i,uri:s",
         .params     = "[-d] [-b] [-i] uri",
diff --git a/hmp.c b/hmp.c
index ef85736..d716410 100644
--- a/hmp.c
+++ b/hmp.c
@@ -1255,3 +1255,55 @@ void hmp_screen_dump(Monitor *mon, const QDict *qdict)
     qmp_screendump(filename, &err);
     hmp_handle_error(mon, &err);
 }
+
+enum escape_char
+{
+    ESCAPE_CHAR_CTRL_GS = 0x1d  /* ctrl-] used for escape */
+};
+
+static void hmp_read_console(Monitor *mon, const char *data,
+                             void *opaque)
+{
+    CharDriverState *chr = opaque;
+    uint32_t size = strlen(data);
+    enum DataFormat format = DATA_FORMAT_UTF8;
+    enum escape_char console_escape = ESCAPE_CHAR_CTRL_GS;
+
+    Error *err = NULL;
+
+    if (*data == console_escape) {
+        monitor_resume(mon);
+        return;
+    }
+
+    qmp_memchar_write(chr->label, size, data, 0, format, &err);
+
+    if (err) {
+        monitor_printf(mon, "%s\n", error_get_pretty(err));
+        error_free(err);
+        return;
+    }
+
+    monitor_read_command(mon, 1);
+}
+
+void hmp_console(Monitor *mon, const QDict *qdict)
+{
+    const char *device = qdict_get_str(qdict, "chardev");
+    CharDriverState *chr;
+    Error *err = NULL;
+
+    chr = qemu_chr_find(device);
+
+    if (!chr) {
+        error_set(&err, QERR_DEVICE_NOT_FOUND, device);
+        goto out;
+    }
+
+    if (monitor_read_console(mon, device, hmp_read_console, chr) < 0) {
+        monitor_printf(mon, "Connect to console %s failed\n", device);
+    }
+
+out:
+    hmp_handle_error(mon, &err);
+}
diff --git a/hmp.h b/hmp.h
index a5a0cfe..5b54a79 100644
--- a/hmp.h
+++ b/hmp.h
@@ -77,5 +77,6 @@ void hmp_getfd(Monitor *mon, const QDict *qdict);
 void hmp_closefd(Monitor *mon, const QDict *qdict);
 void hmp_send_key(Monitor *mon, const QDict *qdict);
 void hmp_screen_dump(Monitor *mon, const QDict *qdict);
+void hmp_console(Monitor *mon, const QDict *qdict);
 
 #endif
diff --git a/monitor.c b/monitor.c
index d17ae2d..7e90115 100644
--- a/monitor.c
+++ b/monitor.c
@@ -256,6 +256,21 @@ int monitor_read_password(Monitor *mon, ReadLineFunc *readline_func,
     }
 }
 
+int monitor_read_console(Monitor *mon, const char *device,
+                         ReadLineFunc *readline_func, void *opaque)
+{
+    char prompt[60];
+
+    if (!mon->rs) {
+        return -1;
+    }
+
+    snprintf(prompt, sizeof(prompt), "%s: ", device);
+    readline_start(mon->rs, prompt, 0, readline_func, opaque);
+
+    return 0;
+}
+
 void monitor_flush(Monitor *mon)
 {
     if (mon && mon->outbuf_index != 0 && !mon->mux_out) {
diff --git a/monitor.h b/monitor.h
index b6e7d95..735bd1b 100644
--- a/monitor.h
+++ b/monitor.h
@@ -86,6 +86,9 @@ ReadLineState *monitor_get_rs(Monitor *mon);
 int monitor_read_password(Monitor *mon, ReadLineFunc *readline_func,
                           void *opaque);
 
+int monitor_read_console(Monitor *mon, const char *device,
+                         ReadLineFunc *readline_func, void *opaque);
+
 int qmp_qom_set(Monitor *mon, const QDict *qdict, QObject **ret);
 
 int qmp_qom_get(Monitor *mon, const QDict *qdict, QObject **ret);
-- 
1.7.7.6

[Qemu-devel] [PATCH 0/4 V5] char: Add CirMemCharDriver and provide QMP interface

[Lei Li]

This patch series attempts to add new char backend CirMemCharDriver with
a circular buffer and expose it to users by introducing QMP interface
memchar-write and memchar-read and via the command line like the other
CharDriverStates.

Serial ports in qemu always use CharDriverStates as there backends,
Right now, all of our backends always try to write the data from the
guest to a socket or file. The concern from OpenStack is that this could
lead to unbounded disk space usage since they log the serial output.
For more detail of the background info:
https://bugs.launchpad.net/nova/+bug/832507

So we want to use a circular buffer in QEMU instead, and then OpenStack
can periodically read the buffer in QEMU and log it.

The QMP commands introduced like:

{ 'command': 'memchar-write',
  'data': {'chardev': 'str', 'size': 'int', 'data': 'str',
           'format': 'str' } }

{ 'command': 'memchar-read',
  'data': {'chardev': 'str', 'size': 'int', 'format': 'str' },
  'returns': 'str' }

Expose CirMemCharDriver via the command line like:

qemu -chardev memchr,id=foo,maxcapacity=640k -serial chardev:foo

Introduce HMP command 'console' like:

(qemu) console foo
foo: Input data

Note:
Now all of the feature were implemented, and the pervious comments
are fixed up too. Please comment and let me know if there is anything
else need to be improved. Your suggestion would be very appreciated!

Changes since v4:
  - Get rid of all CongestionControl bits, and assume a dropping behavior
    based on Luiz's suggestion for now. Will add it when we add async
    support to QMP.
  - Squashed the patches about CirMemCharDriver in one.
  - Other fixups from Luiz.

Changes since v3:
  - Improve the algorithm of circular buffer based on Anthony's
    suggestion.
  - Some changes suggested by Luiz and Blue.
  - And other fixups.

Changes since v2:
  - Add congestion mechanism. For the 'block' option as sync command,
    will support it later when we gain the necessary infrastructure
    enhancement.
  - Add HMP 'console' command so that can interact with multiple
    chardevs via a single monitor socket.
  - Make the circular buffer backend and the current MemCharDriver
    live in parallel, expose a new char backend with circular buffer
    CirMemCharDriver suggested by Luiz.
  - Other fixs from Eric and Markus.

Changes since v1:
  - Exposing the MemCharDriver via command line.
  - Support base64 data format suggested by Anthony and Eric.
  - Follow the new rule for the name of qmp command from Eric.


Lei Li (4):
  qemu-char: Add new char backend CirMemCharDriver
  QAPI: Introduce memchar-write QMP command
  QAPI: Introduce memchar-read QMP command
  HMP: Introduce console command

 hmp-commands.hx  |   72 +++++++++++++++++++
 hmp.c            |   99 +++++++++++++++++++++++
 hmp.h            |    3 +
 monitor.c        |   15 ++++
 monitor.h        |    3 +
 qapi-schema.json |   96 +++++++++++++++++++++++++
 qemu-char.c      |  217 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 qemu-config.c    |    3 +
 qemu-options.hx  |   10 +++
 qmp-commands.hx  |   89 +++++++++++++++++++++
 10 files changed, 607 insertions(+), 0 deletions(-)

Re: [Qemu-devel] [PATCH 1/4] qemu-char: Add new char backend CirMemCharDriver

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1657 bytes --]

On 10/25/2012 01:48 PM, Lei Li wrote:
> Signed-off-by: Lei Li <lilei@linux.vnet.ibm.com>
> ---
>  qemu-char.c     |  136 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  qemu-config.c   |    3 +
>  qemu-options.hx |   10 ++++
>  3 files changed, 149 insertions(+), 0 deletions(-)
> 
> diff --git a/qemu-char.c b/qemu-char.c
> index b082bae..45d2a86 100644
> --- a/qemu-char.c
> +++ b/qemu-char.c
> @@ -99,6 +99,7 @@
>  #include "ui/qemu-spice.h"
>  
>  #define READ_BUF_LEN 4096
> +#define CBUFF_SIZE 65536
>  
>  /***********************************************************/
>  /* character device */
> @@ -2588,6 +2589,130 @@ size_t qemu_chr_mem_osize(const CharDriverState *chr)
>      return d->outbuf_size;
>  }
>  
> +/*********************************************************/
> +/*CircularMemoryr chardev*/

s/CircularMemoryr/CircularMemory/

This typo was pointed out to you in v4; makes me wonder what else you
missed while preparing v5.

> @@ -1824,6 +1826,14 @@ the console, in pixels.
>  @option{cols} and @option{rows} specify that the console be sized to fit a text
>  console with the given dimensions.
>  
> +@item -chardev memchr ,id=@var{id} ,maxcapacity=@var{maxcapacity}
> +
> +Create a circular buffer with fixed size indicated by optionally @option{maxcapacity}
> +which will be default 64K if it is not given.
> +
> +@option{maxcapacity} specify the max capacity of the size of circular buffer
> +want to create. Should be power of 2.

s/specify/specifies/
s/want//

-- 
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 617 bytes --]

Re: [Qemu-devel] [PATCH 0/4 V5] char: Add CirMemCharDriver and provide QMP interface

[Lei Li]

Please ignore this thread and review the other thread also titled V5.
Sorry for the mis-sending mostly caused by the unconsciousness
at midnight...

On 10/26/2012 03:48 AM, Lei Li wrote:
> This patch series attempts to add new char backend CirMemCharDriver with
> a circular buffer and expose it to users by introducing QMP interface
> memchar-write and memchar-read and via the command line like the other
> CharDriverStates.
>
> Serial ports in qemu always use CharDriverStates as there backends,
> Right now, all of our backends always try to write the data from the
> guest to a socket or file. The concern from OpenStack is that this could
> lead to unbounded disk space usage since they log the serial output.
> For more detail of the background info:
> https://bugs.launchpad.net/nova/+bug/832507
>
> So we want to use a circular buffer in QEMU instead, and then OpenStack
> can periodically read the buffer in QEMU and log it.
>
> The QMP commands introduced like:
>
> { 'command': 'memchar-write',
>    'data': {'chardev': 'str', 'size': 'int', 'data': 'str',
>             'format': 'str' } }
>
> { 'command': 'memchar-read',
>    'data': {'chardev': 'str', 'size': 'int', 'format': 'str' },
>    'returns': 'str' }
>
> Expose CirMemCharDriver via the command line like:
>
> qemu -chardev memchr,id=foo,maxcapacity=640k -serial chardev:foo
>
> Introduce HMP command 'console' like:
>
> (qemu) console foo
> foo: Input data
>
> Note:
> Now all of the feature were implemented, and the pervious comments
> are fixed up too. Please comment and let me know if there is anything
> else need to be improved. Your suggestion would be very appreciated!
>
> Changes since v4:
>    - Get rid of all CongestionControl bits, and assume a dropping behavior
>      based on Luiz's suggestion for now. Will add it when we add async
>      support to QMP.
>    - Squashed the patches about CirMemCharDriver in one.
>    - Other fixups from Luiz.
>
> Changes since v3:
>    - Improve the algorithm of circular buffer based on Anthony's
>      suggestion.
>    - Some changes suggested by Luiz and Blue.
>    - And other fixups.
>
> Changes since v2:
>    - Add congestion mechanism. For the 'block' option as sync command,
>      will support it later when we gain the necessary infrastructure
>      enhancement.
>    - Add HMP 'console' command so that can interact with multiple
>      chardevs via a single monitor socket.
>    - Make the circular buffer backend and the current MemCharDriver
>      live in parallel, expose a new char backend with circular buffer
>      CirMemCharDriver suggested by Luiz.
>    - Other fixs from Eric and Markus.
>
> Changes since v1:
>    - Exposing the MemCharDriver via command line.
>    - Support base64 data format suggested by Anthony and Eric.
>    - Follow the new rule for the name of qmp command from Eric.
>
>
> Lei Li (4):
>    qemu-char: Add new char backend CirMemCharDriver
>    QAPI: Introduce memchar-write QMP command
>    QAPI: Introduce memchar-read QMP command
>    HMP: Introduce console command
>
>   hmp-commands.hx  |   72 +++++++++++++++++++
>   hmp.c            |   99 +++++++++++++++++++++++
>   hmp.h            |    3 +
>   monitor.c        |   15 ++++
>   monitor.h        |    3 +
>   qapi-schema.json |   96 +++++++++++++++++++++++++
>   qemu-char.c      |  217 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
>   qemu-config.c    |    3 +
>   qemu-options.hx  |   10 +++
>   qmp-commands.hx  |   89 +++++++++++++++++++++
>   10 files changed, 607 insertions(+), 0 deletions(-)
>
>


-- 
Lei