* [Qemu-devel] [QAPI+QGA 1/3] Error propagation and JSON parser fix-ups
@ 2011-06-01 17:14 Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 01/14] QError: Introduce qerror_format_desc() Michael Roth
` (13 more replies)
0 siblings, 14 replies; 16+ messages in thread
From: Michael Roth @ 2011-06-01 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: aliguori, Jes.Sorensen, agl, mdroth, lcapitulino
This is Set 1/3 of the QAPI+QGA patchsets.
These patches apply on top of master (5-31), and can also be obtained from:
git://repo.or.cz/qemu/mdroth.git qapi-backport-set1-v1
(Set1+2 are a backport of some of the QAPI-related work from Anthony's
glib tree. The main goal is to get the basic code generation infrastructure in
place so that it can be used by the guest agent to implement a QMP-like guest
interface, and so that future work regarding the QMP conversion to QAPI can be
decoupled from the infrastructure bits. Set3 is the Qemu Guest Agent
(virtagent), rebased on the new code QAPI code generation infrastructure. This
is the first user of QAPI, QMP will follow.)
___
This patchset introduces the following:
- a new error-handling framework with support for exception-like error
propagation. This error-handling will be used by QAPI and the guest agent,
initially, and other users will be converted over time.
- various hardening of QEMU's json parsing routines, mainly: limits on max
token size and token count, immediately propagating lexer errors to the
parser (inducing a NULL qobject to be output) to avoid churning on tokens
after a lexer error, stricter handling of invalid UTF-8 characters, and
some minor bug fixes.
CHANGES SINCE V0 ("QAPI Infrastructure Round 1"):
- Rebased on Luiz's backport of the error-handling patches
- Added JSON-related patches
Makefile | 4 +-
Makefile.objs | 2 +-
error.c | 140 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
error.h | 70 +++++++++++++++++++++++++++
error_int.h | 29 +++++++++++
json-lexer.c | 47 ++++++++++++++++---
json-lexer.h | 1 +
json-parser.c | 83 ++++++++++++++++++++++++++++++--
json-parser.h | 2 +
json-streamer.c | 42 +++++++++++++++--
json-streamer.h | 1 +
qerror.c | 59 ++++++++++++++++-------
qerror.h | 4 ++
13 files changed, 446 insertions(+), 38 deletions(-)
^ permalink raw reply [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v1][ 01/14] QError: Introduce qerror_format_desc()
2011-06-01 17:14 [Qemu-devel] [QAPI+QGA 1/3] Error propagation and JSON parser fix-ups Michael Roth
@ 2011-06-01 17:14 ` Michael Roth
2011-06-07 19:18 ` Anthony Liguori
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 02/14] QError: Introduce qerror_format() Michael Roth
` (12 subsequent siblings)
13 siblings, 1 reply; 16+ messages in thread
From: Michael Roth @ 2011-06-01 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: aliguori, Jes.Sorensen, agl, mdroth, lcapitulino
From: Luiz Capitulino <lcapitulino@redhat.com>
Refactor non-QError-specific bits out of qerror_human() into general
function that can be used by the error_get_pretty() analogue in the
new error-propagation framework.
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
qerror.c | 44 +++++++++++++++++++++++++-------------------
1 files changed, 25 insertions(+), 19 deletions(-)
diff --git a/qerror.c b/qerror.c
index 4855604..af6ed39 100644
--- a/qerror.c
+++ b/qerror.c
@@ -326,12 +326,14 @@ QError *qerror_from_info(const char *file, int linenr, const char *func,
return qerr;
}
-static void parse_error(const QError *qerror, int c)
+static void parse_error(const QErrorStringTable *entry, int c)
{
- qerror_abort(qerror, "expected '%c' in '%s'", c, qerror->entry->desc);
+ fprintf(stderr, "expected '%c' in '%s'", c, entry->desc);
+ abort();
}
-static const char *append_field(QString *outstr, const QError *qerror,
+static const char *append_field(QDict *error, QString *outstr,
+ const QErrorStringTable *entry,
const char *start)
{
QObject *obj;
@@ -340,23 +342,23 @@ static const char *append_field(QString *outstr, const QError *qerror,
const char *end, *key;
if (*start != '%')
- parse_error(qerror, '%');
+ parse_error(entry, '%');
start++;
if (*start != '(')
- parse_error(qerror, '(');
+ parse_error(entry, '(');
start++;
end = strchr(start, ')');
if (!end)
- parse_error(qerror, ')');
+ parse_error(entry, ')');
key_qs = qstring_from_substr(start, 0, end - start - 1);
key = qstring_get_str(key_qs);
- qdict = qobject_to_qdict(qdict_get(qerror->error, "data"));
+ qdict = qobject_to_qdict(qdict_get(error, "data"));
obj = qdict_get(qdict, key);
if (!obj) {
- qerror_abort(qerror, "key '%s' not found in QDict", key);
+ abort();
}
switch (qobject_type(obj)) {
@@ -367,35 +369,31 @@ static const char *append_field(QString *outstr, const QError *qerror,
qstring_append_int(outstr, qdict_get_int(qdict, key));
break;
default:
- qerror_abort(qerror, "invalid type '%c'", qobject_type(obj));
+ abort();
}
QDECREF(key_qs);
return ++end;
}
-/**
- * qerror_human(): Format QError data into human-readable string.
- *
- * Formats according to member 'desc' of the specified QError object.
- */
-QString *qerror_human(const QError *qerror)
+static QString *qerror_format_desc(QDict *error,
+ const QErrorStringTable *entry)
{
- const char *p;
QString *qstring;
+ const char *p;
- assert(qerror->entry != NULL);
+ assert(entry != NULL);
qstring = qstring_new();
- for (p = qerror->entry->desc; *p != '\0';) {
+ for (p = entry->desc; *p != '\0';) {
if (*p != '%') {
qstring_append_chr(qstring, *p++);
} else if (*(p + 1) == '%') {
qstring_append_chr(qstring, '%');
p += 2;
} else {
- p = append_field(qstring, qerror, p);
+ p = append_field(error, qstring, entry, p);
}
}
@@ -403,6 +401,14 @@ QString *qerror_human(const QError *qerror)
}
/**
+ * qerror_human(): Format QError data into human-readable string.
+ */
+QString *qerror_human(const QError *qerror)
+{
+ return qerror_format_desc(qerror->error, qerror->entry);
+}
+
+/**
* qerror_print(): Print QError data
*
* This function will print the member 'desc' of the specified QError object,
--
1.7.0.4
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v1][ 02/14] QError: Introduce qerror_format()
2011-06-01 17:14 [Qemu-devel] [QAPI+QGA 1/3] Error propagation and JSON parser fix-ups Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 01/14] QError: Introduce qerror_format_desc() Michael Roth
@ 2011-06-01 17:14 ` Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 03/14] Introduce the new error framework Michael Roth
` (11 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Michael Roth @ 2011-06-01 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: aliguori, Jes.Sorensen, agl, mdroth, lcapitulino
From: Luiz Capitulino <lcapitulino@redhat.com>
Will be used by new error propagation framework to convert Error objects
into human-readable form.
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
qerror.c | 15 +++++++++++++++
qerror.h | 1 +
2 files changed, 16 insertions(+), 0 deletions(-)
diff --git a/qerror.c b/qerror.c
index af6ed39..c18641f 100644
--- a/qerror.c
+++ b/qerror.c
@@ -400,6 +400,21 @@ static QString *qerror_format_desc(QDict *error,
return qstring;
}
+QString *qerror_format(const char *fmt, QDict *error)
+{
+ const QErrorStringTable *entry = NULL;
+ int i;
+
+ for (i = 0; qerror_table[i].error_fmt; i++) {
+ if (strcmp(qerror_table[i].error_fmt, fmt) == 0) {
+ entry = &qerror_table[i];
+ break;
+ }
+ }
+
+ return qerror_format_desc(error, entry);
+}
+
/**
* qerror_human(): Format QError data into human-readable string.
*/
diff --git a/qerror.h b/qerror.h
index df61d2c..13ad9d4 100644
--- a/qerror.h
+++ b/qerror.h
@@ -39,6 +39,7 @@ QString *qerror_human(const QError *qerror);
void qerror_print(QError *qerror);
void qerror_report_internal(const char *file, int linenr, const char *func,
const char *fmt, ...) GCC_FMT_ATTR(4, 5);
+QString *qerror_format(const char *fmt, QDict *error);
#define qerror_report(fmt, ...) \
qerror_report_internal(__FILE__, __LINE__, __func__, fmt, ## __VA_ARGS__)
QError *qobject_to_qerror(const QObject *obj);
--
1.7.0.4
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v1][ 03/14] Introduce the new error framework
2011-06-01 17:14 [Qemu-devel] [QAPI+QGA 1/3] Error propagation and JSON parser fix-ups Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 01/14] QError: Introduce qerror_format_desc() Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 02/14] QError: Introduce qerror_format() Michael Roth
@ 2011-06-01 17:14 ` Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 04/14] json-parser: propagate error from parser Michael Roth
` (10 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Michael Roth @ 2011-06-01 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: aliguori, Jes.Sorensen, agl, mdroth, lcapitulino
From: Luiz Capitulino <lcapitulino@redhat.com>
New error-handling framework that allows for exception-like error
propagation.
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
Makefile.objs | 2 +-
error.c | 140 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
error.h | 70 ++++++++++++++++++++++++++++
error_int.h | 29 ++++++++++++
4 files changed, 240 insertions(+), 1 deletions(-)
create mode 100644 error.c
create mode 100644 error.h
create mode 100644 error_int.h
diff --git a/Makefile.objs b/Makefile.objs
index 90838f6..0f7bb52 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -59,7 +59,7 @@ fsdev-obj-$(CONFIG_VIRTFS) += $(addprefix fsdev/, $(fsdev-nested-y))
# system emulation, i.e. a single QEMU executable should support all
# CPUs and machines.
-common-obj-y = $(block-obj-y) blockdev.o
+common-obj-y = $(block-obj-y) blockdev.o error.o
common-obj-y += $(net-obj-y)
common-obj-y += $(qobject-obj-y)
common-obj-$(CONFIG_LINUX) += $(fsdev-obj-$(CONFIG_LINUX))
diff --git a/error.c b/error.c
new file mode 100644
index 0000000..867eec2
--- /dev/null
+++ b/error.c
@@ -0,0 +1,140 @@
+/*
+ * QEMU Error Objects
+ *
+ * Copyright IBM, Corp. 2011
+ *
+ * Authors:
+ * Anthony Liguori <aliguori@us.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2. See
+ * the COPYING.LIB file in the top-level directory.
+ */
+#include "error.h"
+#include "error_int.h"
+#include "qemu-objects.h"
+#include "qerror.h"
+#include <assert.h>
+
+struct Error
+{
+ QDict *obj;
+ const char *fmt;
+ char *msg;
+};
+
+void error_set(Error **errp, const char *fmt, ...)
+{
+ Error *err;
+ va_list ap;
+
+ if (errp == NULL) {
+ return;
+ }
+
+ err = qemu_mallocz(sizeof(*err));
+
+ va_start(ap, fmt);
+ err->obj = qobject_to_qdict(qobject_from_jsonv(fmt, &ap));
+ va_end(ap);
+ err->fmt = fmt;
+
+ *errp = err;
+}
+
+bool error_is_set(Error **errp)
+{
+ return (errp && *errp);
+}
+
+const char *error_get_pretty(Error *err)
+{
+ if (err->msg == NULL) {
+ QString *str;
+ str = qerror_format(err->fmt, err->obj);
+ err->msg = qemu_strdup(qstring_get_str(str));
+ QDECREF(str);
+ }
+
+ return err->msg;
+}
+
+const char *error_get_field(Error *err, const char *field)
+{
+ if (strcmp(field, "class") == 0) {
+ return qdict_get_str(err->obj, field);
+ } else {
+ QDict *dict = qdict_get_qdict(err->obj, "data");
+ return qdict_get_str(dict, field);
+ }
+}
+
+QDict *error_get_data(Error *err)
+{
+ QDict *data = qdict_get_qdict(err->obj, "data");
+ QINCREF(data);
+ return data;
+}
+
+void error_set_field(Error *err, const char *field, const char *value)
+{
+ QDict *dict = qdict_get_qdict(err->obj, "data");
+ return qdict_put(dict, field, qstring_from_str(value));
+}
+
+void error_free(Error *err)
+{
+ if (err) {
+ QDECREF(err->obj);
+ qemu_free(err->msg);
+ qemu_free(err);
+ }
+}
+
+bool error_is_type(Error *err, const char *fmt)
+{
+ const char *error_class;
+ char *ptr;
+ char *end;
+
+ ptr = strstr(fmt, "'class': '");
+ assert(ptr != NULL);
+ ptr += strlen("'class': '");
+
+ end = strchr(ptr, '\'');
+ assert(end != NULL);
+
+ error_class = error_get_field(err, "class");
+ if (strlen(error_class) != end - ptr) {
+ return false;
+ }
+
+ return strncmp(ptr, error_class, end - ptr) == 0;
+}
+
+void error_propagate(Error **dst_err, Error *local_err)
+{
+ if (dst_err) {
+ *dst_err = local_err;
+ } else if (local_err) {
+ error_free(local_err);
+ }
+}
+
+QObject *error_get_qobject(Error *err)
+{
+ QINCREF(err->obj);
+ return QOBJECT(err->obj);
+}
+
+void error_set_qobject(Error **errp, QObject *obj)
+{
+ Error *err;
+ if (errp == NULL) {
+ return;
+ }
+ err = qemu_mallocz(sizeof(*err));
+ err->obj = qobject_to_qdict(obj);
+ qobject_incref(obj);
+
+ *errp = err;
+}
diff --git a/error.h b/error.h
new file mode 100644
index 0000000..003c855
--- /dev/null
+++ b/error.h
@@ -0,0 +1,70 @@
+/*
+ * QEMU Error Objects
+ *
+ * Copyright IBM, Corp. 2011
+ *
+ * Authors:
+ * Anthony Liguori <aliguori@us.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2. See
+ * the COPYING.LIB file in the top-level directory.
+ */
+#ifndef ERROR_H
+#define ERROR_H
+
+#include <stdbool.h>
+
+/**
+ * A class representing internal errors within QEMU. An error has a string
+ * typename and optionally a set of named string parameters.
+ */
+typedef struct Error Error;
+
+/**
+ * Set an indirect pointer to an error given a printf-style format parameter.
+ * Currently, qerror.h defines these error formats. This function is not
+ * meant to be used outside of QEMU.
+ */
+void error_set(Error **err, const char *fmt, ...)
+ __attribute__((format(printf, 2, 3)));
+
+/**
+ * Returns true if an indirect pointer to an error is pointing to a valid
+ * error object.
+ */
+bool error_is_set(Error **err);
+
+/**
+ * Get a human readable representation of an error object.
+ */
+const char *error_get_pretty(Error *err);
+
+/**
+ * Get an individual named error field.
+ */
+const char *error_get_field(Error *err, const char *field);
+
+/**
+ * Get an individual named error field.
+ */
+void error_set_field(Error *err, const char *field, const char *value);
+
+/**
+ * Propagate an error to an indirect pointer to an error. This function will
+ * always transfer ownership of the error reference and handles the case where
+ * dst_err is NULL correctly.
+ */
+void error_propagate(Error **dst_err, Error *local_err);
+
+/**
+ * Free an error object.
+ */
+void error_free(Error *err);
+
+/**
+ * Determine if an error is of a speific type (based on the qerror format).
+ * Non-QEMU users should get the `class' field to identify the error type.
+ */
+bool error_is_type(Error *err, const char *fmt);
+
+#endif
diff --git a/error_int.h b/error_int.h
new file mode 100644
index 0000000..5e39424
--- /dev/null
+++ b/error_int.h
@@ -0,0 +1,29 @@
+/*
+ * QEMU Error Objects
+ *
+ * Copyright IBM, Corp. 2011
+ *
+ * Authors:
+ * Anthony Liguori <aliguori@us.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2. See
+ * the COPYING.LIB file in the top-level directory.
+ */
+#ifndef QEMU_ERROR_INT_H
+#define QEMU_ERROR_INT_H
+
+#include "qemu-common.h"
+#include "qobject.h"
+#include "qdict.h"
+#include "error.h"
+
+/**
+ * Internal QEMU functions for working with Error.
+ *
+ * These are used to convert QErrors to Errors
+ */
+QDict *error_get_data(Error *err);
+QObject *error_get_qobject(Error *err);
+void error_set_qobject(Error **errp, QObject *obj);
+
+#endif
--
1.7.0.4
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v1][ 04/14] json-parser: propagate error from parser
2011-06-01 17:14 [Qemu-devel] [QAPI+QGA 1/3] Error propagation and JSON parser fix-ups Michael Roth
` (2 preceding siblings ...)
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 03/14] Introduce the new error framework Michael Roth
@ 2011-06-01 17:14 ` Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 05/14] json-streamer: allow recovery after bad input Michael Roth
` (9 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Michael Roth @ 2011-06-01 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: aliguori, Jes.Sorensen, agl, mdroth, lcapitulino
From: Anthony Liguori <aliguori@us.ibm.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
Makefile | 4 ++--
Makefile.objs | 4 ++--
json-parser.c | 19 ++++++++++++++++---
json-parser.h | 2 ++
qerror.h | 3 +++
5 files changed, 25 insertions(+), 7 deletions(-)
diff --git a/Makefile b/Makefile
index 2b0438c..d7944c8 100644
--- a/Makefile
+++ b/Makefile
@@ -134,14 +134,14 @@ qemu-img-cmds.h: $(SRC_PATH)/qemu-img-cmds.hx
check-qint.o check-qstring.o check-qdict.o check-qlist.o check-qfloat.o check-qjson.o: $(GENERATED_HEADERS)
-CHECK_PROG_DEPS = qemu-malloc.o $(oslib-obj-y) $(trace-obj-y)
+CHECK_PROG_DEPS = qemu-malloc.o $(oslib-obj-y) $(trace-obj-y) qemu-tool.o
check-qint: check-qint.o qint.o $(CHECK_PROG_DEPS)
check-qstring: check-qstring.o qstring.o $(CHECK_PROG_DEPS)
check-qdict: check-qdict.o qdict.o qfloat.o qint.o qstring.o qbool.o qlist.o $(CHECK_PROG_DEPS)
check-qlist: check-qlist.o qlist.o qint.o $(CHECK_PROG_DEPS)
check-qfloat: check-qfloat.o qfloat.o $(CHECK_PROG_DEPS)
-check-qjson: check-qjson.o qfloat.o qint.o qdict.o qstring.o qlist.o qbool.o qjson.o json-streamer.o json-lexer.o json-parser.o $(CHECK_PROG_DEPS)
+check-qjson: check-qjson.o qfloat.o qint.o qdict.o qstring.o qlist.o qbool.o qjson.o json-streamer.o json-lexer.o json-parser.o error.o qerror.o qemu-error.o $(CHECK_PROG_DEPS)
QEMULIBS=libhw32 libhw64 libuser libdis libdis-user
diff --git a/Makefile.objs b/Makefile.objs
index 0f7bb52..66ffad4 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -2,7 +2,7 @@
# QObject
qobject-obj-y = qint.o qstring.o qdict.o qlist.o qfloat.o qbool.o
qobject-obj-y += qjson.o json-lexer.o json-streamer.o json-parser.o
-qobject-obj-y += qerror.o
+qobject-obj-y += qerror.o error.o
#######################################################################
# oslib-obj-y is code depending on the OS (win32 vs posix)
@@ -59,7 +59,7 @@ fsdev-obj-$(CONFIG_VIRTFS) += $(addprefix fsdev/, $(fsdev-nested-y))
# system emulation, i.e. a single QEMU executable should support all
# CPUs and machines.
-common-obj-y = $(block-obj-y) blockdev.o error.o
+common-obj-y = $(block-obj-y) blockdev.o
common-obj-y += $(net-obj-y)
common-obj-y += $(qobject-obj-y)
common-obj-$(CONFIG_LINUX) += $(fsdev-obj-$(CONFIG_LINUX))
diff --git a/json-parser.c b/json-parser.c
index 6c06ef9..ac4063a 100644
--- a/json-parser.c
+++ b/json-parser.c
@@ -22,9 +22,11 @@
#include "qbool.h"
#include "json-parser.h"
#include "json-lexer.h"
+#include "qerror.h"
typedef struct JSONParserContext
{
+ Error *err;
} JSONParserContext;
#define BUG_ON(cond) assert(!(cond))
@@ -95,11 +97,15 @@ static void GCC_FMT_ATTR(3, 4) parse_error(JSONParserContext *ctxt,
QObject *token, const char *msg, ...)
{
va_list ap;
+ char message[1024];
va_start(ap, msg);
- fprintf(stderr, "parse error: ");
- vfprintf(stderr, msg, ap);
- fprintf(stderr, "\n");
+ vsnprintf(message, sizeof(message), msg, ap);
va_end(ap);
+ if (ctxt->err) {
+ error_free(ctxt->err);
+ ctxt->err = NULL;
+ }
+ error_set(&ctxt->err, QERR_JSON_PARSE_ERROR, message);
}
/**
@@ -565,6 +571,11 @@ static QObject *parse_value(JSONParserContext *ctxt, QList **tokens, va_list *ap
QObject *json_parser_parse(QList *tokens, va_list *ap)
{
+ return json_parser_parse_err(tokens, ap, NULL);
+}
+
+QObject *json_parser_parse_err(QList *tokens, va_list *ap, Error **errp)
+{
JSONParserContext ctxt = {};
QList *working = qlist_copy(tokens);
QObject *result;
@@ -573,5 +584,7 @@ QObject *json_parser_parse(QList *tokens, va_list *ap)
QDECREF(working);
+ error_propagate(errp, ctxt.err);
+
return result;
}
diff --git a/json-parser.h b/json-parser.h
index 97f43f6..8f2b5ec 100644
--- a/json-parser.h
+++ b/json-parser.h
@@ -16,7 +16,9 @@
#include "qemu-common.h"
#include "qlist.h"
+#include "error.h"
QObject *json_parser_parse(QList *tokens, va_list *ap);
+QObject *json_parser_parse_err(QList *tokens, va_list *ap, Error **errp);
#endif
diff --git a/qerror.h b/qerror.h
index 13ad9d4..8b971fd 100644
--- a/qerror.h
+++ b/qerror.h
@@ -121,6 +121,9 @@ QError *qobject_to_qerror(const QObject *obj);
#define QERR_JSON_PARSING \
"{ 'class': 'JSONParsing', 'data': {} }"
+#define QERR_JSON_PARSE_ERROR \
+ "{ 'class': 'JSONParseError', 'data': { 'message': %s } }"
+
#define QERR_KVM_MISSING_CAP \
"{ 'class': 'KVMMissingCap', 'data': { 'capability': %s, 'feature': %s } }"
--
1.7.0.4
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v1][ 05/14] json-streamer: allow recovery after bad input
2011-06-01 17:14 [Qemu-devel] [QAPI+QGA 1/3] Error propagation and JSON parser fix-ups Michael Roth
` (3 preceding siblings ...)
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 04/14] json-parser: propagate error from parser Michael Roth
@ 2011-06-01 17:14 ` Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 06/14] json-lexer: limit the maximum size of a given token Michael Roth
` (8 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Michael Roth @ 2011-06-01 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: aliguori, Jes.Sorensen, agl, mdroth, lcapitulino
From: Anthony Liguori <aliguori@us.ibm.com>
Once we detect a malformed message, make sure to reset our state.
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
json-streamer.c | 8 ++++++--
1 files changed, 6 insertions(+), 2 deletions(-)
diff --git a/json-streamer.c b/json-streamer.c
index f7e7a68..549e9b7 100644
--- a/json-streamer.c
+++ b/json-streamer.c
@@ -51,8 +51,12 @@ static void json_message_process_token(JSONLexer *lexer, QString *token, JSONTok
qlist_append(parser->tokens, dict);
- if (parser->brace_count == 0 &&
- parser->bracket_count == 0) {
+ if (parser->brace_count < 0 ||
+ parser->bracket_count < 0 ||
+ (parser->brace_count == 0 &&
+ parser->bracket_count == 0)) {
+ parser->brace_count = 0;
+ parser->bracket_count = 0;
parser->emit(parser, parser->tokens);
QDECREF(parser->tokens);
parser->tokens = qlist_new();
--
1.7.0.4
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v1][ 06/14] json-lexer: limit the maximum size of a given token
2011-06-01 17:14 [Qemu-devel] [QAPI+QGA 1/3] Error propagation and JSON parser fix-ups Michael Roth
` (4 preceding siblings ...)
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 05/14] json-streamer: allow recovery after bad input Michael Roth
@ 2011-06-01 17:14 ` Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 07/14] json-streamer: limit the maximum recursion depth and maximum token count Michael Roth
` (7 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Michael Roth @ 2011-06-01 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: aliguori, Jes.Sorensen, agl, mdroth, lcapitulino
From: Anthony Liguori <aliguori@us.ibm.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
json-lexer.c | 13 +++++++++++++
1 files changed, 13 insertions(+), 0 deletions(-)
diff --git a/json-lexer.c b/json-lexer.c
index 65c9720..fe5a060 100644
--- a/json-lexer.c
+++ b/json-lexer.c
@@ -18,6 +18,8 @@
#include "qemu-common.h"
#include "json-lexer.h"
+#define MAX_TOKEN_SIZE (64ULL << 20)
+
/*
* \"([^\\\"]|(\\\"\\'\\\\\\/\\b\\f\\n\\r\\t\\u[0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F]))*\"
* '([^\\']|(\\\"\\'\\\\\\/\\b\\f\\n\\r\\t\\u[0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F]))*'
@@ -309,6 +311,17 @@ static int json_lexer_feed_char(JSONLexer *lexer, char ch)
}
lexer->state = new_state;
} while (!char_consumed);
+
+ /* Do not let a single token grow to an arbitrarily large size,
+ * this is a security consideration.
+ */
+ if (lexer->token->length > MAX_TOKEN_SIZE) {
+ lexer->emit(lexer, lexer->token, lexer->state, lexer->x, lexer->y);
+ QDECREF(lexer->token);
+ lexer->token = qstring_new();
+ lexer->state = IN_START;
+ }
+
return 0;
}
--
1.7.0.4
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v1][ 07/14] json-streamer: limit the maximum recursion depth and maximum token count
2011-06-01 17:14 [Qemu-devel] [QAPI+QGA 1/3] Error propagation and JSON parser fix-ups Michael Roth
` (5 preceding siblings ...)
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 06/14] json-lexer: limit the maximum size of a given token Michael Roth
@ 2011-06-01 17:14 ` Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 08/14] json-streamer: make sure to reset token_size after emitting a token list Michael Roth
` (6 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Michael Roth @ 2011-06-01 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: aliguori, Jes.Sorensen, agl, mdroth, lcapitulino
From: Anthony Liguori <aliguori@us.ibm.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
json-streamer.c | 17 +++++++++++++++++
1 files changed, 17 insertions(+), 0 deletions(-)
diff --git a/json-streamer.c b/json-streamer.c
index 549e9b7..6b9af63 100644
--- a/json-streamer.c
+++ b/json-streamer.c
@@ -18,6 +18,9 @@
#include "json-lexer.h"
#include "json-streamer.h"
+#define MAX_TOKEN_SIZE (64ULL << 20)
+#define MAX_NESTING (1ULL << 10)
+
static void json_message_process_token(JSONLexer *lexer, QString *token, JSONTokenType type, int x, int y)
{
JSONMessageParser *parser = container_of(lexer, JSONMessageParser, lexer);
@@ -49,6 +52,8 @@ static void json_message_process_token(JSONLexer *lexer, QString *token, JSONTok
qdict_put(dict, "x", qint_from_int(x));
qdict_put(dict, "y", qint_from_int(y));
+ parser->token_size += token->length;
+
qlist_append(parser->tokens, dict);
if (parser->brace_count < 0 ||
@@ -60,6 +65,17 @@ static void json_message_process_token(JSONLexer *lexer, QString *token, JSONTok
parser->emit(parser, parser->tokens);
QDECREF(parser->tokens);
parser->tokens = qlist_new();
+ } else if (parser->token_size > MAX_TOKEN_SIZE ||
+ parser->bracket_count > MAX_NESTING ||
+ parser->brace_count > MAX_NESTING) {
+ /* Security consideration, we limit total memory allocated per object
+ * and the maximum recursion depth that a message can force.
+ */
+ parser->brace_count = 0;
+ parser->bracket_count = 0;
+ parser->emit(parser, parser->tokens);
+ QDECREF(parser->tokens);
+ parser->tokens = qlist_new();
}
}
@@ -70,6 +86,7 @@ void json_message_parser_init(JSONMessageParser *parser,
parser->brace_count = 0;
parser->bracket_count = 0;
parser->tokens = qlist_new();
+ parser->token_size = 0;
json_lexer_init(&parser->lexer, json_message_process_token);
}
--
1.7.0.4
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v1][ 08/14] json-streamer: make sure to reset token_size after emitting a token list
2011-06-01 17:14 [Qemu-devel] [QAPI+QGA 1/3] Error propagation and JSON parser fix-ups Michael Roth
` (6 preceding siblings ...)
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 07/14] json-streamer: limit the maximum recursion depth and maximum token count Michael Roth
@ 2011-06-01 17:14 ` Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 09/14] json-parser: detect premature EOI Michael Roth
` (5 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Michael Roth @ 2011-06-01 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: aliguori, Jes.Sorensen, agl, mdroth, lcapitulino
From: Anthony Liguori <aliguori@us.ibm.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
json-streamer.c | 2 ++
json-streamer.h | 1 +
2 files changed, 3 insertions(+), 0 deletions(-)
diff --git a/json-streamer.c b/json-streamer.c
index 6b9af63..a6cb28f 100644
--- a/json-streamer.c
+++ b/json-streamer.c
@@ -65,6 +65,7 @@ static void json_message_process_token(JSONLexer *lexer, QString *token, JSONTok
parser->emit(parser, parser->tokens);
QDECREF(parser->tokens);
parser->tokens = qlist_new();
+ parser->token_size = 0;
} else if (parser->token_size > MAX_TOKEN_SIZE ||
parser->bracket_count > MAX_NESTING ||
parser->brace_count > MAX_NESTING) {
@@ -76,6 +77,7 @@ static void json_message_process_token(JSONLexer *lexer, QString *token, JSONTok
parser->emit(parser, parser->tokens);
QDECREF(parser->tokens);
parser->tokens = qlist_new();
+ parser->token_size = 0;
}
}
diff --git a/json-streamer.h b/json-streamer.h
index 09f3bd7..f09bc4d 100644
--- a/json-streamer.h
+++ b/json-streamer.h
@@ -24,6 +24,7 @@ typedef struct JSONMessageParser
int brace_count;
int bracket_count;
QList *tokens;
+ uint64_t token_size;
} JSONMessageParser;
void json_message_parser_init(JSONMessageParser *parser,
--
1.7.0.4
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v1][ 09/14] json-parser: detect premature EOI
2011-06-01 17:14 [Qemu-devel] [QAPI+QGA 1/3] Error propagation and JSON parser fix-ups Michael Roth
` (7 preceding siblings ...)
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 08/14] json-streamer: make sure to reset token_size after emitting a token list Michael Roth
@ 2011-06-01 17:14 ` Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 10/14] json-lexer: reset the lexer state on an invalid token Michael Roth
` (4 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Michael Roth @ 2011-06-01 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: aliguori, Jes.Sorensen, agl, mdroth, lcapitulino
From: Anthony Liguori <aliguori@us.ibm.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
json-parser.c | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 files changed, 57 insertions(+), 1 deletions(-)
diff --git a/json-parser.c b/json-parser.c
index ac4063a..58e973b 100644
--- a/json-parser.c
+++ b/json-parser.c
@@ -275,10 +275,15 @@ out:
*/
static int parse_pair(JSONParserContext *ctxt, QDict *dict, QList **tokens, va_list *ap)
{
- QObject *key, *token = NULL, *value, *peek;
+ QObject *key = NULL, *token = NULL, *value, *peek;
QList *working = qlist_copy(*tokens);
peek = qlist_peek(working);
+ if (peek == NULL) {
+ parse_error(ctxt, NULL, "premature EOI");
+ goto out;
+ }
+
key = parse_value(ctxt, &working, ap);
if (!key || qobject_type(key) != QTYPE_QSTRING) {
parse_error(ctxt, peek, "key is not a string in object");
@@ -286,6 +291,11 @@ static int parse_pair(JSONParserContext *ctxt, QDict *dict, QList **tokens, va_l
}
token = qlist_pop(working);
+ if (token == NULL) {
+ parse_error(ctxt, NULL, "premature EOI");
+ goto out;
+ }
+
if (!token_is_operator(token, ':')) {
parse_error(ctxt, token, "missing : in object pair");
goto out;
@@ -321,6 +331,10 @@ static QObject *parse_object(JSONParserContext *ctxt, QList **tokens, va_list *a
QList *working = qlist_copy(*tokens);
token = qlist_pop(working);
+ if (token == NULL) {
+ goto out;
+ }
+
if (!token_is_operator(token, '{')) {
goto out;
}
@@ -330,12 +344,22 @@ static QObject *parse_object(JSONParserContext *ctxt, QList **tokens, va_list *a
dict = qdict_new();
peek = qlist_peek(working);
+ if (peek == NULL) {
+ parse_error(ctxt, NULL, "premature EOI");
+ goto out;
+ }
+
if (!token_is_operator(peek, '}')) {
if (parse_pair(ctxt, dict, &working, ap) == -1) {
goto out;
}
token = qlist_pop(working);
+ if (token == NULL) {
+ parse_error(ctxt, NULL, "premature EOI");
+ goto out;
+ }
+
while (!token_is_operator(token, '}')) {
if (!token_is_operator(token, ',')) {
parse_error(ctxt, token, "expected separator in dict");
@@ -349,6 +373,10 @@ static QObject *parse_object(JSONParserContext *ctxt, QList **tokens, va_list *a
}
token = qlist_pop(working);
+ if (token == NULL) {
+ parse_error(ctxt, NULL, "premature EOI");
+ goto out;
+ }
}
qobject_decref(token);
token = NULL;
@@ -377,6 +405,10 @@ static QObject *parse_array(JSONParserContext *ctxt, QList **tokens, va_list *ap
QList *working = qlist_copy(*tokens);
token = qlist_pop(working);
+ if (token == NULL) {
+ goto out;
+ }
+
if (!token_is_operator(token, '[')) {
goto out;
}
@@ -386,6 +418,11 @@ static QObject *parse_array(JSONParserContext *ctxt, QList **tokens, va_list *ap
list = qlist_new();
peek = qlist_peek(working);
+ if (peek == NULL) {
+ parse_error(ctxt, NULL, "premature EOI");
+ goto out;
+ }
+
if (!token_is_operator(peek, ']')) {
QObject *obj;
@@ -398,6 +435,11 @@ static QObject *parse_array(JSONParserContext *ctxt, QList **tokens, va_list *ap
qlist_append_obj(list, obj);
token = qlist_pop(working);
+ if (token == NULL) {
+ parse_error(ctxt, NULL, "premature EOI");
+ goto out;
+ }
+
while (!token_is_operator(token, ']')) {
if (!token_is_operator(token, ',')) {
parse_error(ctxt, token, "expected separator in list");
@@ -416,6 +458,10 @@ static QObject *parse_array(JSONParserContext *ctxt, QList **tokens, va_list *ap
qlist_append_obj(list, obj);
token = qlist_pop(working);
+ if (token == NULL) {
+ parse_error(ctxt, NULL, "premature EOI");
+ goto out;
+ }
}
qobject_decref(token);
@@ -444,6 +490,9 @@ static QObject *parse_keyword(JSONParserContext *ctxt, QList **tokens)
QList *working = qlist_copy(*tokens);
token = qlist_pop(working);
+ if (token == NULL) {
+ goto out;
+ }
if (token_get_type(token) != JSON_KEYWORD) {
goto out;
@@ -481,6 +530,9 @@ static QObject *parse_escape(JSONParserContext *ctxt, QList **tokens, va_list *a
}
token = qlist_pop(working);
+ if (token == NULL) {
+ goto out;
+ }
if (token_is_escape(token, "%p")) {
obj = va_arg(*ap, QObject *);
@@ -520,6 +572,10 @@ static QObject *parse_literal(JSONParserContext *ctxt, QList **tokens)
QList *working = qlist_copy(*tokens);
token = qlist_pop(working);
+ if (token == NULL) {
+ goto out;
+ }
+
switch (token_get_type(token)) {
case JSON_STRING:
obj = QOBJECT(qstring_from_escaped_str(ctxt, token));
--
1.7.0.4
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v1][ 10/14] json-lexer: reset the lexer state on an invalid token
2011-06-01 17:14 [Qemu-devel] [QAPI+QGA 1/3] Error propagation and JSON parser fix-ups Michael Roth
` (8 preceding siblings ...)
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 09/14] json-parser: detect premature EOI Michael Roth
@ 2011-06-01 17:14 ` Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 11/14] json-lexer: fix flushing logic to not always go to error state Michael Roth
` (3 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Michael Roth @ 2011-06-01 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: aliguori, Jes.Sorensen, agl, mdroth, lcapitulino
From: Anthony Liguori <aliguori@us.ibm.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
json-lexer.c | 3 +++
1 files changed, 3 insertions(+), 0 deletions(-)
diff --git a/json-lexer.c b/json-lexer.c
index fe5a060..a5bbe9e 100644
--- a/json-lexer.c
+++ b/json-lexer.c
@@ -305,6 +305,9 @@ static int json_lexer_feed_char(JSONLexer *lexer, char ch)
new_state = IN_START;
break;
case IN_ERROR:
+ QDECREF(lexer->token);
+ lexer->token = qstring_new();
+ new_state = IN_START;
return -EINVAL;
default:
break;
--
1.7.0.4
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v1][ 11/14] json-lexer: fix flushing logic to not always go to error state
2011-06-01 17:14 [Qemu-devel] [QAPI+QGA 1/3] Error propagation and JSON parser fix-ups Michael Roth
` (9 preceding siblings ...)
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 10/14] json-lexer: reset the lexer state on an invalid token Michael Roth
@ 2011-06-01 17:14 ` Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 12/14] json-lexer: make lexer error-recovery more deterministic Michael Roth
` (2 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Michael Roth @ 2011-06-01 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: aliguori, Jes.Sorensen, agl, mdroth, lcapitulino
Currently we flush the lexer by passing in a NULL character. This
generally forces the lexer to go to the corresponding TERMINAL() state
for whatever token type it is currently parsing, emits the token to the
parser, then puts the lexer back into IN_START state. However, since a
NULL character causes char_consumed to be 0, we always do a second pass
after this, which puts us in the IN_ERROR state. Fix this behavior by
adding a "flush" flag that tells the lexer not to do a more than 1
iteration.
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
json-lexer.c | 6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/json-lexer.c b/json-lexer.c
index a5bbe9e..6b49047 100644
--- a/json-lexer.c
+++ b/json-lexer.c
@@ -274,7 +274,7 @@ void json_lexer_init(JSONLexer *lexer, JSONLexerEmitter func)
lexer->x = lexer->y = 0;
}
-static int json_lexer_feed_char(JSONLexer *lexer, char ch)
+static int json_lexer_feed_char(JSONLexer *lexer, char ch, bool flush)
{
int char_consumed, new_state;
@@ -313,7 +313,7 @@ static int json_lexer_feed_char(JSONLexer *lexer, char ch)
break;
}
lexer->state = new_state;
- } while (!char_consumed);
+ } while (!char_consumed && !flush);
/* Do not let a single token grow to an arbitrarily large size,
* this is a security consideration.
@@ -335,7 +335,7 @@ int json_lexer_feed(JSONLexer *lexer, const char *buffer, size_t size)
for (i = 0; i < size; i++) {
int err;
- err = json_lexer_feed_char(lexer, buffer[i]);
+ err = json_lexer_feed_char(lexer, buffer[i], false);
if (err < 0) {
return err;
}
--
1.7.0.4
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v1][ 12/14] json-lexer: make lexer error-recovery more deterministic
2011-06-01 17:14 [Qemu-devel] [QAPI+QGA 1/3] Error propagation and JSON parser fix-ups Michael Roth
` (10 preceding siblings ...)
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 11/14] json-lexer: fix flushing logic to not always go to error state Michael Roth
@ 2011-06-01 17:14 ` Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 13/14] json-streamer: add handling for JSON_ERROR token/state Michael Roth
2011-06-01 17:15 ` [Qemu-devel] [PATCH v1][ 14/14] json-parser: add handling for NULL token list Michael Roth
13 siblings, 0 replies; 16+ messages in thread
From: Michael Roth @ 2011-06-01 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: aliguori, Jes.Sorensen, agl, mdroth, lcapitulino
Currently when we reach an error state we effectively flush everything
fed to the lexer, which can put us in a state where we keep feeding
tokens into the parser at arbitrary offsets in the stream. This makes it
difficult for the lexer/tokenizer/parser to get back in sync when bad
input is made by the client.
With these changes we emit an error state/token up to the tokenizer as
soon as we reach an error state, and continue processing any data passed
in rather than bailing out. The reset token will be used to reset the
tokenizer and parser, such that they'll recover state as soon as the
lexer begins generating valid token sequences again.
We also map chr(192,193,245-255) to an error state here, since they are
invalid UTF-8 characters. QMP guest proxy/agent will use chr(255) to
force a flush/reset of previous input for reliable delivery of certain
events, so also we document that thoroughly here.
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
json-lexer.c | 25 +++++++++++++++++++++----
json-lexer.h | 1 +
2 files changed, 22 insertions(+), 4 deletions(-)
diff --git a/json-lexer.c b/json-lexer.c
index 6b49047..c21338f 100644
--- a/json-lexer.c
+++ b/json-lexer.c
@@ -105,7 +105,8 @@ static const uint8_t json_lexer[][256] = {
['u'] = IN_DQ_UCODE0,
},
[IN_DQ_STRING] = {
- [1 ... 0xFF] = IN_DQ_STRING,
+ [1 ... 0xBF] = IN_DQ_STRING,
+ [0xC2 ... 0xF4] = IN_DQ_STRING,
['\\'] = IN_DQ_STRING_ESCAPE,
['"'] = JSON_STRING,
},
@@ -144,7 +145,8 @@ static const uint8_t json_lexer[][256] = {
['u'] = IN_SQ_UCODE0,
},
[IN_SQ_STRING] = {
- [1 ... 0xFF] = IN_SQ_STRING,
+ [1 ... 0xBF] = IN_SQ_STRING,
+ [0xC2 ... 0xF4] = IN_SQ_STRING,
['\\'] = IN_SQ_STRING_ESCAPE,
['\''] = JSON_STRING,
},
@@ -305,10 +307,25 @@ static int json_lexer_feed_char(JSONLexer *lexer, char ch, bool flush)
new_state = IN_START;
break;
case IN_ERROR:
+ /* XXX: To avoid having previous bad input leaving the parser in an
+ * unresponsive state where we consume unpredictable amounts of
+ * subsequent "good" input, percolate this error state up to the
+ * tokenizer/parser by forcing a NULL object to be emitted, then
+ * reset state.
+ *
+ * Also note that this handling is required for reliable channel
+ * negotiation between QMP and the guest agent, since chr(0xFF)
+ * is placed at the beginning of certain events to ensure proper
+ * delivery when the channel is in an unknown state. chr(0xFF) is
+ * never a valid ASCII/UTF-8 sequence, so this should reliably
+ * induce an error/flush state.
+ */
+ lexer->emit(lexer, lexer->token, JSON_ERROR, lexer->x, lexer->y);
QDECREF(lexer->token);
lexer->token = qstring_new();
new_state = IN_START;
- return -EINVAL;
+ lexer->state = new_state;
+ return 0;
default:
break;
}
@@ -346,7 +363,7 @@ int json_lexer_feed(JSONLexer *lexer, const char *buffer, size_t size)
int json_lexer_flush(JSONLexer *lexer)
{
- return lexer->state == IN_START ? 0 : json_lexer_feed_char(lexer, 0);
+ return lexer->state == IN_START ? 0 : json_lexer_feed_char(lexer, 0, true);
}
void json_lexer_destroy(JSONLexer *lexer)
diff --git a/json-lexer.h b/json-lexer.h
index 3b50c46..10bc0a7 100644
--- a/json-lexer.h
+++ b/json-lexer.h
@@ -25,6 +25,7 @@ typedef enum json_token_type {
JSON_STRING,
JSON_ESCAPE,
JSON_SKIP,
+ JSON_ERROR,
} JSONTokenType;
typedef struct JSONLexer JSONLexer;
--
1.7.0.4
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v1][ 13/14] json-streamer: add handling for JSON_ERROR token/state
2011-06-01 17:14 [Qemu-devel] [QAPI+QGA 1/3] Error propagation and JSON parser fix-ups Michael Roth
` (11 preceding siblings ...)
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 12/14] json-lexer: make lexer error-recovery more deterministic Michael Roth
@ 2011-06-01 17:14 ` Michael Roth
2011-06-01 17:15 ` [Qemu-devel] [PATCH v1][ 14/14] json-parser: add handling for NULL token list Michael Roth
13 siblings, 0 replies; 16+ messages in thread
From: Michael Roth @ 2011-06-01 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: aliguori, Jes.Sorensen, agl, mdroth, lcapitulino
This allows a JSON_ERROR state to be passed to the streamer to force a
flush of the current tokens and pass a NULL token list to the parser
rather that have it churn on bad data. (Alternatively we could just not
pass it to the parser at all, but it may be useful to push there errors
up the stack. NULL token lists are not currently handled by the parser,
the next patch will address that)
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
json-streamer.c | 35 +++++++++++++++++++++++------------
1 files changed, 23 insertions(+), 12 deletions(-)
diff --git a/json-streamer.c b/json-streamer.c
index a6cb28f..c255c78 100644
--- a/json-streamer.c
+++ b/json-streamer.c
@@ -56,29 +56,40 @@ static void json_message_process_token(JSONLexer *lexer, QString *token, JSONTok
qlist_append(parser->tokens, dict);
- if (parser->brace_count < 0 ||
+ if (type == JSON_ERROR) {
+ goto out_emit_bad;
+ } else if (parser->brace_count < 0 ||
parser->bracket_count < 0 ||
(parser->brace_count == 0 &&
parser->bracket_count == 0)) {
- parser->brace_count = 0;
- parser->bracket_count = 0;
- parser->emit(parser, parser->tokens);
- QDECREF(parser->tokens);
- parser->tokens = qlist_new();
- parser->token_size = 0;
+ goto out_emit;
} else if (parser->token_size > MAX_TOKEN_SIZE ||
parser->bracket_count > MAX_NESTING ||
parser->brace_count > MAX_NESTING) {
/* Security consideration, we limit total memory allocated per object
* and the maximum recursion depth that a message can force.
*/
- parser->brace_count = 0;
- parser->bracket_count = 0;
- parser->emit(parser, parser->tokens);
+ goto out_emit;
+ }
+
+ return;
+
+out_emit_bad:
+ /* clear out token list and tell the parser to emit and error
+ * indication by passing it a NULL list
+ */
+ QDECREF(parser->tokens);
+ parser->tokens = NULL;
+out_emit:
+ /* send current list of tokens to parser and reset tokenizer */
+ parser->brace_count = 0;
+ parser->bracket_count = 0;
+ parser->emit(parser, parser->tokens);
+ if (parser->tokens) {
QDECREF(parser->tokens);
- parser->tokens = qlist_new();
- parser->token_size = 0;
}
+ parser->tokens = qlist_new();
+ parser->token_size = 0;
}
void json_message_parser_init(JSONMessageParser *parser,
--
1.7.0.4
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v1][ 14/14] json-parser: add handling for NULL token list
2011-06-01 17:14 [Qemu-devel] [QAPI+QGA 1/3] Error propagation and JSON parser fix-ups Michael Roth
` (12 preceding siblings ...)
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 13/14] json-streamer: add handling for JSON_ERROR token/state Michael Roth
@ 2011-06-01 17:15 ` Michael Roth
13 siblings, 0 replies; 16+ messages in thread
From: Michael Roth @ 2011-06-01 17:15 UTC (permalink / raw)
To: qemu-devel; +Cc: aliguori, Jes.Sorensen, agl, mdroth, lcapitulino
Currently a NULL token list will crash the parser, instead we have it
pass back a NULL QObject.
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
json-parser.c | 6 +++++-
1 files changed, 5 insertions(+), 1 deletions(-)
diff --git a/json-parser.c b/json-parser.c
index 58e973b..849e215 100644
--- a/json-parser.c
+++ b/json-parser.c
@@ -633,9 +633,13 @@ QObject *json_parser_parse(QList *tokens, va_list *ap)
QObject *json_parser_parse_err(QList *tokens, va_list *ap, Error **errp)
{
JSONParserContext ctxt = {};
- QList *working = qlist_copy(tokens);
+ QList *working;
QObject *result;
+ if (!tokens) {
+ return NULL;
+ }
+ working = qlist_copy(tokens);
result = parse_value(&ctxt, &working, ap);
QDECREF(working);
--
1.7.0.4
^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [Qemu-devel] [PATCH v1][ 01/14] QError: Introduce qerror_format_desc()
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 01/14] QError: Introduce qerror_format_desc() Michael Roth
@ 2011-06-07 19:18 ` Anthony Liguori
0 siblings, 0 replies; 16+ messages in thread
From: Anthony Liguori @ 2011-06-07 19:18 UTC (permalink / raw)
To: Michael Roth; +Cc: aliguori, lcapitulino, agl, qemu-devel, Jes.Sorensen
On 06/01/2011 12:14 PM, Michael Roth wrote:
> From: Luiz Capitulino<lcapitulino@redhat.com>
>
> Refactor non-QError-specific bits out of qerror_human() into general
> function that can be used by the error_get_pretty() analogue in the
> new error-propagation framework.
>
> Signed-off-by: Luiz Capitulino<lcapitulino@redhat.com>
> Signed-off-by: Michael Roth<mdroth@linux.vnet.ibm.com>
Applied series1. Thanks.
Regards,
Anthony Liguori
> ---
> qerror.c | 44 +++++++++++++++++++++++++-------------------
> 1 files changed, 25 insertions(+), 19 deletions(-)
>
> diff --git a/qerror.c b/qerror.c
> index 4855604..af6ed39 100644
> --- a/qerror.c
> +++ b/qerror.c
> @@ -326,12 +326,14 @@ QError *qerror_from_info(const char *file, int linenr, const char *func,
> return qerr;
> }
>
> -static void parse_error(const QError *qerror, int c)
> +static void parse_error(const QErrorStringTable *entry, int c)
> {
> - qerror_abort(qerror, "expected '%c' in '%s'", c, qerror->entry->desc);
> + fprintf(stderr, "expected '%c' in '%s'", c, entry->desc);
> + abort();
> }
>
> -static const char *append_field(QString *outstr, const QError *qerror,
> +static const char *append_field(QDict *error, QString *outstr,
> + const QErrorStringTable *entry,
> const char *start)
> {
> QObject *obj;
> @@ -340,23 +342,23 @@ static const char *append_field(QString *outstr, const QError *qerror,
> const char *end, *key;
>
> if (*start != '%')
> - parse_error(qerror, '%');
> + parse_error(entry, '%');
> start++;
> if (*start != '(')
> - parse_error(qerror, '(');
> + parse_error(entry, '(');
> start++;
>
> end = strchr(start, ')');
> if (!end)
> - parse_error(qerror, ')');
> + parse_error(entry, ')');
>
> key_qs = qstring_from_substr(start, 0, end - start - 1);
> key = qstring_get_str(key_qs);
>
> - qdict = qobject_to_qdict(qdict_get(qerror->error, "data"));
> + qdict = qobject_to_qdict(qdict_get(error, "data"));
> obj = qdict_get(qdict, key);
> if (!obj) {
> - qerror_abort(qerror, "key '%s' not found in QDict", key);
> + abort();
> }
>
> switch (qobject_type(obj)) {
> @@ -367,35 +369,31 @@ static const char *append_field(QString *outstr, const QError *qerror,
> qstring_append_int(outstr, qdict_get_int(qdict, key));
> break;
> default:
> - qerror_abort(qerror, "invalid type '%c'", qobject_type(obj));
> + abort();
> }
>
> QDECREF(key_qs);
> return ++end;
> }
>
> -/**
> - * qerror_human(): Format QError data into human-readable string.
> - *
> - * Formats according to member 'desc' of the specified QError object.
> - */
> -QString *qerror_human(const QError *qerror)
> +static QString *qerror_format_desc(QDict *error,
> + const QErrorStringTable *entry)
> {
> - const char *p;
> QString *qstring;
> + const char *p;
>
> - assert(qerror->entry != NULL);
> + assert(entry != NULL);
>
> qstring = qstring_new();
>
> - for (p = qerror->entry->desc; *p != '\0';) {
> + for (p = entry->desc; *p != '\0';) {
> if (*p != '%') {
> qstring_append_chr(qstring, *p++);
> } else if (*(p + 1) == '%') {
> qstring_append_chr(qstring, '%');
> p += 2;
> } else {
> - p = append_field(qstring, qerror, p);
> + p = append_field(error, qstring, entry, p);
> }
> }
>
> @@ -403,6 +401,14 @@ QString *qerror_human(const QError *qerror)
> }
>
> /**
> + * qerror_human(): Format QError data into human-readable string.
> + */
> +QString *qerror_human(const QError *qerror)
> +{
> + return qerror_format_desc(qerror->error, qerror->entry);
> +}
> +
> +/**
> * qerror_print(): Print QError data
> *
> * This function will print the member 'desc' of the specified QError object,
^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2011-06-07 19:18 UTC | newest]
Thread overview: 16+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-06-01 17:14 [Qemu-devel] [QAPI+QGA 1/3] Error propagation and JSON parser fix-ups Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 01/14] QError: Introduce qerror_format_desc() Michael Roth
2011-06-07 19:18 ` Anthony Liguori
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 02/14] QError: Introduce qerror_format() Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 03/14] Introduce the new error framework Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 04/14] json-parser: propagate error from parser Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 05/14] json-streamer: allow recovery after bad input Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 06/14] json-lexer: limit the maximum size of a given token Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 07/14] json-streamer: limit the maximum recursion depth and maximum token count Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 08/14] json-streamer: make sure to reset token_size after emitting a token list Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 09/14] json-parser: detect premature EOI Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 10/14] json-lexer: reset the lexer state on an invalid token Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 11/14] json-lexer: fix flushing logic to not always go to error state Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 12/14] json-lexer: make lexer error-recovery more deterministic Michael Roth
2011-06-01 17:14 ` [Qemu-devel] [PATCH v1][ 13/14] json-streamer: add handling for JSON_ERROR token/state Michael Roth
2011-06-01 17:15 ` [Qemu-devel] [PATCH v1][ 14/14] json-parser: add handling for NULL token list Michael Roth
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).