From: Markus Armbruster <armbru@redhat.com>
To: qemu-devel@nongnu.org
Subject: [Qemu-devel] [PULL 25/58] json: Accept overlong \xC0\x80 as U+0000 ("modified UTF-8")
Date: Fri, 24 Aug 2018 21:31:33 +0200 [thread overview]
Message-ID: <20180824193206.25475-26-armbru@redhat.com> (raw)
In-Reply-To: <20180824193206.25475-1-armbru@redhat.com>
Since the JSON grammer doesn't accept U+0000 anywhere, this merely
exchanges one kind of parse error for another. It's purely for
consistency with qobject_to_json(), which accepts \xC0\x80 (see commit
e2ec3f97680).
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-26-armbru@redhat.com>
---
qobject/json-lexer.c | 2 +-
qobject/json-parser.c | 2 +-
tests/check-qjson.c | 8 +-------
3 files changed, 3 insertions(+), 9 deletions(-)
diff --git a/qobject/json-lexer.c b/qobject/json-lexer.c
index 93fa2737e6..4c402f62d3 100644
--- a/qobject/json-lexer.c
+++ b/qobject/json-lexer.c
@@ -93,7 +93,7 @@
* interpolation = %((l|ll|I64)[du]|[ipsf])
*
* Note:
- * - Input must be encoded in UTF-8.
+ * - Input must be encoded in modified UTF-8.
* - Decoding and validating is left to the parser.
*/
diff --git a/qobject/json-parser.c b/qobject/json-parser.c
index b77931614b..a9b227f56c 100644
--- a/qobject/json-parser.c
+++ b/qobject/json-parser.c
@@ -200,7 +200,7 @@ static QString *qstring_from_escaped_str(JSONParserContext *ctxt,
}
} else {
cp = mod_utf8_codepoint(ptr, 6, &end);
- if (cp <= 0) {
+ if (cp < 0) {
parse_error(ctxt, token, "invalid UTF-8 sequence in string");
goto out;
}
diff --git a/tests/check-qjson.c b/tests/check-qjson.c
index 71c77d2f70..3abf12b4d2 100644
--- a/tests/check-qjson.c
+++ b/tests/check-qjson.c
@@ -152,12 +152,6 @@ static void string_with_quotes(void)
static void utf8_string(void)
{
/*
- * Problem: we can't easily deal with embedded U+0000. Parsing
- * the JSON string "this \\u0000" is fun" yields "this \0 is fun",
- * which gets misinterpreted as NUL-terminated "this ". We should
- * consider using overlong encoding \xC0\x80 for U+0000 ("modified
- * UTF-8").
- *
* Most test cases are scraped from Markus Kuhn's UTF-8 decoder
* capability and stress test at
* http://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-test.txt
@@ -586,7 +580,7 @@ static void utf8_string(void)
{
/* \U+0000 */
"\xC0\x80",
- NULL,
+ "\xC0\x80",
"\\u0000",
},
{
--
2.17.1
next prev parent reply other threads:[~2018-08-24 19:32 UTC|newest]
Thread overview: 60+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-08-24 19:31 [Qemu-devel] [PULL 00/58] QObject patches for 2018-08-24 Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 01/58] docs/interop/qmp-spec: How to force known good parser state Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 02/58] check-qjson: Cover multiple JSON objects in same string Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 03/58] check-qjson: Cover blank and lexically erroneous input Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 04/58] check-qjson: Cover whitespace more thoroughly Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 05/58] qmp-cmd-test: Split off qmp-test Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 06/58] qmp-test: Cover syntax and lexical errors Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 07/58] test-qga: Clean up how we test QGA synchronization Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 08/58] check-qjson: Cover escaped characters more thoroughly, part 1 Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 09/58] check-qjson: Streamline escaped_string()'s test strings Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 10/58] check-qjson: Cover escaped characters more thoroughly, part 2 Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 11/58] check-qjson: Consolidate partly redundant string tests Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 12/58] check-qjson: Cover UTF-8 in single quoted strings Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 13/58] check-qjson: Simplify utf8_string() Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 14/58] check-qjson: Fix utf8_string() to test all invalid sequences Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 15/58] check-qjson qmp-test: Cover control characters more thoroughly Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 16/58] check-qjson: Cover interpolation " Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 17/58] json: Fix lexer to include the bad character in JSON_ERROR token Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 18/58] json: Reject unescaped control characters Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 19/58] json: Revamp lexer documentation Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 20/58] json: Tighten and simplify qstring_from_escaped_str()'s loop Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 21/58] check-qjson: Document we expect invalid UTF-8 to be rejected Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 22/58] json: Reject invalid UTF-8 sequences Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 23/58] json: Report first rather than last parse error Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 24/58] json: Leave rejecting invalid UTF-8 to parser Markus Armbruster
2018-08-24 19:31 ` Markus Armbruster [this message]
2018-08-24 19:31 ` [Qemu-devel] [PULL 26/58] json: Leave rejecting invalid escape sequences " Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 27/58] json: Simplify parse_string() Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 28/58] json: Reject invalid \uXXXX, fix \u0000 Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 29/58] json: Fix \uXXXX for surrogate pairs Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 30/58] check-qjson: Fix and enable utf8_string()'s disabled part Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 31/58] json: remove useless return value from lexer/parser Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 32/58] json-parser: simplify and avoid JSONParserContext allocation Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 33/58] json: Have lexer call streamer directly Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 34/58] json: Redesign the callback to consume JSON values Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 35/58] json: Don't pass null @tokens to json_parser_parse() Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 36/58] json: Don't create JSON_ERROR tokens that won't be used Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 37/58] json: Rename token JSON_ESCAPE & friends to JSON_INTERP Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 38/58] json: Treat unwanted interpolation as lexical error Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 39/58] json: Pass lexical errors and limit violations to callback Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 40/58] json: Leave rejecting invalid interpolation to parser Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 41/58] json: Replace %I64d, %I64u by %PRId64, %PRIu64 Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 42/58] json: Improve names of lexer states related to numbers Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 43/58] qjson: Fix qobject_from_json() & friends for multiple values Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 44/58] json: Fix latent parser aborts at end of input Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 45/58] json: Fix streamer not to ignore trailing unterminated structures Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 46/58] json: Assert json_parser_parse() consumes all tokens on success Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 47/58] qjson: Have qobject_from_json() & friends reject empty and blank Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 48/58] json: Enforce token count and size limits more tightly Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 49/58] json: Streamline json_message_process_token() Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 50/58] json: Unbox tokens queue in JSONMessageParser Markus Armbruster
2018-08-24 19:31 ` [Qemu-devel] [PULL 51/58] json: Make JSONToken opaque outside json-parser.c Markus Armbruster
2018-08-24 19:32 ` [Qemu-devel] [PULL 52/58] qobject: Drop superfluous includes of qemu-common.h Markus Armbruster
2018-08-24 19:32 ` [Qemu-devel] [PULL 53/58] json: Clean up headers Markus Armbruster
2018-08-24 19:32 ` [Qemu-devel] [PULL 54/58] tests/drive_del-test: Fix harmless JSON interpolation bug Markus Armbruster
2018-08-24 19:32 ` [Qemu-devel] [PULL 55/58] json: Keep interpolation state in JSONParserContext Markus Armbruster
2018-08-24 19:32 ` [Qemu-devel] [PULL 56/58] json: Improve safety of qobject_from_jsonf_nofail() & friends Markus Armbruster
2018-08-24 19:32 ` [Qemu-devel] [PULL 57/58] json: Support %% in JSON strings when interpolating Markus Armbruster
2018-08-24 19:32 ` [Qemu-devel] [PULL 58/58] json: Update references to RFC 7159 to RFC 8259 Markus Armbruster
2018-08-25 9:58 ` [Qemu-devel] [PULL 00/58] QObject patches for 2018-08-24 Peter Maydell
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180824193206.25475-26-armbru@redhat.com \
--to=armbru@redhat.com \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).