Re: [Qemu-devel] [PATCH 01/11] QMP: Introduce specification file

[Jamie Lokier <jamie@shareable.org>]

> On Wed, Jun 24, 2009 at 7:39 PM, Vincent Hanquez<tab@snarc.org> wrote:
> > On Wed, Jun 24, 2009 at 05:22:07PM +0100, Jamie Lokier wrote:
> >> You can code a minimal XML parser in straight C quite easily, if it's
> >> a restricted subset.
> >
> > even the restricted subset is not as straighforward as a json parser. and
> > usually using a subset means you can't interact correctly with the one that
> > does the full spec.
> >
> >> XML and JSON both have the same ugly problem with binary data: they
> >> can't carry it.  It's usually base64 encoded.  Then again the QEMU
> >> monitor is no better this respect :-)
> >
> > JSon ***DOES*** do binary data.
> >
> > C String "abc\0\xff" -> Json String "abc\0000\00ff"

Any reason you can't simply put the UTF-8 encoding of U+0000 and
U+00FF directly in the string?

(Let's ignore Java, which encodes U+0000 in "Java-UTF-8" differently
from everyone else's UTF-8!)

Filip Navara wrote:
> I find the Json representation problematic. In C you have two distinct
> data types - null-terminated string where the length is implicitly
> known from the content (char *string) and a binary data blob (char
> *buffer, int size). If you encode them into the same JSON data type
> and don't supply "out-of-band" information about which one of the C
> types is it, the receiver has no way to decide what to decode it into.
> JSONRPC allows supplying this "out-of-band" information only for the
> JSON data types which is very limiting.
> 
> For text based protocols it's vital to separate the syntax from
> semantics and decoding the above would require knowning the specific
> context and semantics.
> 
> A more natural representation of binary blob in JSON would be array of
> numbers, but that would have a big overhead.

Actually, an array doesn't add much more overhead :-)

Binary has a big overhead in JSON as a string.

Encode the blob [0,1,2,3,252,253,254,255]:

    JSON: "\0\1\2\3\xfc\xfd\xfe\xff"

That's fine if you've just got a few non-ASCII bytes.  But for general
binary, base64 is much more compact.

Base64 expands by about 4/3, whereas a JSON \x-encoded string expands
by up to 4 times.  Even a hex string is more compact :-)

A further complication is that a JSON string carries Unicode (which is
good for text), so at some point you have to know to signal a parse
error when any characters are outside the range 0-255.

But I don't know why we're talking about binary, as the monitor (old
or proposed) doesn't handle binary either, and no particular need for
binary has come up...

-- Jamie