Re: [Qemu-devel] [PATCH 01/11] QMP: Introduce specification file

[Filip Navara <filip.navara@gmail.com>]

On Wed, Jun 24, 2009 at 2:46 PM, Anthony Liguori<anthony@codemonkey.ws> wrote:
> Vincent Hanquez wrote:
>>
>> On Tue, Jun 23, 2009 at 05:50:24PM -0500, Anthony Liguori wrote:
>>
>>>>
>>>> this one is mentioned on the website and has been changed recently (may
>>>> 2009):
>>>>
>>>> http://fara.cs.uni-potsdam.de/~jsg/json_parser/
>>>>
>>>
>>> That's not a jsonrpc library, that's just a json parser.
>>>
>>
>> but this is completly trivial at this point. jsonrpc is a json message put
>> in a
>> certain way.
>>
>> now providing there was a maintained library out there, would you use it ?
>> or do you have more concerns that were unanswered ?
>
> There are two questions to resolve.  The first is whether we should continue
> with the current direction (line-based protocol) or whether we should switch
> to an RPC.  The second question is which RPC we should use.

I've spent the last 5 years working on different projects that
involved text/line-based protocols. In fact my current day job
involves working on a program that works with wide variety of such
protocols (POP3, SMTP, IMAP, ACAP, HTTP to name a few) and based on my
experience I would seriously recommend using an existing RPC protocol
instead. While it's simple to design a synchronous protocol like the
original POP3 it gets a lot harder with asynchronous protocols like
IMAP. Even if you decide for a custom line-level protocol I'd suggest
to learn from the past mistakes, more details below.

The current draft specification used a mix of semantics of the
forementioned protocols - error handling based on POP3 (+OK/-ERR),
asynchronous events based on IMAP (* prefix). IMAP-like tagging of
commands was also suggested (to deal with asynchronous commands) and
issues have been raised concerning multi-line responses. Note that all
this gets a bit hairy when you combine it. Just a few issues that come
to mind:

* An event happens while a multi-line response is currently written to
the stream. The behavior has to be defined somehow - either by
specifying that the multi-line response mustn't be interrupted by
asynchronous events or that it can be interrupted, but only on line
breaks. This issue don't arise with message-based protocols.

* Should the asynchronous replies (* EVENT) be tagged (ie. C: "<tag1>
reboot", S: "* <tag1> reset") when they result from a command issued
by the manager? If they are not tagged then it's impossible to find
out what (if any) of the commands caused the event when multiple
asynchronous commands are in progress. That's what basic IMAP does (as
specified by RFC 3501). What IMAP did wrong [1] is that even the basic
responses to commands like SEARCH are returned as untagged
asynchronous notifications, which makes the client implementation
hard. Several extensions were later developed to mitigate it and the
protocol is now a messy combination of both tagged and untagged
responses with no universal syntax. On the other hand, if you tag the
events then it's hard for the server (in this case QEMU) to keep track
of it and a special syntax still has to be present for truly
asynchronous events.

* It is necessary to separate the syntax of the data protocol from the
semantics of the commands itself. The syntax should be generic enough
to support transferring all the necessary information (integers,
strings, lists) and simple enough to be parsable even if you receive
the data in chunks (assuming a text-based protocol). This is where
IMAP went really wrong, since parsing the server responses requires
knowning the semantics of ALL the commands.

* Many of you asked for facility to negotiate capabilities. I'd
suggest the following - the server would lists capabilities by name as
a list on connect and the client would be able to specifically enable
those that change the semantics of commands. Example:

S: + OK QEMU 0.10.50 QMP 0.1 (pci-addr, x-some-extension)
C: enable pci-addr
S: + OK pci-addr extension enabled

The "enable" command would optionally allow for a list to be
specified, to enable more extensions at once.

Best regards,
Filip Navara

[1] ...according to large amount of people with the exception of Mark Crispin.