[Qemu-devel] [PATCH] QMP: Spec: Private Extensions support

[Qemu-devel] [PATCH] QMP: Spec: Private Extensions support

[Luiz Capitulino]

Vendors might want to add their own extensions to QMP, as JSON itself
(and several other protocols) allow this someway, I think QMP should
allow too.

We just have to choose a naming convention that is guaranteed not to
clash with any future new commands, arguments, parameters and event
names.

Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
---
 QMP/qmp-spec.txt |   23 +++++++++++++++++++++++
 1 files changed, 23 insertions(+), 0 deletions(-)

diff --git a/QMP/qmp-spec.txt b/QMP/qmp-spec.txt
index f3c0327..bc92c7e 100644
--- a/QMP/qmp-spec.txt
+++ b/QMP/qmp-spec.txt
@@ -215,3 +215,26 @@ Additionally, Clients must not assume any particular:
 - Order of json-object members or json-array elements
 - Amount of errors generated by a command, that is, new errors can be added
   to any existing command in newer versions of the Server
+
+6 Private Extensions 
+--------------------
+
+QMP provides a special naming convention to allow the creation of independent
+namespaces, which allows vendors to introduce private extensions to the
+protocol. It is guaranteed that no future QMP version will expose any name
+that follows this convention.
+
+Private extensions must be in the following format:
+
+v_NAMESPACE__NAME
+
+ Where,
+
+- NAME is any argument, command, event or parameter name
+- NAMESPACE is the namespace that NAME belongs to
+
+For example, the following command:
+
+v_ABC__insert
+
+Is called 'insert' and is part of the 'ABC' namespace.
-- 
1.6.6

Re: [Qemu-devel] [PATCH] QMP: Spec: Private Extensions support

[Anthony Liguori]

On 02/18/2010 02:24 PM, Luiz Capitulino wrote:
> Vendors might want to add their own extensions to QMP, as JSON itself
> (and several other protocols) allow this someway, I think QMP should
> allow too.
>
> We just have to choose a naming convention that is guaranteed not to
> clash with any future new commands, arguments, parameters and event
> names.
>
> Signed-off-by: Luiz Capitulino<lcapitulino@redhat.com>
> ---
>   QMP/qmp-spec.txt |   23 +++++++++++++++++++++++
>   1 files changed, 23 insertions(+), 0 deletions(-)
>
> diff --git a/QMP/qmp-spec.txt b/QMP/qmp-spec.txt
> index f3c0327..bc92c7e 100644
> --- a/QMP/qmp-spec.txt
> +++ b/QMP/qmp-spec.txt
> @@ -215,3 +215,26 @@ Additionally, Clients must not assume any particular:
>   - Order of json-object members or json-array elements
>   - Amount of errors generated by a command, that is, new errors can be added
>     to any existing command in newer versions of the Server
> +
> +6 Private Extensions
> +--------------------
> +
> +QMP provides a special naming convention to allow the creation of independent
> +namespaces, which allows vendors to introduce private extensions to the
> +protocol. It is guaranteed that no future QMP version will expose any name
> +that follows this convention.
> +
> +Private extensions must be in the following format:
> +
> +v_NAMESPACE__NAME
> +
> + Where,
> +
> +- NAME is any argument, command, event or parameter name
> +- NAMESPACE is the namespace that NAME belongs to
> +
> +For example, the following command:
> +
> +v_ABC__insert
> +
> +Is called 'insert' and is part of the 'ABC' namespace.
>    

We need a bit more than just this.  Here's my suggestion:

6. Downstream modification of QMP
-----------------------------------------------------------

We strongly recommend that downstream consumers of QEMU do _not_
modify the behaviour of any QMP command or introduce new QMP commands.
This is important to allow management tools to support both upstream and
downstream versions of QMP without special logic.

However, we recognize that it is sometimes impossible for downstreams to
avoid modifying QMP.  If this happens, the following rules should be 
observed
to attempt to preserve long term compatibility and interoperability.

1) Only introduce new commands.  If you want to change an existing command,
leave the old command in place and introduce a new one with the new 
behaviour.

2) Only introduce new asynchronous messages.  Do not change an existing 
message.

3) Only introduce new error types and only use those error types in new 
commands.
New commands can use existing error types, but you should never add a 
new error type
to an existing command.

4) Do not introduce any new capabilities.  Capabilities directly affect 
a client's ability to
parse the protocol correctly and new capabilities can not be supported 
in an upstream
compatible way.  Please work capabilities through upstream first.

5) QMP names will never begin with '__' (double underscore).  When 
introducing new
commands, asynchronous events, or error messages, a downstream must 
prefix those
names with a '__' to ensure future compatibility with upstream.

6) To ensure compatibility with other downstreams, it is strongly 
recommended that
you prefix the commands with '__RFQDN_' where RFQDN is a valid, reverse 
fully
qualified domain name which you control.  For example, a qemu-kvm 
specific monitor
command would be:

  (qemu) __org.linux-kvm_enable_irqchip

7) Do not change the QMP banner.  QMP supports introspection which will 
allow a
management tool to differentiate between a downstream QMP session and an
upstream QMP session.

Regards,

Anthony Liguori

Re: [Qemu-devel] [PATCH] QMP: Spec: Private Extensions support

[Luiz Capitulino]

On Thu, 18 Feb 2010 15:54:04 -0600
Anthony Liguori <anthony@codemonkey.ws> wrote:

> On 02/18/2010 02:24 PM, Luiz Capitulino wrote:
> > Vendors might want to add their own extensions to QMP, as JSON itself
> > (and several other protocols) allow this someway, I think QMP should
> > allow too.
> >
> > We just have to choose a naming convention that is guaranteed not to
> > clash with any future new commands, arguments, parameters and event
> > names.
> >
> > Signed-off-by: Luiz Capitulino<lcapitulino@redhat.com>
> > ---
> >   QMP/qmp-spec.txt |   23 +++++++++++++++++++++++
> >   1 files changed, 23 insertions(+), 0 deletions(-)
> >
> > diff --git a/QMP/qmp-spec.txt b/QMP/qmp-spec.txt
> > index f3c0327..bc92c7e 100644
> > --- a/QMP/qmp-spec.txt
> > +++ b/QMP/qmp-spec.txt
> > @@ -215,3 +215,26 @@ Additionally, Clients must not assume any particular:
> >   - Order of json-object members or json-array elements
> >   - Amount of errors generated by a command, that is, new errors can be added
> >     to any existing command in newer versions of the Server
> > +
> > +6 Private Extensions
> > +--------------------
> > +
> > +QMP provides a special naming convention to allow the creation of independent
> > +namespaces, which allows vendors to introduce private extensions to the
> > +protocol. It is guaranteed that no future QMP version will expose any name
> > +that follows this convention.
> > +
> > +Private extensions must be in the following format:
> > +
> > +v_NAMESPACE__NAME
> > +
> > + Where,
> > +
> > +- NAME is any argument, command, event or parameter name
> > +- NAMESPACE is the namespace that NAME belongs to
> > +
> > +For example, the following command:
> > +
> > +v_ABC__insert
> > +
> > +Is called 'insert' and is part of the 'ABC' namespace.
> >    
> 
> We need a bit more than just this.  Here's my suggestion:

 I like it very much, feel free to apply.

Re: [Qemu-devel] [PATCH] QMP: Spec: Private Extensions support

[Markus Armbruster]

Anthony Liguori <anthony@codemonkey.ws> writes:

> We need a bit more than just this.  Here's my suggestion:

I think this is much more restrictive than necessary.  Unnecessarily
restrictive rules are more likely to be ignored, and we don't want that.
Details below.

> 6. Downstream modification of QMP
> -----------------------------------------------------------
>
> We strongly recommend that downstream consumers of QEMU do _not_
> modify the behaviour of any QMP command or introduce new QMP commands.
> This is important to allow management tools to support both upstream and
> downstream versions of QMP without special logic.
>
> However, we recognize that it is sometimes impossible for downstreams to
> avoid modifying QMP.  If this happens, the following rules should be
> observed

Bad line break.

> to attempt to preserve long term compatibility and interoperability.

Better move your items 5-6) here, and make them talk about "names", not
just commands.  Items 1-4) make much more sense with the name space
issue settled.

> 1) Only introduce new commands.  If you want to change an existing command,
> leave the old command in place and introduce a new one with the new
> behaviour.

Extending an existing command in a compatible way should be fine.  An
extension is "compatible" if

1. it adds only optional arguments, and

2. it behaves differently only when new optional arguments are supplied.

> 2) Only introduce new asynchronous messages.  Do not change an
> existing message.

Why outlaw adding new data members?  What would that break?

> 3) Only introduce new error types and only use those error types in
> new commands.
> New commands can use existing error types, but you should never add a
> new error type
> to an existing command.

A client that can't cope with unknown error types will only ever work on
a single upstream version.  Such clients will go the way of the Dodo
real quick.

With that established: why is adding new error types bad?

> 4) Do not introduce any new capabilities.  Capabilities directly
> affect a client's ability to
> parse the protocol correctly and new capabilities can not be supported
> in an upstream
> compatible way.  Please work capabilities through upstream first.

This is factually incorrect.  Clients that don't recognize new
capabilities just ignore them, and continue to work as before.

The *point* of having capabilities is to permit protocol evolution
without breaking clients.  Why reserve evolution privileges to upstream?
Yes, downstream should probably think twice before mucking with the
basic protocol.  But we should not attempt to stop downstreams from
doing stupid things, as that would also stop them from doing clever
things.

> 5) QMP names will never begin with '__' (double underscore).  When
> introducing new
> commands, asynchronous events, or error messages, a downstream must
> prefix those
> names with a '__' to ensure future compatibility with upstream.
>
> 6) To ensure compatibility with other downstreams, it is strongly
> recommended that
> you prefix the commands with '__RFQDN_' where RFQDN is a valid,
> reverse fully
> qualified domain name which you control.  For example, a qemu-kvm
> specific monitor
> command would be:
>
>  (qemu) __org.linux-kvm_enable_irqchip
>
> 7) Do not change the QMP banner.  QMP supports introspection which
> will allow a
> management tool to differentiate between a downstream QMP session and an
> upstream QMP session.

Re: [Qemu-devel] [PATCH] QMP: Spec: Private Extensions support

[Anthony Liguori]

On 02/19/2010 07:04 AM, Markus Armbruster wrote:
> Anthony Liguori<anthony@codemonkey.ws>  writes:
>
>    
>> We need a bit more than just this.  Here's my suggestion:
>>      
> I think this is much more restrictive than necessary.  Unnecessarily
> restrictive rules are more likely to be ignored, and we don't want that.
> Details below.
>    

I disagree.  The point of QMP is to allow third party tools to manage 
qemu in a robust way.  Compatibility is a key part of this.

Downstream QMP extension is a bad thing.  It means management tools have 
to have a lot of special logic to deal with different downstreams.  I 
believe we need to strongly discourage downstreams from making protocol 
changes while leaving them enough wiggle room to make the changes when 
they have no other choice.

>> 6. Downstream modification of QMP
>> -----------------------------------------------------------
>>
>> We strongly recommend that downstream consumers of QEMU do _not_
>> modify the behaviour of any QMP command or introduce new QMP commands.
>> This is important to allow management tools to support both upstream and
>> downstream versions of QMP without special logic.
>>
>> However, we recognize that it is sometimes impossible for downstreams to
>> avoid modifying QMP.  If this happens, the following rules should be
>> observed
>>      
> Bad line break.
>
>    
>> to attempt to preserve long term compatibility and interoperability.
>>      
> Better move your items 5-6) here, and make them talk about "names", not
> just commands.  Items 1-4) make much more sense with the name space
> issue settled.
>    

Good suggestion.

>> 1) Only introduce new commands.  If you want to change an existing command,
>> leave the old command in place and introduce a new one with the new
>> behaviour.
>>      
> Extending an existing command in a compatible way should be fine.  An
> extension is "compatible" if
>
> 1. it adds only optional arguments, and
>
> 2. it behaves differently only when new optional arguments are supplied.
>    

 From an implementation perspective, it's very easy to modify an 
existing command, call it a new name, then add a new command with the 
old name that passes only the compatible parameters.  So I have a hard 
time believing this is really a burden for downstreams.

Think about it from an end-to-end perspective.  If I'm writing a client 
library, and I expose a function qmp_commit(session, disk, force=false), 
and the QMP protocol introduces a vendor extension to qmp_commit(), I 
now need to change the qmp_commit() interface in my library to support 
that new parameter.  Except since this is a vendor parameter, now every 
caller has to have knowledge about dealing with different vendors.

On the other handle, a qmp_rhel_commit() command happily lives in a 
separate namespace.  In fact, in a well written client library, it could 
easily be a plugin.  The normal library consumer never has to think 
about vendor functions unless they explicitly need a function that only 
lives in a vendor namespace.

>> 2) Only introduce new asynchronous messages.  Do not change an
>> existing message.
>>      
> Why outlaw adding new data members?  What would that break?
>    

It's the same argument as above.  If you add new data members, you 
expose vendor extensions to the core protocol which has to propagate all 
the way down to the end user.

>> 3) Only introduce new error types and only use those error types in
>> new commands.
>> New commands can use existing error types, but you should never add a
>> new error type
>> to an existing command.
>>      
> A client that can't cope with unknown error types will only ever work on
> a single upstream version.  Such clients will go the way of the Dodo
> real quick.
>
> With that established: why is adding new error types bad?
>    

Same as above.  The core API should not behave any differently in a 
downstream.  Clients who only use the core API should not have to adapt 
their behaviour for downstreams.

With respect to ignoring error types, I disagree.  There's certainly 
going to be a class (perhaps the majority) of clients that treat errors 
roughly the same.  However, there is going to be a class of clients that 
very clearly handle errors in special ways depending on the error type.

>> 4) Do not introduce any new capabilities.  Capabilities directly
>> affect a client's ability to
>> parse the protocol correctly and new capabilities can not be supported
>> in an upstream
>> compatible way.  Please work capabilities through upstream first.
>>      
> This is factually incorrect.  Clients that don't recognize new
> capabilities just ignore them, and continue to work as before.
>    

I think you misinterpreted my statement.  Capabilities absolute affect 
the client's ability to parse the protocol if you enable them.  If you 
support downstream specific capabilities, then knowledge of downstream 
needs to be very deep within any client library.  That's an unnecessary 
level of complexity IMHO.

Capabilities will be rare.  I doubt a downstream would ever reasonably 
need to add a capability that isn't upstream.

N.B. backporting upstream capabilities/commands/error messages is not 
what we're talking about here.  All of those things are perfectly fine 
without using a vendor namespace.  We're specifically talking about 
adding new features that are not present upstream.  I think it's 
completely reasonable to suggest that capabilities should be done 
upstream first.

Regards,

Anthony Liguori

Re: [Qemu-devel] [PATCH] QMP: Spec: Private Extensions support

[Markus Armbruster]

Anthony Liguori <anthony@codemonkey.ws> writes:

> On 02/19/2010 07:04 AM, Markus Armbruster wrote:
>> Anthony Liguori<anthony@codemonkey.ws>  writes:
>>
>>    
>>> We need a bit more than just this.  Here's my suggestion:
>>>      
>> I think this is much more restrictive than necessary.  Unnecessarily
>> restrictive rules are more likely to be ignored, and we don't want that.
>> Details below.
>>    
>
> I disagree.  The point of QMP is to allow third party tools to manage
> qemu in a robust way.  Compatibility is a key part of this.
>
> Downstream QMP extension is a bad thing.  It means management tools
> have to have a lot of special logic to deal with different
> downstreams.  I believe we need to strongly discourage downstreams
> from making protocol changes while leaving them enough wiggle room to
> make the changes when they have no other choice.

Wearing my upstream hat: Yes, downstream extensions are undesirable.
Yes, it makes sense to discourage them.  And yes, downstreams will be
occasionally compelled to add extensions anyway.

The only truly normative bit in the whole text is our promise to reserve
the "__" prefix for vendors.  The rest is merely advice.  We have no
power whatsoever to make downstream take it.  Upstream has to lead by
argument and example.

Putting advice forward in the form of arbitrary "rules" is a good way to
ensure it's ignored.  Advice needs a rationale.  The stronger the
recommendation, the more convincing its rationale needs to be.

We already established rules for protocol evolution: we won't change
existing commands, arguments, and so forth; we won't add new mandatory
arguments; we may add optional arguments, new errors, ...  If these
rules are good enough for us, why aren't they good enough for
downstream?  I don't think we can expect downstream to follow rules we
don't intend to follow ourselves.

What follows is merely elaborating on these maxims.

>>> 6. Downstream modification of QMP
>>> -----------------------------------------------------------
>>>
>>> We strongly recommend that downstream consumers of QEMU do _not_
>>> modify the behaviour of any QMP command or introduce new QMP commands.
>>> This is important to allow management tools to support both upstream and
>>> downstream versions of QMP without special logic.
>>>
>>> However, we recognize that it is sometimes impossible for downstreams to
>>> avoid modifying QMP.  If this happens, the following rules should be
>>> observed
>>>      
>> Bad line break.
>>
>>    
>>> to attempt to preserve long term compatibility and interoperability.
>>>      
>> Better move your items 5-6) here, and make them talk about "names", not
>> just commands.  Items 1-4) make much more sense with the name space
>> issue settled.
>>    
>
> Good suggestion.
>
>>> 1) Only introduce new commands.  If you want to change an existing command,
>>> leave the old command in place and introduce a new one with the new
>>> behaviour.
>>>      
>> Extending an existing command in a compatible way should be fine.  An
>> extension is "compatible" if
>>
>> 1. it adds only optional arguments, and
>>
>> 2. it behaves differently only when new optional arguments are supplied.
>>    
>
> From an implementation perspective, it's very easy to modify an
> existing command, call it a new name, then add a new command with the
> old name that passes only the compatible parameters.  So I have a hard
> time believing this is really a burden for downstreams.
>
> Think about it from an end-to-end perspective.  If I'm writing a
> client library, and I expose a function qmp_commit(session, disk,
> force=false), and the QMP protocol introduces a vendor extension to
> qmp_commit(), I now need to change the qmp_commit() interface in my
> library to support that new parameter.  Except since this is a vendor
> parameter, now every caller has to have knowledge about dealing with
> different vendors.
>
> On the other handle, a qmp_rhel_commit() command happily lives in a
> separate namespace.  In fact, in a well written client library, it
> could easily be a plugin.  The normal library consumer never has to
> think about vendor functions unless they explicitly need a function
> that only lives in a vendor namespace.

With a bit more precision: consider monitor command "commit", available
in QMP (not yet true, but that doesn't matter).  Say we have a client
library that provides function qmp_commit(), which is merely a
straightforward marshalling wrapper around the QMP command.

Observations:

1. The client library and its users continues to work unchanged even
   when the server (QEMU) adds vendor-specific optional arguments.

2. The client library may elect to expose vendor-specific arguments in
   qmp_commit() to its users.

3. The client library may elect to provide a vendor-specific wrapper
   qmp_<vendor>_commit() around QMP command commit that exposes
   vendor-specific arguments.  In fact, that could easily be a plugin.

4. Regardless of whether the client library does 2., 3. or neither, its
   unmodified users continue to work unchanged.

I don't think we have convincing rationale for a "rule" here.

>>> 2) Only introduce new asynchronous messages.  Do not change an
>>> existing message.
>>>      
>> Why outlaw adding new data members?  What would that break?
>>    
>
> It's the same argument as above.  If you add new data members, you
> expose vendor extensions to the core protocol which has to propagate
> all the way down to the end user.

I don't think so.

As everywhere else, new members are "opt-in".  Any robust QMP client
already has to ignore members it doesn't know, to have a chance to work
with multiple upstream versions.

In fact, I can't see any qualitative difference between dealing with
multiple upstream versions and dealing with vendor-specific extensions
in different downstream versions.

Back to events.  Say I want to provide some additional information with
an existing event.  Admittedly contrived example: add a simple error
code to BLOCK_IO_ERROR, to further explain the nature of the error.  Two
options: 1. add it to the existing event, and 2. create a new event and
send it in addition to the old one (I find that clumsy).  Non-option,
because it breaks clients that know only BLOCK_IO_ERROR: create a new
event and send it instead of BLOCK_IO_ERROR

Say I'm working upstream.  I'd surely do 1.

Say I'm working downstream.  Your "rule" quite arbitrarily demands I do
2. instead.  I say "arbitrarily", because I find the rationale for it
unconvincing.

>>> 3) Only introduce new error types and only use those error types in
>>> new commands.
>>> New commands can use existing error types, but you should never add a
>>> new error type
>>> to an existing command.
>>>      
>> A client that can't cope with unknown error types will only ever work on
>> a single upstream version.  Such clients will go the way of the Dodo
>> real quick.
>>
>> With that established: why is adding new error types bad?
>>    
>
> Same as above.  The core API should not behave any differently in a
> downstream.  Clients who only use the core API should not have to
> adapt their behaviour for downstreams.

We explicitly require clients to cope with new errors (qmp-spec.txt
section 5 Compatibility Considerations).  Why reserve the privilege to
use that flexibility to upstream?

> With respect to ignoring error types, I disagree.  There's certainly
> going to be a class (perhaps the majority) of clients that treat
> errors roughly the same.  However, there is going to be a class of
> clients that very clearly handle errors in special ways depending on
> the error type.

And even they will need a fallback for unknown errors, to avoid tight
coupling to a specific upstream version.

>>> 4) Do not introduce any new capabilities.  Capabilities directly
>>> affect a client's ability to
>>> parse the protocol correctly and new capabilities can not be supported
>>> in an upstream
>>> compatible way.  Please work capabilities through upstream first.
>>>      
>> This is factually incorrect.  Clients that don't recognize new
>> capabilities just ignore them, and continue to work as before.
>>    
>
> I think you misinterpreted my statement.  Capabilities absolute affect
> the client's ability to parse the protocol if you enable them.  If you
> support downstream specific capabilities, then knowledge of downstream
> needs to be very deep within any client library.  That's an
> unnecessary level of complexity IMHO.

Capabilities are "opt-in".

Supporting downstream-specific new capabilities is no different to
supporting new capabilities added by upstream.  Why privilege upstream?

> Capabilities will be rare.  I doubt a downstream would ever reasonably
> need to add a capability that isn't upstream.

I quite agree that new capabilities are the least desirable of all
downstream extensions.

> N.B. backporting upstream capabilities/commands/error messages is not
> what we're talking about here.  All of those things are perfectly fine
> without using a vendor namespace.  We're specifically talking about
> adding new features that are not present upstream.  I think it's
> completely reasonable to suggest that capabilities should be done
> upstream first.

"Upstream first" is sensible policy, and as with all sensible policies,
it's practical policy in most cases (provided you make an honest
effort), but not in all cases.

Re: [Qemu-devel] [PATCH] QMP: Spec: Private Extensions support

[Markus Armbruster]

Anthony asked me to take a stab at rewriting his draft to something more
along the lines of what I'm thinking.  Here goes.  I put some remarks
[in brackets].

FYI, I'll be out of town until Wednesday.


6. Downstream extension of QMP
------------------------------

We recommend that downstream consumers of QEMU do *not* modify QMP.
Management tools should be able to support both upstream and downstream
versions of QMP without special logic, and downstream extensions are
inherently at odds with that.

However, we recognize that it is sometimes impossible for downstreams to
avoid modifying QMP.  Both upstream and downstream need to take care to
preserve long-term compatibility and interoperability.

To help with that, QMP reserves JSON object member names beginning with
'__' (double underscore) for downstream use ("downstream names").  This
means upstream will never use any downstream names for its commands,
arguments, errors, asynchronous events, and so forth.

Any new names downstream wishes to add must begin with '__'.  To ensure
compatibility with other downstreams, it is strongly recommended that
you prefix the commands with '__RFQDN_' where RFQDN is a valid, reverse
fully qualified domain name which you control.  For example, a qemu-kvm
specific monitor command would be:

    (qemu) __org.linux-kvm_enable_irqchip

Downstream must not change the server greeting (section 2.2) other than
to offer additional capabilities.  But see below for why even that is
discouraged.

Section '5 Compatibility Considerations' applies to downstream as well
as to upstream, obviously.  [That section needs work!]  It follows that
downstream must behave exactly like upstream for any input not
containing members with downstream names ("downstream members"), except
it may add members with downstream names to its output.

Thus, a client should not be able to distinguish downstream from
upstream as long as it doesn't send input with downstream members, and
properly ignores any downstream members in the output it receives.

[I fully support everything up to this point.  I have some reservations
on the rest, and I doubt it'll be all that useful, but I don't really
mind having it, at least not in this form.]

Advice on downstream modifications:
[I made a honest effort at capturing Anthony's intentions here,
my apologies if I screwed it up.]

1. Introducing new commands is okay.  If you want to extend an existing
   command, consider introducing a new one with the new behaviour
   instead.  [FIXME Could use a rationale: why is extending bad?  Make
   sure to cover errors, because that's needed for 3.]

2. Introducing new asynchronous messages is okay.  If you want to extend
   an existing message, consider adding a new one instead.  [FIXME Could
   use a rationale: why is extending bad?]

3. Introducing new errors for use in new commands is okay.  Adding new
   errors to existing commands counts as extension, so 1. applies.

4. New capabilities are strongly discouraged.  Capabilities are for
   evolving the basic protocol, and multiple diverging basic protocol
   dialects are most undesirable.

Re: [Qemu-devel] [PATCH] QMP: Spec: Private Extensions support

[Luiz Capitulino]

On Fri, 05 Mar 2010 20:00:25 +0100
Markus Armbruster <armbru@redhat.com> wrote:

> Anthony asked me to take a stab at rewriting his draft to something more
> along the lines of what I'm thinking.  Here goes.  I put some remarks
> [in brackets].

 I don't have strong opinions on the differences, so this look ok to me.

 Anthony?

Re: [Qemu-devel] [PATCH] QMP: Spec: Private Extensions support

[Markus Armbruster]

Anthony, no reply from you; did it fall through the cracks?  If you're
fine with my draft, I'll turn it into a proper patch.


Markus Armbruster <armbru@redhat.com> writes:

> Anthony asked me to take a stab at rewriting his draft to something more
> along the lines of what I'm thinking.  Here goes.  I put some remarks
> [in brackets].
>
> FYI, I'll be out of town until Wednesday.
>
>
> 6. Downstream extension of QMP
> ------------------------------
>
> We recommend that downstream consumers of QEMU do *not* modify QMP.
> Management tools should be able to support both upstream and downstream
> versions of QMP without special logic, and downstream extensions are
> inherently at odds with that.
>
> However, we recognize that it is sometimes impossible for downstreams to
> avoid modifying QMP.  Both upstream and downstream need to take care to
> preserve long-term compatibility and interoperability.
>
> To help with that, QMP reserves JSON object member names beginning with
> '__' (double underscore) for downstream use ("downstream names").  This
> means upstream will never use any downstream names for its commands,
> arguments, errors, asynchronous events, and so forth.
>
> Any new names downstream wishes to add must begin with '__'.  To ensure
> compatibility with other downstreams, it is strongly recommended that
> you prefix the commands with '__RFQDN_' where RFQDN is a valid, reverse
> fully qualified domain name which you control.  For example, a qemu-kvm
> specific monitor command would be:
>
>     (qemu) __org.linux-kvm_enable_irqchip
>
> Downstream must not change the server greeting (section 2.2) other than
> to offer additional capabilities.  But see below for why even that is
> discouraged.
>
> Section '5 Compatibility Considerations' applies to downstream as well
> as to upstream, obviously.  [That section needs work!]  It follows that
> downstream must behave exactly like upstream for any input not
> containing members with downstream names ("downstream members"), except
> it may add members with downstream names to its output.
>
> Thus, a client should not be able to distinguish downstream from
> upstream as long as it doesn't send input with downstream members, and
> properly ignores any downstream members in the output it receives.
>
> [I fully support everything up to this point.  I have some reservations
> on the rest, and I doubt it'll be all that useful, but I don't really
> mind having it, at least not in this form.]
>
> Advice on downstream modifications:
> [I made a honest effort at capturing Anthony's intentions here,
> my apologies if I screwed it up.]
>
> 1. Introducing new commands is okay.  If you want to extend an existing
>    command, consider introducing a new one with the new behaviour
>    instead.  [FIXME Could use a rationale: why is extending bad?  Make
>    sure to cover errors, because that's needed for 3.]
>
> 2. Introducing new asynchronous messages is okay.  If you want to extend
>    an existing message, consider adding a new one instead.  [FIXME Could
>    use a rationale: why is extending bad?]
>
> 3. Introducing new errors for use in new commands is okay.  Adding new
>    errors to existing commands counts as extension, so 1. applies.
>
> 4. New capabilities are strongly discouraged.  Capabilities are for
>    evolving the basic protocol, and multiple diverging basic protocol
>    dialects are most undesirable.

Re: [Qemu-devel] [PATCH] QMP: Spec: Private Extensions support

[Anthony Liguori]

On 05/06/2010 10:52 AM, Markus Armbruster wrote:
> Anthony, no reply from you; did it fall through the cracks?  If you're
> fine with my draft, I'll turn it into a proper patch.
>    

Yes, sorry, I thought I had already responded as such.

Regards,

Anthony Liguori

> Markus Armbruster<armbru@redhat.com>  writes:
>
>    
>> Anthony asked me to take a stab at rewriting his draft to something more
>> along the lines of what I'm thinking.  Here goes.  I put some remarks
>> [in brackets].
>>
>> FYI, I'll be out of town until Wednesday.
>>
>>
>> 6. Downstream extension of QMP
>> ------------------------------
>>
>> We recommend that downstream consumers of QEMU do *not* modify QMP.
>> Management tools should be able to support both upstream and downstream
>> versions of QMP without special logic, and downstream extensions are
>> inherently at odds with that.
>>
>> However, we recognize that it is sometimes impossible for downstreams to
>> avoid modifying QMP.  Both upstream and downstream need to take care to
>> preserve long-term compatibility and interoperability.
>>
>> To help with that, QMP reserves JSON object member names beginning with
>> '__' (double underscore) for downstream use ("downstream names").  This
>> means upstream will never use any downstream names for its commands,
>> arguments, errors, asynchronous events, and so forth.
>>
>> Any new names downstream wishes to add must begin with '__'.  To ensure
>> compatibility with other downstreams, it is strongly recommended that
>> you prefix the commands with '__RFQDN_' where RFQDN is a valid, reverse
>> fully qualified domain name which you control.  For example, a qemu-kvm
>> specific monitor command would be:
>>
>>      (qemu) __org.linux-kvm_enable_irqchip
>>
>> Downstream must not change the server greeting (section 2.2) other than
>> to offer additional capabilities.  But see below for why even that is
>> discouraged.
>>
>> Section '5 Compatibility Considerations' applies to downstream as well
>> as to upstream, obviously.  [That section needs work!]  It follows that
>> downstream must behave exactly like upstream for any input not
>> containing members with downstream names ("downstream members"), except
>> it may add members with downstream names to its output.
>>
>> Thus, a client should not be able to distinguish downstream from
>> upstream as long as it doesn't send input with downstream members, and
>> properly ignores any downstream members in the output it receives.
>>
>> [I fully support everything up to this point.  I have some reservations
>> on the rest, and I doubt it'll be all that useful, but I don't really
>> mind having it, at least not in this form.]
>>
>> Advice on downstream modifications:
>> [I made a honest effort at capturing Anthony's intentions here,
>> my apologies if I screwed it up.]
>>
>> 1. Introducing new commands is okay.  If you want to extend an existing
>>     command, consider introducing a new one with the new behaviour
>>     instead.  [FIXME Could use a rationale: why is extending bad?  Make
>>     sure to cover errors, because that's needed for 3.]
>>
>> 2. Introducing new asynchronous messages is okay.  If you want to extend
>>     an existing message, consider adding a new one instead.  [FIXME Could
>>     use a rationale: why is extending bad?]
>>
>> 3. Introducing new errors for use in new commands is okay.  Adding new
>>     errors to existing commands counts as extension, so 1. applies.
>>
>> 4. New capabilities are strongly discouraged.  Capabilities are for
>>     evolving the basic protocol, and multiple diverging basic protocol
>>     dialects are most undesirable.
>>      

[Qemu-devel] [PATCH] QMP: Add "Downstream extension of QMP" to spec

[Markus Armbruster]

Rules for how to extend QMP downstream (if you really have to) without
creating interoparability hassles.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
Aside:

* Advice on downstream modifications, items 1. and 2. could use a
  rationale.

* Section '5 Compatibility Considerations' could use some love.

 QMP/qmp-spec.txt |   55 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 55 insertions(+), 0 deletions(-)

diff --git a/QMP/qmp-spec.txt b/QMP/qmp-spec.txt
index f3c0327..dc2eb4b 100644
--- a/QMP/qmp-spec.txt
+++ b/QMP/qmp-spec.txt
@@ -215,3 +215,58 @@ Additionally, Clients must not assume any particular:
 - Order of json-object members or json-array elements
 - Amount of errors generated by a command, that is, new errors can be added
   to any existing command in newer versions of the Server
+
+6. Downstream extension of QMP
+------------------------------
+
+We recommend that downstream consumers of QEMU do *not* modify QMP.
+Management tools should be able to support both upstream and downstream
+versions of QMP without special logic, and downstream extensions are
+inherently at odds with that.
+
+However, we recognize that it is sometimes impossible for downstreams to
+avoid modifying QMP.  Both upstream and downstream need to take care to
+preserve long-term compatibility and interoperability.
+
+To help with that, QMP reserves JSON object member names beginning with
+'__' (double underscore) for downstream use ("downstream names").  This
+means upstream will never use any downstream names for its commands,
+arguments, errors, asynchronous events, and so forth.
+
+Any new names downstream wishes to add must begin with '__'.  To ensure
+compatibility with other downstreams, it is strongly recommended that
+you prefix the commands with '__RFQDN_' where RFQDN is a valid, reverse
+fully qualified domain name which you control.  For example, a qemu-kvm
+specific monitor command would be:
+
+    (qemu) __org.linux-kvm_enable_irqchip
+
+Downstream must not change the server greeting (section 2.2) other than
+to offer additional capabilities.  But see below for why even that is
+discouraged.
+
+Section '5 Compatibility Considerations' applies to downstream as well
+as to upstream, obviously.  It follows that downstream must behave
+exactly like upstream for any input not containing members with
+downstream names ("downstream members"), except it may add members
+with downstream names to its output.
+
+Thus, a client should not be able to distinguish downstream from
+upstream as long as it doesn't send input with downstream members, and
+properly ignores any downstream members in the output it receives.
+
+Advice on downstream modifications:
+
+1. Introducing new commands is okay.  If you want to extend an existing
+   command, consider introducing a new one with the new behaviour
+   instead.
+
+2. Introducing new asynchronous messages is okay.  If you want to extend
+   an existing message, consider adding a new one instead.
+
+3. Introducing new errors for use in new commands is okay.  Adding new
+   errors to existing commands counts as extension, so 1. applies.
+
+4. New capabilities are strongly discouraged.  Capabilities are for
+   evolving the basic protocol, and multiple diverging basic protocol
+   dialects are most undesirable.
-- 
1.6.6.1

[Qemu-devel] Re: [PATCH] QMP: Add "Downstream extension of QMP" to spec

[Luiz Capitulino]

On Fri, 07 May 2010 11:49:42 +0200
Markus Armbruster <armbru@redhat.com> wrote:

> +Any new names downstream wishes to add must begin with '__'.  To ensure
> +compatibility with other downstreams, it is strongly recommended that
> +you prefix the commands with '__RFQDN_' where RFQDN is a valid, reverse
> +fully qualified domain name which you control.  For example, a qemu-kvm
> +specific monitor command would be:
> +
> +    (qemu) __org.linux-kvm_enable_irqchip

 It's not only for commands, but for async messages and errors as well,
otherwise looks good to me.

[Qemu-devel] [PATCH v2] QMP: Add "Downstream extension of QMP" to spec

[Markus Armbruster]

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
v2: Clarify use of __RFQDN (Thanks, Luiz!)

 QMP/qmp-spec.txt |   55 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 55 insertions(+), 0 deletions(-)

diff --git a/QMP/qmp-spec.txt b/QMP/qmp-spec.txt
index f3c0327..9d30a8c 100644
--- a/QMP/qmp-spec.txt
+++ b/QMP/qmp-spec.txt
@@ -215,3 +215,58 @@ Additionally, Clients must not assume any particular:
 - Order of json-object members or json-array elements
 - Amount of errors generated by a command, that is, new errors can be added
   to any existing command in newer versions of the Server
+
+6. Downstream extension of QMP
+------------------------------
+
+We recommend that downstream consumers of QEMU do *not* modify QMP.
+Management tools should be able to support both upstream and downstream
+versions of QMP without special logic, and downstream extensions are
+inherently at odds with that.
+
+However, we recognize that it is sometimes impossible for downstreams to
+avoid modifying QMP.  Both upstream and downstream need to take care to
+preserve long-term compatibility and interoperability.
+
+To help with that, QMP reserves JSON object member names beginning with
+'__' (double underscore) for downstream use ("downstream names").  This
+means upstream will never use any downstream names for its commands,
+arguments, errors, asynchronous events, and so forth.
+
+Any new names downstream wishes to add must begin with '__'.  To
+ensure compatibility with other downstreams, it is strongly
+recommended that you prefix your downstram names with '__RFQDN_' where
+RFQDN is a valid, reverse fully qualified domain name which you
+control.  For example, a qemu-kvm specific monitor command would be:
+
+    (qemu) __org.linux-kvm_enable_irqchip
+
+Downstream must not change the server greeting (section 2.2) other than
+to offer additional capabilities.  But see below for why even that is
+discouraged.
+
+Section '5 Compatibility Considerations' applies to downstream as well
+as to upstream, obviously.  It follows that downstream must behave
+exactly like upstream for any input not containing members with
+downstream names ("downstream members"), except it may add members
+with downstream names to its output.
+
+Thus, a client should not be able to distinguish downstream from
+upstream as long as it doesn't send input with downstream members, and
+properly ignores any downstream members in the output it receives.
+
+Advice on downstream modifications:
+
+1. Introducing new commands is okay.  If you want to extend an existing
+   command, consider introducing a new one with the new behaviour
+   instead.
+
+2. Introducing new asynchronous messages is okay.  If you want to extend
+   an existing message, consider adding a new one instead.
+
+3. Introducing new errors for use in new commands is okay.  Adding new
+   errors to existing commands counts as extension, so 1. applies.
+
+4. New capabilities are strongly discouraged.  Capabilities are for
+   evolving the basic protocol, and multiple diverging basic protocol
+   dialects are most undesirable.
-- 
1.6.6.1

[Qemu-devel] Re: [PATCH v2] QMP: Add "Downstream extension of QMP" to spec

[Luiz Capitulino]

On Mon, 10 May 2010 09:16:05 +0200
Markus Armbruster <armbru@redhat.com> wrote:

> Signed-off-by: Markus Armbruster <armbru@redhat.com>

 Looks good to me.

> ---
> v2: Clarify use of __RFQDN (Thanks, Luiz!)
> 
>  QMP/qmp-spec.txt |   55 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 files changed, 55 insertions(+), 0 deletions(-)
> 
> diff --git a/QMP/qmp-spec.txt b/QMP/qmp-spec.txt
> index f3c0327..9d30a8c 100644
> --- a/QMP/qmp-spec.txt
> +++ b/QMP/qmp-spec.txt
> @@ -215,3 +215,58 @@ Additionally, Clients must not assume any particular:
>  - Order of json-object members or json-array elements
>  - Amount of errors generated by a command, that is, new errors can be added
>    to any existing command in newer versions of the Server
> +
> +6. Downstream extension of QMP
> +------------------------------
> +
> +We recommend that downstream consumers of QEMU do *not* modify QMP.
> +Management tools should be able to support both upstream and downstream
> +versions of QMP without special logic, and downstream extensions are
> +inherently at odds with that.
> +
> +However, we recognize that it is sometimes impossible for downstreams to
> +avoid modifying QMP.  Both upstream and downstream need to take care to
> +preserve long-term compatibility and interoperability.
> +
> +To help with that, QMP reserves JSON object member names beginning with
> +'__' (double underscore) for downstream use ("downstream names").  This
> +means upstream will never use any downstream names for its commands,
> +arguments, errors, asynchronous events, and so forth.
> +
> +Any new names downstream wishes to add must begin with '__'.  To
> +ensure compatibility with other downstreams, it is strongly
> +recommended that you prefix your downstram names with '__RFQDN_' where
> +RFQDN is a valid, reverse fully qualified domain name which you
> +control.  For example, a qemu-kvm specific monitor command would be:
> +
> +    (qemu) __org.linux-kvm_enable_irqchip
> +
> +Downstream must not change the server greeting (section 2.2) other than
> +to offer additional capabilities.  But see below for why even that is
> +discouraged.
> +
> +Section '5 Compatibility Considerations' applies to downstream as well
> +as to upstream, obviously.  It follows that downstream must behave
> +exactly like upstream for any input not containing members with
> +downstream names ("downstream members"), except it may add members
> +with downstream names to its output.
> +
> +Thus, a client should not be able to distinguish downstream from
> +upstream as long as it doesn't send input with downstream members, and
> +properly ignores any downstream members in the output it receives.
> +
> +Advice on downstream modifications:
> +
> +1. Introducing new commands is okay.  If you want to extend an existing
> +   command, consider introducing a new one with the new behaviour
> +   instead.
> +
> +2. Introducing new asynchronous messages is okay.  If you want to extend
> +   an existing message, consider adding a new one instead.
> +
> +3. Introducing new errors for use in new commands is okay.  Adding new
> +   errors to existing commands counts as extension, so 1. applies.
> +
> +4. New capabilities are strongly discouraged.  Capabilities are for
> +   evolving the basic protocol, and multiple diverging basic protocol
> +   dialects are most undesirable.