[Qemu-devel] QMP forward compatibility support

[Qemu-devel] QMP forward compatibility support

[Luiz Capitulino]

 Hi.

 We (Markus and I) are working on getting QMP forward compatibility support,
supported. :)

 We have a plan for it and I'd like to ask the CC'ed people to review it.

 Needless to say, but the objective here is to add new commands, arguments,
async messages and protocol features w/o breaking existing clients.

General Plan
------------

1. QMP should describe itself, ie. it should dump all accepted commands,
their replies and arguments, async messages and protocol features. All in
JSON format

2. Protocol features are advertised by the (already existent) capabilities
array, but are _disabled_ by default. The exception is async messages,
which are _enabled_ by default

3. We should add command(s) to enable/disable protocol features

4. Proper feature negotiation is done in pause mode. That's, clients
interested in enabling new protocol features should start QEMU in
pause mode and enable the features they are interested in using

Work to do
----------

1. Markus is already working on getting QMP to describe itself, we
should get a status/proposal soon. This is the most important part,
as we may want to change query-commands output, query-events is
missing and hopefully we can generate user docs from this

2. We need to add commands to enable/disable features. Not high priority,
as the only existing feature is async messages which are enabled by
default

Re: [Qemu-devel] QMP forward compatibility support

[Anthony Liguori]

On 01/11/2010 12:34 PM, Luiz Capitulino wrote:
>   Hi.
>
>   We (Markus and I) are working on getting QMP forward compatibility support,
> supported. :)
>
>   We have a plan for it and I'd like to ask the CC'ed people to review it.
>
>   Needless to say, but the objective here is to add new commands, arguments,
> async messages and protocol features w/o breaking existing clients.
>
> General Plan
> ------------
>
> 1. QMP should describe itself, ie. it should dump all accepted commands,
> their replies and arguments, async messages and protocol features. All in
> JSON format
>
> 2. Protocol features are advertised by the (already existent) capabilities
> array, but are _disabled_ by default. The exception is async messages,
> which are _enabled_ by default
>    

Any reason to have an exception like this?

> 3. We should add command(s) to enable/disable protocol features
>
> 4. Proper feature negotiation is done in pause mode. That's, clients
> interested in enabling new protocol features should start QEMU in
> pause mode and enable the features they are interested in using
>    

Why does this matter?

We should be careful to support connecting to a VM long after it's been 
started so any requirement like this is likely to cause trouble.

Regards,

Anthony Liguori

Re: [Qemu-devel] QMP forward compatibility support

[Daniel P. Berrange]

On Mon, Jan 11, 2010 at 12:57:15PM -0600, Anthony Liguori wrote:
> On 01/11/2010 12:34 PM, Luiz Capitulino wrote:
> >  Hi.
> >
> >  We (Markus and I) are working on getting QMP forward compatibility 
> >  support,
> >supported. :)
> >
> >  We have a plan for it and I'd like to ask the CC'ed people to review it.
> >
> >  Needless to say, but the objective here is to add new commands, 
> >  arguments,
> >async messages and protocol features w/o breaking existing clients.
> >
> >General Plan
> >------------
> >
> >1. QMP should describe itself, ie. it should dump all accepted commands,
> >their replies and arguments, async messages and protocol features. All in
> >JSON format
> >
> >2. Protocol features are advertised by the (already existent) capabilities
> >array, but are _disabled_ by default. The exception is async messages,
> >which are _enabled_ by default
> 
> Any reason to have an exception like this?

Personally I'd be fine with async events being off by default too.
There's no point in me receiving events I'm not able to handle,
so might as well only give me events I actually request.

> >3. We should add command(s) to enable/disable protocol features
> >
> >4. Proper feature negotiation is done in pause mode. That's, clients
> >interested in enabling new protocol features should start QEMU in
> >pause mode and enable the features they are interested in using
> >   
> 
> Why does this matter?
> 
> We should be careful to support connecting to a VM long after it's been 
> started so any requirement like this is likely to cause trouble.

Indeed, this won't work for libvirt. If you stop the libvirtd daemon
while a guest is running, and then start the libvirtd daemon again, it
will need to re-negotiate features while the guest is running.

Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: [Qemu-devel] QMP forward compatibility support

[Luiz Capitulino]

On Mon, 11 Jan 2010 12:57:15 -0600
Anthony Liguori <anthony@codemonkey.ws> wrote:

> On 01/11/2010 12:34 PM, Luiz Capitulino wrote:
> >   Hi.
> >
> >   We (Markus and I) are working on getting QMP forward compatibility support,
> > supported. :)
> >
> >   We have a plan for it and I'd like to ask the CC'ed people to review it.
> >
> >   Needless to say, but the objective here is to add new commands, arguments,
> > async messages and protocol features w/o breaking existing clients.
> >
> > General Plan
> > ------------
> >
> > 1. QMP should describe itself, ie. it should dump all accepted commands,
> > their replies and arguments, async messages and protocol features. All in
> > JSON format
> >
> > 2. Protocol features are advertised by the (already existent) capabilities
> > array, but are _disabled_ by default. The exception is async messages,
> > which are _enabled_ by default
> >    
> 
> Any reason to have an exception like this?

 As async messages were one of the reasons for having QMP, I thought
that there was a consensus that making it part of the "original"
protocol was ok, meaning that they would be always available.

 That's the only reason.

> > 3. We should add command(s) to enable/disable protocol features
> >
> > 4. Proper feature negotiation is done in pause mode. That's, clients
> > interested in enabling new protocol features should start QEMU in
> > pause mode and enable the features they are interested in using
> >    
> 
> Why does this matter?
> 
> We should be careful to support connecting to a VM long after it's been 
> started so any requirement like this is likely to cause trouble.

 If I understood Markus's concerns correctly, he thinks that feature
negotiation should happen before the protocol is "running", ie. make
it part of the initial handshake.

 Now, if everything is disabled by default and qemu might be running
already, do we really need to have a handshake?

Re: [Qemu-devel] QMP forward compatibility support

[Anthony Liguori]

On 01/11/2010 06:04 PM, Luiz Capitulino wrote:
>
>   As async messages were one of the reasons for having QMP, I thought
> that there was a consensus that making it part of the "original"
> protocol was ok, meaning that they would be always available.
>
>   That's the only reason.
>    

Right, but then it's not a capability, it's a core feature.  I just 
think it would be odd to advertise something as a capability and have it 
not behave like other ones.

>>> 3. We should add command(s) to enable/disable protocol features
>>>
>>> 4. Proper feature negotiation is done in pause mode. That's, clients
>>> interested in enabling new protocol features should start QEMU in
>>> pause mode and enable the features they are interested in using
>>>
>>>        
>> Why does this matter?
>>
>> We should be careful to support connecting to a VM long after it's been
>> started so any requirement like this is likely to cause trouble.
>>      
>   If I understood Markus's concerns correctly, he thinks that feature
> negotiation should happen before the protocol is "running", ie. make
> it part of the initial handshake.
>    

I think forcing the negotiation before executing any commands is a good 
idea.  But I don't think requiring the guest not to be running is 
necessary or even useful.

You don't want to have to support disabling and enabling features in the 
middle of a protocol session because then you have to deal with weird 
semantics.

>   Now, if everything is disabled by default and qemu might be running
> already, do we really need to have a handshake?
>    

I think it's valuable to have a discrete period of time when no commands 
have been executed where features can be enabled.  It simplifies some 
nasty edge conditions regarding enabling features while there are 
outstanding commands in flight.

Regards,

Anthony Liguori

Re: [Qemu-devel] QMP forward compatibility support

[Markus Armbruster]

Anthony Liguori <anthony@codemonkey.ws> writes:

> On 01/11/2010 06:04 PM, Luiz Capitulino wrote:
[...]
>>>> 3. We should add command(s) to enable/disable protocol features
>>>>
>>>> 4. Proper feature negotiation is done in pause mode. That's, clients
>>>> interested in enabling new protocol features should start QEMU in
>>>> pause mode and enable the features they are interested in using
>>>>
>>>>        
>>> Why does this matter?
>>>
>>> We should be careful to support connecting to a VM long after it's been
>>> started so any requirement like this is likely to cause trouble.
>>>      
>>   If I understood Markus's concerns correctly, he thinks that feature
>> negotiation should happen before the protocol is "running", ie. make
>> it part of the initial handshake.
>>    
>
> I think forcing the negotiation before executing any commands is a
> good idea.  But I don't think requiring the guest not to be running is
> necessary or even useful.
>
> You don't want to have to support disabling and enabling features in
> the middle of a protocol session because then you have to deal with
> weird semantics.

Agreed.

>>   Now, if everything is disabled by default and qemu might be running
>> already, do we really need to have a handshake?
>>    
>
> I think it's valuable to have a discrete period of time when no
> commands have been executed where features can be enabled.  It
> simplifies some nasty edge conditions regarding enabling features
> while there are outstanding commands in flight.

That's exactly why I lobbied for feature negotiation in the initial
handshake, i.e. client connects, server sends greeting with features,
client sends features it wants enabled, and only then we enter the
normal command loop.  Protocol that don't have that tend to get it
retrofitted when they evolve.

Let's do it now, before backward compatibility concerns force us to do
it in an ugly way.

Re: [Qemu-devel] QMP forward compatibility support

[Luiz Capitulino]

On Mon, 11 Jan 2010 18:24:24 -0600
Anthony Liguori <anthony@codemonkey.ws> wrote:

> On 01/11/2010 06:04 PM, Luiz Capitulino wrote:
> >
> >   As async messages were one of the reasons for having QMP, I thought
> > that there was a consensus that making it part of the "original"
> > protocol was ok, meaning that they would be always available.
> >
> >   That's the only reason.
> >    
> 
> Right, but then it's not a capability, it's a core feature.  I just 
> think it would be odd to advertise something as a capability and have it 
> not behave like other ones.

 Ok, so options are: call it a core feature and don't change anything
or call it a capability and make it behave like any other capability.

 What's the better? Should we vote? :) Daniel seems to prefer the
later.

> >>> 3. We should add command(s) to enable/disable protocol features
> >>>
> >>> 4. Proper feature negotiation is done in pause mode. That's, clients
> >>> interested in enabling new protocol features should start QEMU in
> >>> pause mode and enable the features they are interested in using
> >>>
> >>>        
> >> Why does this matter?
> >>
> >> We should be careful to support connecting to a VM long after it's been
> >> started so any requirement like this is likely to cause trouble.
> >>      
> >   If I understood Markus's concerns correctly, he thinks that feature
> > negotiation should happen before the protocol is "running", ie. make
> > it part of the initial handshake.
> >    
> 
> I think forcing the negotiation before executing any commands is a good 
> idea.  But I don't think requiring the guest not to be running is 
> necessary or even useful.
> 
> You don't want to have to support disabling and enabling features in the 
> middle of a protocol session because then you have to deal with weird 
> semantics.

 That's true, but I thought that doing that with pause mode was
going to be better because it didn't require any change on QMP side.

 If this is a bad approach, then I was wrong.

 Now, making this part of the initial handshake brings some more
design decisions and by making async messages a capability helps
to test them.

 I'm thinking in something like this:

1. Connection is made, the greeting message is sent and QMP is
in 'handshake mode'

2. In this mode only commands to enable/disable protocol
capabilities are allowed

3. When the client is done with the setup, it issues the
command 'enable-qmp', which puts the protocol into 'running mode',
where any command is accepted

Re: [Qemu-devel] QMP forward compatibility support

[Luiz Capitulino]

On Tue, 12 Jan 2010 09:16:43 +0100
Markus Armbruster <armbru@redhat.com> wrote:

> >>   Now, if everything is disabled by default and qemu might be running
> >> already, do we really need to have a handshake?
> >>    
> >
> > I think it's valuable to have a discrete period of time when no
> > commands have been executed where features can be enabled.  It
> > simplifies some nasty edge conditions regarding enabling features
> > while there are outstanding commands in flight.
> 
> That's exactly why I lobbied for feature negotiation in the initial
> handshake, i.e. client connects, server sends greeting with features,
> client sends features it wants enabled, and only then we enter the
> normal command loop.  Protocol that don't have that tend to get it
> retrofitted when they evolve.
> 
> Let's do it now, before backward compatibility concerns force us to do
> it in an ugly way.

 Yes, it was good to bring out the issue after all.

 Please, review the design I'm proposing in my reply to Anthony.

Re: [Qemu-devel] QMP forward compatibility support

[Markus Armbruster]

Luiz Capitulino <lcapitulino@redhat.com> writes:

> On Mon, 11 Jan 2010 18:24:24 -0600
> Anthony Liguori <anthony@codemonkey.ws> wrote:
>
>> On 01/11/2010 06:04 PM, Luiz Capitulino wrote:
>> >
>> >   As async messages were one of the reasons for having QMP, I thought
>> > that there was a consensus that making it part of the "original"
>> > protocol was ok, meaning that they would be always available.
>> >
>> >   That's the only reason.
>> >    
>> 
>> Right, but then it's not a capability, it's a core feature.  I just 
>> think it would be odd to advertise something as a capability and have it 
>> not behave like other ones.
>
>  Ok, so options are: call it a core feature and don't change anything
> or call it a capability and make it behave like any other capability.
>
>  What's the better? Should we vote? :) Daniel seems to prefer the
> later.

If it's optional, leave it off by default because the consensus seems to
be to leave all optional features off by default.

It should be optional if we want to support clients that don't want it.
I don't think coping with it would be a terrible burden on clients, but
neither is having to ask for it.  Personally, I'd make it optional.

>> >>> 3. We should add command(s) to enable/disable protocol features
>> >>>
>> >>> 4. Proper feature negotiation is done in pause mode. That's, clients
>> >>> interested in enabling new protocol features should start QEMU in
>> >>> pause mode and enable the features they are interested in using
>> >>>
>> >>>        
>> >> Why does this matter?
>> >>
>> >> We should be careful to support connecting to a VM long after it's been
>> >> started so any requirement like this is likely to cause trouble.
>> >>      
>> >   If I understood Markus's concerns correctly, he thinks that feature
>> > negotiation should happen before the protocol is "running", ie. make
>> > it part of the initial handshake.
>> >    
>> 
>> I think forcing the negotiation before executing any commands is a good 
>> idea.  But I don't think requiring the guest not to be running is 
>> necessary or even useful.
>> 
>> You don't want to have to support disabling and enabling features in the 
>> middle of a protocol session because then you have to deal with weird 
>> semantics.
>
>  That's true, but I thought that doing that with pause mode was
> going to be better because it didn't require any change on QMP side.
>
>  If this is a bad approach, then I was wrong.
>
>  Now, making this part of the initial handshake brings some more
> design decisions and by making async messages a capability helps
> to test them.
>
>  I'm thinking in something like this:
>
> 1. Connection is made, the greeting message is sent and QMP is
> in 'handshake mode'
>
> 2. In this mode only commands to enable/disable protocol
> capabilities are allowed
>
> 3. When the client is done with the setup, it issues the
> command 'enable-qmp', which puts the protocol into 'running mode',
> where any command is accepted

Really "any command"?  What about commands to enable/disable protocol
capabilities?

Re: [Qemu-devel] QMP forward compatibility support

[Luiz Capitulino]

On Wed, 13 Jan 2010 17:53:38 +0100
Markus Armbruster <armbru@redhat.com> wrote:

> Luiz Capitulino <lcapitulino@redhat.com> writes:
> 
> > On Mon, 11 Jan 2010 18:24:24 -0600
> > Anthony Liguori <anthony@codemonkey.ws> wrote:
> >
> >> On 01/11/2010 06:04 PM, Luiz Capitulino wrote:
> >> >
> >> >   As async messages were one of the reasons for having QMP, I thought
> >> > that there was a consensus that making it part of the "original"
> >> > protocol was ok, meaning that they would be always available.
> >> >
> >> >   That's the only reason.
> >> >    
> >> 
> >> Right, but then it's not a capability, it's a core feature.  I just 
> >> think it would be odd to advertise something as a capability and have it 
> >> not behave like other ones.
> >
> >  Ok, so options are: call it a core feature and don't change anything
> > or call it a capability and make it behave like any other capability.
> >
> >  What's the better? Should we vote? :) Daniel seems to prefer the
> > later.
> 
> If it's optional, leave it off by default because the consensus seems to
> be to leave all optional features off by default.
> 
> It should be optional if we want to support clients that don't want it.
> I don't think coping with it would be a terrible burden on clients, but
> neither is having to ask for it.  Personally, I'd make it optional.

 Ok, will do.

> >> >>> 3. We should add command(s) to enable/disable protocol features
> >> >>>
> >> >>> 4. Proper feature negotiation is done in pause mode. That's, clients
> >> >>> interested in enabling new protocol features should start QEMU in
> >> >>> pause mode and enable the features they are interested in using
> >> >>>
> >> >>>        
> >> >> Why does this matter?
> >> >>
> >> >> We should be careful to support connecting to a VM long after it's been
> >> >> started so any requirement like this is likely to cause trouble.
> >> >>      
> >> >   If I understood Markus's concerns correctly, he thinks that feature
> >> > negotiation should happen before the protocol is "running", ie. make
> >> > it part of the initial handshake.
> >> >    
> >> 
> >> I think forcing the negotiation before executing any commands is a good 
> >> idea.  But I don't think requiring the guest not to be running is 
> >> necessary or even useful.
> >> 
> >> You don't want to have to support disabling and enabling features in the 
> >> middle of a protocol session because then you have to deal with weird 
> >> semantics.
> >
> >  That's true, but I thought that doing that with pause mode was
> > going to be better because it didn't require any change on QMP side.
> >
> >  If this is a bad approach, then I was wrong.
> >
> >  Now, making this part of the initial handshake brings some more
> > design decisions and by making async messages a capability helps
> > to test them.
> >
> >  I'm thinking in something like this:
> >
> > 1. Connection is made, the greeting message is sent and QMP is
> > in 'handshake mode'
> >
> > 2. In this mode only commands to enable/disable protocol
> > capabilities are allowed
> >
> > 3. When the client is done with the setup, it issues the
> > command 'enable-qmp', which puts the protocol into 'running mode',
> > where any command is accepted
> 
> Really "any command"?  What about commands to enable/disable protocol
> capabilities?

 I think that playing with some protocol bits might be safe, like
enabling async messages.

 I'm not saying this is a good practice, but forbidding it seems a bit
extreme at first.

Re: [Qemu-devel] QMP forward compatibility support

[Markus Armbruster]

Luiz Capitulino <lcapitulino@redhat.com> writes:

> On Wed, 13 Jan 2010 17:53:38 +0100
> Markus Armbruster <armbru@redhat.com> wrote:
>
>> Luiz Capitulino <lcapitulino@redhat.com> writes:
[...]
>> >  I'm thinking in something like this:
>> >
>> > 1. Connection is made, the greeting message is sent and QMP is
>> > in 'handshake mode'
>> >
>> > 2. In this mode only commands to enable/disable protocol
>> > capabilities are allowed
>> >
>> > 3. When the client is done with the setup, it issues the
>> > command 'enable-qmp', which puts the protocol into 'running mode',
>> > where any command is accepted
>> 
>> Really "any command"?  What about commands to enable/disable protocol
>> capabilities?
>
>  I think that playing with some protocol bits might be safe, like
> enabling async messages.
>
>  I'm not saying this is a good practice, but forbidding it seems a bit
> extreme at first.

Allowing stuff when it turns out to be needed is less painful than
outlawing stuff when it turns out to be problematic.

Re: [Qemu-devel] QMP forward compatibility support

[Luiz Capitulino]

On Wed, 13 Jan 2010 18:38:57 +0100
Markus Armbruster <armbru@redhat.com> wrote:

> Luiz Capitulino <lcapitulino@redhat.com> writes:
> 
> > On Wed, 13 Jan 2010 17:53:38 +0100
> > Markus Armbruster <armbru@redhat.com> wrote:
> >
> >> Luiz Capitulino <lcapitulino@redhat.com> writes:
> [...]
> >> >  I'm thinking in something like this:
> >> >
> >> > 1. Connection is made, the greeting message is sent and QMP is
> >> > in 'handshake mode'
> >> >
> >> > 2. In this mode only commands to enable/disable protocol
> >> > capabilities are allowed
> >> >
> >> > 3. When the client is done with the setup, it issues the
> >> > command 'enable-qmp', which puts the protocol into 'running mode',
> >> > where any command is accepted
> >> 
> >> Really "any command"?  What about commands to enable/disable protocol
> >> capabilities?
> >
> >  I think that playing with some protocol bits might be safe, like
> > enabling async messages.
> >
> >  I'm not saying this is a good practice, but forbidding it seems a bit
> > extreme at first.
> 
> Allowing stuff when it turns out to be needed is less painful than
> outlawing stuff when it turns out to be problematic.

 I forgot to mention we can block them, that's fine. :)

 So, do we agree with the general design? I'll cook a RFC series.

Re: [Qemu-devel] QMP forward compatibility support

[Jamie Lokier]

Markus Armbruster wrote:
> It should be optional if we want to support clients that don't want it.
> I don't think coping with it would be a terrible burden on clients, but
> neither is having to ask for it.  Personally, I'd make it optional.

It wouldn't be a terrible burden, but it'll be easier to write "quick
and dirty" synchronous clients if they know they don't have to
continuously consume output, to make sure the pipe doesn't fill up,
except immediately after they send a command they wait for the response.

So I'd make async responses optional too.

By quick and dirty, I am talking about one-liner Perl invocations in
shell scripts which control a qemu instance.  Something which is easy
with multi-monitor support :-)

-- Jamie