[Qemu-devel] Machine-readable or parseable qemu output

[Qemu-devel] Machine-readable or parseable qemu output

[Amit Shah]

Hello,

Continuing from the thread at [1], building on Daniel's suggestions, I
have jot down a few points as to how a libqemumonitor API could be
developed.

To recap, there has to be an interface to the qemu monitor in
a way that wouldn't break even if the monitor output changes. This API
will remain the same, so that consumers (libvirt, etc) can safely
integrate with the monitor.

Please see Dan's email at [1] to get the background details.

[1] http://www.mail-archive.com/qemu-devel@nongnu.org/msg14615.html

Here are some initial thoughts. Comments?

- Have a libqemumonitor.so that will abstract out output from qemu and
  provide a machine-readble output for the consumer

- Registering with a particular qemu instance:
  - qemu_instance = attach_to_qemu("/path/to/socket");

- The strings in qemu monitor can change but the libqemumonitor has to
  change accordingly as well to keep the API consistent.

- The API could be something like:
  - execute_qemu_command(qemu_instance, command, args...);

  This will then return with a value of:
  - 0: Success (no output)
  - -ve: Failure
  - +ve: Log level type from QEMU monitor output (see below)

- For qemu, a loglevel can be added so that the message output is
  disciplined (as seen in kernel printk messages):
  - term_printf(QINFO, "Beginning migration\n");
  - term_printf(QERR, "Migration failed\n");

- -ve return values can be reserved to indicate that the link with the qemu
  monitor went down or some other problem with the libqemumonitor and
  the monitor interface

- In addition to the return value seen of the loglevel type from the
  monitor, there can be additional return values that can be provided
  based on the command executed. This can be queried by:
  - get_info_on_prev_command(qemu_instance, &ret);
  - This can give command-specific return values, like success,
  invalid parameter, etc.

- The string generated by qemu monitor output should never be exposed to
  the consumer

- Some monitor commands will generate some string that might be of use
  to the consumer. Such a string will be passed to the consumer via some
  other api, like
  - get_string_from_last_command(qemu_instance, some_context);

- There might be output that's asynchronous to the command. Finding the
  correlation between the command and the output should not be the
  library's job; an async_socket can be created per qemu instance and
  the consumer can select() on this socket to poll for data. Whenever
  some information becomes available, we can push it out. If the qemu
  output would contain more information for async output, this can be
  passed on to the consumer.

Amit.

[Qemu-devel] Re: Machine-readable or parseable qemu output

[Daniel P. Berrange]

On Wed, Jan 14, 2009 at 04:40:05PM +0530, Amit Shah wrote:
> Hello,
> 
> Continuing from the thread at [1], building on Daniel's suggestions, I
> have jot down a few points as to how a libqemumonitor API could be
> developed.
> 
> To recap, there has to be an interface to the qemu monitor in
> a way that wouldn't break even if the monitor output changes. This API
> will remain the same, so that consumers (libvirt, etc) can safely
> integrate with the monitor.
> 
> Please see Dan's email at [1] to get the background details.
> 
> [1] http://www.mail-archive.com/qemu-devel@nongnu.org/msg14615.html
> 
> Here are some initial thoughts. Comments?

I'd like to add one more requirement

 - Works with existing QEMU monitor for releases >= 0.8.0

This is because libvirt currently supports all QEMU >= 0.8.0, so if
we're to be able to make use of this library we can't restrict it
to just new releases. If it can't then we have to keep our existing 
monitor interaction impl around and thus adding a 2nd impl using
this library wouldn't be much of a benefit. 

> - Have a libqemumonitor.so that will abstract out output from qemu and
>   provide a machine-readble output for the consumer
> 
> - Registering with a particular qemu instance:
>   - qemu_instance = attach_to_qemu("/path/to/socket");

Also need ability to attach to a TTY, and/or a pre-opened file
descriptor.

> - The strings in qemu monitor can change but the libqemumonitor has to
>   change accordingly as well to keep the API consistent.
> 
> - The API could be something like:
>   - execute_qemu_command(qemu_instance, command, args...);
> 
>   This will then return with a value of:
>   - 0: Success (no output)
>   - -ve: Failure
>   - +ve: Log level type from QEMU monitor output (see below)
> 
> - For qemu, a loglevel can be added so that the message output is
>   disciplined (as seen in kernel printk messages):
>   - term_printf(QINFO, "Beginning migration\n");
>   - term_printf(QERR, "Migration failed\n");
> 
> - -ve return values can be reserved to indicate that the link with the qemu
>   monitor went down or some other problem with the libqemumonitor and
>   the monitor interface
> 
> - In addition to the return value seen of the loglevel type from the
>   monitor, there can be additional return values that can be provided
>   based on the command executed. This can be queried by:
>   - get_info_on_prev_command(qemu_instance, &ret);
>   - This can give command-specific return values, like success,
>   invalid parameter, etc.
> 
> - The string generated by qemu monitor output should never be exposed to
>   the consumer
> 
> - Some monitor commands will generate some string that might be of use
>   to the consumer. Such a string will be passed to the consumer via some
>   other api, like
>   - get_string_from_last_command(qemu_instance, some_context);

I think it'd be better to return all relevant data in the API used
to invoke the command, rather than just restricting to a int return
code. Having to run a separate API here means we have to have the extra
complexity internally to keep the last command string in a thread-local
data storage. 

> - There might be output that's asynchronous to the command. Finding the
>   correlation between the command and the output should not be the
>   library's job; an async_socket can be created per qemu instance and
>   the consumer can select() on this socket to poll for data. Whenever
>   some information becomes available, we can push it out. If the qemu
>   output would contain more information for async output, this can be
>   passed on to the consumer.

Agreed - having a 2nd monitor console for emitting async events is much
simpler way to go.

Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: [Qemu-devel] Re: Machine-readable or parseable qemu output

[Anthony Liguori]

Daniel P. Berrange wrote:
> On Wed, Jan 14, 2009 at 04:40:05PM +0530, Amit Shah wrote:
>   
>> Hello,
>>
>> Continuing from the thread at [1], building on Daniel's suggestions, I
>> have jot down a few points as to how a libqemumonitor API could be
>> developed.
>>
>> To recap, there has to be an interface to the qemu monitor in
>> a way that wouldn't break even if the monitor output changes. This API
>> will remain the same, so that consumers (libvirt, etc) can safely
>> integrate with the monitor.
>>
>> Please see Dan's email at [1] to get the background details.
>>
>> [1] http://www.mail-archive.com/qemu-devel@nongnu.org/msg14615.html
>>
>> Here are some initial thoughts. Comments?
>>     
>
> I'd like to add one more requirement
>
>  - Works with existing QEMU monitor for releases >= 0.8.0
>   

I think this is a bad requirement.

> This is because libvirt currently supports all QEMU >= 0.8.0,

libvirt has no bearing on upstream QEMU support :-)

>  so if
> we're to be able to make use of this library we can't restrict it
> to just new releases.

But you can conditionally use the new library instead of your custom 
parsing code for newer QEMU versions.

Regards,

Anthony Liguori

Re: [Qemu-devel] Re: Machine-readable or parseable qemu output

[Daniel P. Berrange]

On Thu, Jan 15, 2009 at 02:27:05PM -0600, Anthony Liguori wrote:
> Daniel P. Berrange wrote:
> >On Wed, Jan 14, 2009 at 04:40:05PM +0530, Amit Shah wrote:
> >  
> >>Hello,
> >>
> >>Continuing from the thread at [1], building on Daniel's suggestions, I
> >>have jot down a few points as to how a libqemumonitor API could be
> >>developed.
> >>
> >>To recap, there has to be an interface to the qemu monitor in
> >>a way that wouldn't break even if the monitor output changes. This API
> >>will remain the same, so that consumers (libvirt, etc) can safely
> >>integrate with the monitor.
> >>
> >>Please see Dan's email at [1] to get the background details.
> >>
> >>[1] http://www.mail-archive.com/qemu-devel@nongnu.org/msg14615.html
> >>
> >>Here are some initial thoughts. Comments?
> >>    
> >
> >I'd like to add one more requirement
> >
> > - Works with existing QEMU monitor for releases >= 0.8.0
> >  
> 
> I think this is a bad requirement.
> 
> >This is because libvirt currently supports all QEMU >= 0.8.0,
> 
> libvirt has no bearing on upstream QEMU support :-)
> 
> > so if
> >we're to be able to make use of this library we can't restrict it
> >to just new releases.
> 
> But you can conditionally use the new library instead of your custom 
> parsing code for newer QEMU versions.

That is true, but if there are a number of apps around which want to
support multiple versions of QEMU, it is beneficial to centralize
this conditional logic in libqemumonitor.so instead, of making each
app implement the compat logic for the existing monitor format. I'm
not against adding a new machine friendly monitor format, I'd just
prefer it if one library API could provide impl for both old and new
format, obviously preferring to use the new format where available.

Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: [Qemu-devel] Re: Machine-readable or parseable qemu output

[Anthony Liguori]

Daniel P. Berrange wrote:
> On Thu, Jan 15, 2009 at 02:27:05PM -0600, Anthony Liguori wrote:
>   
>> Daniel P. Berrange wrote:
>>     
>>> On Wed, Jan 14, 2009 at 04:40:05PM +0530, Amit Shah wrote:
>>>  
>>>       
>>>> Hello,
>>>>
>>>> Continuing from the thread at [1], building on Daniel's suggestions, I
>>>> have jot down a few points as to how a libqemumonitor API could be
>>>> developed.
>>>>
>>>> To recap, there has to be an interface to the qemu monitor in
>>>> a way that wouldn't break even if the monitor output changes. This API
>>>> will remain the same, so that consumers (libvirt, etc) can safely
>>>> integrate with the monitor.
>>>>
>>>> Please see Dan's email at [1] to get the background details.
>>>>
>>>> [1] http://www.mail-archive.com/qemu-devel@nongnu.org/msg14615.html
>>>>
>>>> Here are some initial thoughts. Comments?
>>>>    
>>>>         
>>> I'd like to add one more requirement
>>>
>>> - Works with existing QEMU monitor for releases >= 0.8.0
>>>  
>>>       
>> I think this is a bad requirement.
>>
>>     
>>> This is because libvirt currently supports all QEMU >= 0.8.0,
>>>       
>> libvirt has no bearing on upstream QEMU support :-)
>>
>>     
>>> so if
>>> we're to be able to make use of this library we can't restrict it
>>> to just new releases.
>>>       
>> But you can conditionally use the new library instead of your custom 
>> parsing code for newer QEMU versions.
>>     
>
> That is true, but if there are a number of apps around which want to
> support multiple versions of QEMU, it is beneficial to centralize
> this conditional logic in libqemumonitor.so instead, of making each
> app implement the compat logic for the existing monitor format. I'm
> not against adding a new machine friendly monitor format, I'd just
> prefer it if one library API could provide impl for both old and new
> format, obviously preferring to use the new format where available.
>   

But then we have to design the API around what was parsable from the old 
monitor interface or provides complex probing interfaces to see what 
features are supported.  This is a much more constrained problem for 
libvirt because it only accesses a subset of what the monitor does but 
when you do it at the QEMU level, there will be an expectation that we 
need to support everything that's done through the monitor.  I'm 
concerned this will get very ugly very quickly.

Regards,

Anthony Liguori

> Daniel
>   

Re: [Qemu-devel] Re: Machine-readable or parseable qemu output

[Avi Kivity]

Daniel P. Berrange wrote:
>> But you can conditionally use the new library instead of your custom 
>> parsing code for newer QEMU versions.
>>     
>
> That is true, but if there are a number of apps around which want to
> support multiple versions of QEMU, it is beneficial to centralize
> this conditional logic in libqemumonitor.so instead, of making each
> app implement the compat logic for the existing monitor format. I'm
> not against adding a new machine friendly monitor format, I'd just
> prefer it if one library API could provide impl for both old and new
> format, obviously preferring to use the new format where available.
>   

Perhaps libvirt can donate its parsing code for use as a fallback in 
libqemumonitor.so.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: [Qemu-devel] Re: Machine-readable or parseable qemu output

[Anthony Liguori]

Avi Kivity wrote:
> Daniel P. Berrange wrote:
>>> But you can conditionally use the new library instead of your custom 
>>> parsing code for newer QEMU versions.
>>>     
>>
>> That is true, but if there are a number of apps around which want to
>> support multiple versions of QEMU, it is beneficial to centralize
>> this conditional logic in libqemumonitor.so instead, of making each
>> app implement the compat logic for the existing monitor format. I'm
>> not against adding a new machine friendly monitor format, I'd just
>> prefer it if one library API could provide impl for both old and new
>> format, obviously preferring to use the new format where available.
>>   
>
> Perhaps libvirt can donate its parsing code for use as a fallback in 
> libqemumonitor.so.

Or split it out into an another library.  I really don't think it's a 
good idea to maintain it within QEMU because it's against an interface 
that was never supported in QEMU and doesn't exist in the existing QEMU 
code base.

Regards,

Anthony Liguori

[Qemu-devel] Re: Machine-readable or parseable qemu output

[Avi Kivity]

Amit Shah wrote:
> Hello,
>
> Continuing from the thread at [1], building on Daniel's suggestions, I
> have jot down a few points as to how a libqemumonitor API could be
> developed.
>
> To recap, there has to be an interface to the qemu monitor in
> a way that wouldn't break even if the monitor output changes. This API
> will remain the same, so that consumers (libvirt, etc) can safely
> integrate with the monitor.
>
> Please see Dan's email at [1] to get the background details.
>
> [1] http://www.mail-archive.com/qemu-devel@nongnu.org/msg14615.html
>
> Here are some initial thoughts. Comments?
>
> - Have a libqemumonitor.so that will abstract out output from qemu and
>   provide a machine-readble output for the consumer
>
> - Registering with a particular qemu instance:
>   - qemu_instance = attach_to_qemu("/path/to/socket");
>   

qemu chrdev ("tcp:host:port", etc)

> - The strings in qemu monitor can change but the libqemumonitor has to
>   change accordingly as well to keep the API consistent.
>   

No, they can't.  The monitor client and qemu will be upgraded independently.

> - The API could be something like:
>   - execute_qemu_command(qemu_instance, command, args...);
>
>   

It'd much prefer

qemu_hotplug_nic(...);
qemu_migrate(..., callback);

> - In addition to the return value seen of the loglevel type from the
>   monitor, there can be additional return values that can be provided
>   based on the command executed. This can be queried by:
>   - get_info_on_prev_command(qemu_instance, &ret);
>   - This can give command-specific return values, like success,
>   invalid parameter, etc.
>   

No, the command handler should parse the output expected from executing 
the command.  If the command is asynchronous, the command handler should 
provide a callback.

> - The string generated by qemu monitor output should never be exposed to
>   the consumer
>   

Right.

> - Some monitor commands will generate some string that might be of use
>   to the consumer. Such a string will be passed to the consumer via some
>   other api, like
>   - get_string_from_last_command(qemu_instance, some_context);
>   

Output parameters will do fine.

> - There might be output that's asynchronous to the command. Finding the
>   correlation between the command and the output should not be the
>   library's job; an async_socket can be created per qemu instance and
>   the consumer can select() on this socket to poll for data. Whenever
>   some information becomes available, we can push it out. If the qemu
>   output would contain more information for async output, this can be
>   passed on to the consumer.
>   

You're moving the protocol handling out from the protocol parser...

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Re: Machine-readable or parseable qemu output

[Anthony Liguori]

Avi Kivity wrote:
> Amit Shah wrote:
>> Hello,
>>
>> Continuing from the thread at [1], building on Daniel's suggestions, I
>> have jot down a few points as to how a libqemumonitor API could be
>> developed.
>>
>> To recap, there has to be an interface to the qemu monitor in
>> a way that wouldn't break even if the monitor output changes. This API
>> will remain the same, so that consumers (libvirt, etc) can safely
>> integrate with the monitor.
>>
>> Please see Dan's email at [1] to get the background details.
>>
>> [1] http://www.mail-archive.com/qemu-devel@nongnu.org/msg14615.html
>>
>> Here are some initial thoughts. Comments?
>>
>> - Have a libqemumonitor.so that will abstract out output from qemu and
>>   provide a machine-readble output for the consumer
>>
>> - Registering with a particular qemu instance:
>>   - qemu_instance = attach_to_qemu("/path/to/socket");
>>   
>
> qemu chrdev ("tcp:host:port", etc)

Yes.  The QEMU chrdev code can be reused too.

>> - The strings in qemu monitor can change but the libqemumonitor has to
>>   change accordingly as well to keep the API consistent.
>>   
>
> No, they can't.  The monitor client and qemu will be upgraded 
> independently.

+1

>> - The API could be something like:
>>   - execute_qemu_command(qemu_instance, command, args...);
>>
>>   
>
> It'd much prefer
>
> qemu_hotplug_nic(...);
> qemu_migrate(..., callback);

I concur.

>> - In addition to the return value seen of the loglevel type from the
>>   monitor, there can be additional return values that can be provided
>>   based on the command executed. This can be queried by:
>>   - get_info_on_prev_command(qemu_instance, &ret);
>>   - This can give command-specific return values, like success,
>>   invalid parameter, etc.
>>   
>
> No, the command handler should parse the output expected from 
> executing the command.  If the command is asynchronous, the command 
> handler should provide a callback.

Agreed.

Regards,

Anthony Liguori

[Qemu-devel] Re: Machine-readable or parseable qemu output

[Richard W.M. Jones]

On Wed, Jan 14, 2009 at 04:40:05PM +0530, Amit Shah wrote:
> - Have a libqemumonitor.so that will abstract out output from qemu and
>   provide a machine-readble output for the consumer

Why do we need a C API at all?  IMHO it'd be better just to make the
existing qemu monitor output more machine-friendly, meaning consistent
delimiters so that programs can reliably resynchronize with the
output, and consistent guarantees on error messages.

[Replying to Dan's point about qemu >= 0.8]

We could differentiate based on the qemu prompt.  For "machine-
friendly" releases of qemu, change the prompt slightly so we know.

Rich.

-- 
Richard Jones, Emerging Technologies, Red Hat  http://et.redhat.com/~rjones
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://et.redhat.com/~rjones/virt-df/

[Qemu-devel] Re: Machine-readable or parseable qemu output

[Avi Kivity]

Richard W.M. Jones wrote:
> We could differentiate based on the qemu prompt.  For "machine-
> friendly" releases of qemu, change the prompt slightly so we know.
>   

The machine friendly protocol shouldn't have a prompt at all.  Prompts 
are human friendly.

-- 
error compiling committee.c: too many arguments to function

[Qemu-devel] Re: Machine-readable or parseable qemu output

[Richard W.M. Jones]

On Wed, Jan 14, 2009 at 01:31:34PM +0200, Avi Kivity wrote:
> Richard W.M. Jones wrote:
>> We could differentiate based on the qemu prompt.  For "machine-
>> friendly" releases of qemu, change the prompt slightly so we know.
>>   
>
> The machine friendly protocol shouldn't have a prompt at all.  Prompts  
> are human friendly.

By machine friendly, I meant a monitor which humans can still use, but
which provides some guarantees if a machine is trying to parse the
output.  The prompt indicating, 'I'm ready for a command'.

Anyhow, I'm going to stay out of this debate now ...

Rich.

-- 
Richard Jones, Emerging Technologies, Red Hat  http://et.redhat.com/~rjones
Read my OCaml programming blog: http://camltastic.blogspot.com/
Fedora now supports 68 OCaml packages (the OPEN alternative to F#)
http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora

[Qemu-devel] Re: Machine-readable or parseable qemu output

[Daniel P. Berrange]

On Wed, Jan 14, 2009 at 11:29:38AM +0000, Richard W.M. Jones wrote:
> On Wed, Jan 14, 2009 at 04:40:05PM +0530, Amit Shah wrote:
> > - Have a libqemumonitor.so that will abstract out output from qemu and
> >   provide a machine-readble output for the consumer
> 
> Why do we need a C API at all?  IMHO it'd be better just to make the
> existing qemu monitor output more machine-friendly, meaning consistent
> delimiters so that programs can reliably resynchronize with the
> output, and consistent guarantees on error messages.

Well if we have async messages over a separate channel, we can already
reliably resynchronize with the output, because QEMU will eventually
produce its '(qemu) ' prompt on a new line. So if something goes wrong
you just need to skip until you find the prompt again.  Changing QEMU
monitor prompt will just break all existing libvirt deployments, and
any other programs relying on currently prompt.

IMHO, any libqemumonitor.so should be made to work with current monitor
format as its starting point, and then extend from there, not change
any existing characteristics. This provides forwards & backwards compat
for all apps, albeit at cost of slightly more complex internals for
the libqemumonitor.so - but this complexity will be centralized in one
place instead of all apps using QEMU, so this is still a net win.

Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: [Qemu-devel] Re: Machine-readable or parseable qemu output

[Dor Laor]

Daniel P. Berrange wrote:
> On Wed, Jan 14, 2009 at 11:29:38AM +0000, Richard W.M. Jones wrote:
>   
>> On Wed, Jan 14, 2009 at 04:40:05PM +0530, Amit Shah wrote:
>>     
>>> - Have a libqemumonitor.so that will abstract out output from qemu and
>>>   provide a machine-readble output for the consumer
>>>       
>> Why do we need a C API at all?  IMHO it'd be better just to make the
>> existing qemu monitor output more machine-friendly, meaning consistent
>> delimiters so that programs can reliably resynchronize with the
>> output, and consistent guarantees on error messages.
>>     
>
> Well if we have async messages over a separate channel, we can already
> reliably resynchronize with the output, because QEMU will eventually
> produce its '(qemu) ' prompt on a new line. So if something goes wrong
> you just need to skip until you find the prompt again.  Changing QEMU
> monitor prompt will just break all existing libvirt deployments, and
> any other programs relying on currently prompt.
>
> IMHO, any libqemumonitor.so should be made to work with current monitor
> format as its starting point, and then extend from there, not change
> any existing characteristics. This provides forwards & backwards compat
> for all apps, albeit at cost of slightly more complex internals for
> the libqemumonitor.so - but this complexity will be centralized in one
> place instead of all apps using QEMU, so this is still a net win.
>
> Daniel
>   
What about something like this:
- Extend the struct term_cmd_t we have today to support
  async commands, and offer callbacks and status callback (can be the 
same one)
- All the commands return binary results, some with enum for translating 
into strings later on
- The monitor instance is responsible for translating (potentially) the 
binary into text.
- Each such term_cmd_t will be registered (even dynamically) to the 
monitor interface.
- We can have multiple monitors running.
    - Each of them can be defined for sync/async commands.
- Every monitor instance can be a different implementation
  - monitor instance #1 will parse/print the output one way, instance #2 
-  a bit different
- The interface for adding monitor commands:
    - add_cmd(term_cmd_t *cmd)
    - del_cmd(term_cmd_t *cmd)
- The interface for libmonitor:
    - un/register_monitor(monitor_t *m)
- All of the commands define the in/out structures in a common
  location for libmonitor to parse.

This way each monitor instance can define the messages it handles 
(async,sync,in,out,etc).
Several monitors can live in parallel and even several implementations 
for parsing.
The default access would be the qemu_chr_device.
The current behavior should be the first implementation of libmonitor.so.
Others (aka machine readable) will follow soon.

Dor

Re: [Qemu-devel] Re: Machine-readable or parseable qemu output

[Avi Kivity]

Daniel P. Berrange wrote:
> Well if we have async messages over a separate channel, 

I dislike this intensely; IMO this is indicates the protocol is badly 
designed.  Of course, the qemu monitor protocol wasn't designed to do 
this, but we should fix this instead of finding workarounds.

> we can already
> reliably resynchronize with the output, because QEMU will eventually
> produce its '(qemu) ' prompt on a new line.

That's just begging for a nic or some other object named '(qemu) '.

Also, "if something goes wrong", recovering the prompt isn't helpful.  
The VM and the management interface are out of sync, any surprise can 
follow.  Better to disconnect, reconnect, and query for all state (like 
you do when the management application recovers from a crash).

>  So if something goes wrong
> you just need to skip until you find the prompt again.  Changing QEMU
> monitor prompt will just break all existing libvirt deployments, and
> any other programs relying on currently prompt.
>
> IMHO, any libqemumonitor.so should be made to work with current monitor
> format as its starting point, and then extend from there, not change
> any existing characteristics. This provides forwards & backwards compat
> for all apps, albeit at cost of slightly more complex internals for
> the libqemumonitor.so - but this complexity will be centralized in one
> place instead of all apps using QEMU, so this is still a net win.
>   

That means we'll be doomed to ad-hoc extensions forever, with extension 
N+1 needing to check the previous N extensions for compatibility.  I'd 
like to see a protocol with extensibility, asynchronity, and 
discoverability built in, not tacked on.  The old protocol can be 
supported separately but not extended any more.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Re: Machine-readable or parseable qemu output

[Anthony Liguori]

Daniel P. Berrange wrote:
> On Wed, Jan 14, 2009 at 11:29:38AM +0000, Richard W.M. Jones wrote:
>   
>> On Wed, Jan 14, 2009 at 04:40:05PM +0530, Amit Shah wrote:
>>     
>>> - Have a libqemumonitor.so that will abstract out output from qemu and
>>>   provide a machine-readble output for the consumer
>>>       
>> Why do we need a C API at all?  IMHO it'd be better just to make the
>> existing qemu monitor output more machine-friendly, meaning consistent
>> delimiters so that programs can reliably resynchronize with the
>> output, and consistent guarantees on error messages.
>>     
>
> Well if we have async messages over a separate channel, we can already
> reliably resynchronize with the output, because QEMU will eventually
> produce its '(qemu) ' prompt on a new line. So if something goes wrong
> you just need to skip until you find the prompt again.  Changing QEMU
> monitor prompt will just break all existing libvirt deployments, and
> any other programs relying on currently prompt.
>
> IMHO, any libqemumonitor.so should be made to work with current monitor
> format as its starting point, and then extend from there, not change
> any existing characteristics. This provides forwards & backwards compat
> for all apps, albeit at cost of slightly more complex internals for
> the libqemumonitor.so - but this complexity will be centralized in one
> place instead of all apps using QEMU, so this is still a net win.
>   

The monitor today is *not* meant to be parsed.  I know people do it, but 
it's not intended to.  I don't want to introduce all the nutty 
complexity into QEMU to pretend like it's been a supported interface 
since 0.8.0.  Let's make it a supported, stable interface and move forward.

Regards,

Anthony Liguori

> Daniel
>   

Re: [Qemu-devel] Re: Machine-readable or parseable qemu output

[Jamie Lokier]

Although it's nice to have a monitor which is human-readable, many
programs have a separate monitor _program_ which connects over a unix
socket to the running app.  For example, Bind has rndc, ntpd has
ntpdc, apache has apachectl, and Samba has smbcontrol.

This gives all the benefits of a machine-accessible monitor, including
asynchronous events, waiting for long commands (like migrate) while
still doing useful things (like change the CD while migrating? :-)

It's much more scriptable to have a separate program, and you still
have a nice text UI with line-editing, help, etc.  Possibly a nicer
UI, since it's not limited to telnet/serial.

For QEMU, maybe it would work to define a machine-accessible socket
protocol, and move the current QEMU monitor code to a separate app
which connects over that protocol?  If QEMU is invoked with arguments
asking for the current monitor over e.g. a socket, telnet socket or
serial port, it'd be easy for QEMU to fork/exec the monitor app.  The
same app would be invokable from scripts.

-- Jamie