[Qemu-devel] fix/re-do query-command-line-options

[Qemu-devel] fix/re-do query-command-line-options

[Amos Kong]

Hi QEMU/Libvirt list,

When I worked on query-command-line-options, I first used some marcos [1] to
generate two config & option tables. This will cover all the options,
but it returns a string, it's difficult for libvirt to parse and use
it.

Finally I got a suggestion to read info from new interface (QemuOpts),
We add opts info to vm_config_groups[], query-command-line-options
will visit the array. It doesn't conver all the options, but the
latest options are covered, so this implementation is acceptable.

Problem:
* QemuOpts was designed just for options with parameter, some new option
  without parameters is lost in query output (eg: -enable-fips)
* block drive uses three QemuOpts, it's legacy issue.
* QemuOpts of some options aren't updated, it might be difficult to
  updated when we add some new parameters
* other

We discussed to reimplement this command, but it seems DEF maroc is the
only point to cover all the options, all the options are described in
qemu-options.hx

I'm considering to reuse the DEF marocs to generate a table, try to
return the crude info if QemuOpts doesn't cover it.
Or maintain a split array (like vm_config_groups[]), it only contains
the option without parameter (option name & help info).

If you touched some problem of the query-command-line-options, welcome
to reply it, I will try to satisfy your requests.


Thanks, Amos

[1] http://www.redhat.com/archives/libvir-list/2013-January/msg01656.html


|#elif defined(QEMU_OPTIONS_GENERATE_CONFIG)
|
|#define DEF(option, opt_arg, opt_enum, opt_help, arch_mask)    \
|      opt_help,
|
|#define DEFHEADING(text)
|#define ARCHHEADING(text, arch_mask)
|
|#elif defined(QEMU_OPTIONS_GENERATE_NAME)
|
|#define DEF(option, opt_arg, opt_enum, opt_help, arch_mask)    \
|      option,
|
|#define DEFHEADING(text)
|#define ARCHHEADING(text, arch_mask)

    char const *optionstr[] = {
#define QEMU_OPTIONS_GENERATE_NAME
#include "qemu-options-wrapper.h"
    };

    char const *configstr[] = {
#define QEMU_OPTIONS_GENERATE_CONFIG
#include "qemu-options-wrapper.h"
    };

-- 
			Amos.

Re: [Qemu-devel] [resend] fix/re-do query-command-line-options

[Amos Kong]

Hi QEMU/Libvirt list,

When I worked on query-command-line-options, I first used some marcos [1] to
generate two config & option tables. This will cover all the options,
but it returns a string, it's difficult for libvirt to parse and use
it.

| {"execute": "query-config", "arguments" : {"name": "boot"}}
| {"return": {"config": "-boot [order=drives][,once=drives][,menu=on|off]\n
|   ....
|   'rb_timeout': the timeout before guest reboot when boot failed, unit is ms\n"}}


Finally I got a suggestion to read info from new interface (QemuOpts),
We add opts info to vm_config_groups[], query-command-line-options
will visit the array. It doesn't conver all the options, but the
latest options are covered, so this implementation is acceptable.

Problem:
* QemuOpts was designed just for options _have parameter_, some new option
  without parameters is lost in query output (eg: -enable-fips)
* block drive uses three QemuOpts, it's legacy issue. it's fixed by 968854c8.
* QemuOpts of some options aren't updated, it might be difficult to
  updated when we add some new parameters
* other

We discussed to reimplement this command, but it seems DEF maroc is the
only point to cover all the options, all the options are described in
qemu-options.hx

I'm considering to reuse the DEF marocs to generate a table, try to
return the crude info if QemuOpts doesn't cover it.
Or maintain a split array (like vm_config_groups[]), it only contains
the option without parameter (option name & help info).

If you touched some problem of the query-command-line-options, welcome
to reply it, I will try to satisfy your requests.


Thanks, Amos

[1] http://www.redhat.com/archives/libvir-list/2013-January/msg01656.html


|#elif defined(QEMU_OPTIONS_GENERATE_CONFIG)
|
|#define DEF(option, opt_arg, opt_enum, opt_help, arch_mask)    \
|      opt_help,
|
|#define DEFHEADING(text)
|#define ARCHHEADING(text, arch_mask)
|
|#elif defined(QEMU_OPTIONS_GENERATE_NAME)
|
|#define DEF(option, opt_arg, opt_enum, opt_help, arch_mask)    \
|      option,
|
|#define DEFHEADING(text)
|#define ARCHHEADING(text, arch_mask)

    char const *optionstr[] = {
#define QEMU_OPTIONS_GENERATE_NAME
#include "qemu-options-wrapper.h"
    };

    char const *configstr[] = {
#define QEMU_OPTIONS_GENERATE_CONFIG
#include "qemu-options-wrapper.h"
    };

Re: [Qemu-devel] fix/re-do query-command-line-options

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1439 bytes --]

On 12/22/2013 07:19 PM, Amos Kong wrote:

> Problem:
> * QemuOpts was designed just for options with parameter, some new option
>   without parameters is lost in query output (eg: -enable-fips)
> * block drive uses three QemuOpts, it's legacy issue.
> * QemuOpts of some options aren't updated, it might be difficult to
>   updated when we add some new parameters
> * other

Indeed, the existing implementation provides some, but not all, of what
libvirt would like to do, so improving it would be welcome.

> 
> We discussed to reimplement this command, but it seems DEF maroc is the
> only point to cover all the options, all the options are described in
> qemu-options.hx
> 
> I'm considering to reuse the DEF marocs to generate a table, try to
> return the crude info if QemuOpts doesn't cover it.
> Or maintain a split array (like vm_config_groups[]), it only contains
> the option without parameter (option name & help info).

I'll be interested in seeing what happens with that.

> 
> |#elif defined(QEMU_OPTIONS_GENERATE_CONFIG)
> |
> |#define DEF(option, opt_arg, opt_enum, opt_help, arch_mask)    \

Indeed, making use of DEF() to gather more information for use in the
query command seems like the right way to pick up all boolean options,
in addition to options that take arguments.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] fix/re-do query-command-line-options

[Markus Armbruster]

Amos Kong <akong@redhat.com> writes:

> Hi QEMU/Libvirt list,
>
> When I worked on query-command-line-options, I first used some marcos [1] to
> generate two config & option tables. This will cover all the options,
> but it returns a string, it's difficult for libvirt to parse and use
> it.
>
> Finally I got a suggestion to read info from new interface (QemuOpts),
> We add opts info to vm_config_groups[], query-command-line-options
> will visit the array. It doesn't conver all the options, but the
> latest options are covered, so this implementation is acceptable.
>
> Problem:
> * QemuOpts was designed just for options with parameter, some new option
>   without parameters is lost in query output (eg: -enable-fips)

Yes, and that's an issue not just for query-command-line-options, but
also for -writeconfig / -readconfig.

> * block drive uses three QemuOpts, it's legacy issue.

Code goes into contortions to extend the old option without breaking it.

Part of the contortions is your commit 968854c, which adds special-case
code to make the -drive's parameters visible in
query-command-line-options again, after commit 0006383 made them
invisible.

More on the invisible parameters problem below.

> * QemuOpts of some options aren't updated, it might be difficult to
>   updated when we add some new parameters

What do you have in mind here?

> * other

Here's one: QemuOpts *sucks* :)

Apart from its general suckage, which has been discussed at some length
elsewhere, it sucks as a schema, because it's not nearly expressive
enough.

With QemuOpts, we can basically enumerate acceptable keys and how their
values should be parsed, plus a few extras like an implied key.

On value parsing, all we have is "on"/"off", uint64_t in decimal,
uint64_t in fancy size syntax, and "anything".  This is both more and
less than a QAPI schema.

It's more, because it specifies both type (e.g. uint64_t) and external
syntax (e.g. decimal).

It's less, because it has fewer basic types, and no composite types.

On acceptable keys, all we can do is a list fixed at compile-time.

We can't express discriminated unions, such as when key "type" has value
"tap", then keys ... are acceptable, when it has value "bridge", then
keys ... are acceptable, and so forth.  We routinely do that in QAPI
schemata.

We can't collect acceptable keys at run-time.  For instance, -device
accepts general keys "id", "bus", and a discriminated union of device
model properties, with discriminator "driver".  But the available device
models and their properties are only known at runtime.

When we need something like that (and we need it often), we give up and
specify "any key=value accepted", leaving the actual checking to the
code using the option.  All command line introspection can see then is
"the option exists, and it takes a key=value,... argument".  Which
borders on useless.

Options that have this issue now: -acpitable, -smbios, -netdev, -net,
-device, -object.  Note that several of these are fairly new, which
makes me expect more of them.  -drive isn't on this list only because
your commit 968854c got it off the list after commit 0006383 got it on
it.

QAPI schema can't yet solve this problem, either.  That's why it doesn't
cover -device / device_add.  Since all QMP commands need to be covered,
and QMP surely needs a command to plug devices, the problem will have to
be solved there eventually.

> We discussed to reimplement this command, but it seems DEF maroc is the
> only point to cover all the options, all the options are described in
> qemu-options.hx
>
> I'm considering to reuse the DEF marocs to generate a table, try to
> return the crude info if QemuOpts doesn't cover it.
> Or maintain a split array (like vm_config_groups[]), it only contains
> the option without parameter (option name & help info).

I think the data you can usefully collect with this approach is
approximately the data getopt_long()[*] gets: list of named command line
options, and whether they take an argument.

You can use this data to fill in options not covered by QemuOpts.  This
is a definite improvement.

It still falls short of fully solving the command line introspection
problem.

However, I'm not into rejecting imperfect incremental improvements we
can have now in favor of perfect solutions we can maybe have some day.
Go right ahead with your incremental improvement!

> If you touched some problem of the query-command-line-options, welcome
> to reply it, I will try to satisfy your requests.
>
>
> Thanks, Amos
>
> [1] http://www.redhat.com/archives/libvir-list/2013-January/msg01656.html

[*] Which we don't use, because we prefer our command line
idiosyncratic.

Re: [Qemu-devel] fix/re-do query-command-line-options

[Paolo Bonzini]

Il 28/01/2014 10:36, Markus Armbruster ha scritto:
> I think the data you can usefully collect with this approach is
> approximately the data getopt_long()[*] gets: list of named command line
> options, and whether they take an argument.
>
> You can use this data to fill in options not covered by QemuOpts.  This
> is a definite improvement.
>
> It still falls short of fully solving the command line introspection
> problem.
>
> However, I'm not into rejecting imperfect incremental improvements we
> can have now in favor of perfect solutions we can maybe have some day.
> Go right ahead with your incremental improvement!

It depends.  If we can agree on the following:

(a) do not add non-QemuOpts options (we haven't for a while)

(b) document the QemuOpts schema for -acpitable, -smbios, -netdev, -net. 
  These options validate the options with OptsVisitor, so we could do 
without QemuOpts schema, but we know the schema won't bitrot because we 
never remove suboptions.

(c) do not add any more QemuOpts options without a schema, and use 
-object instead.

Then:

(a) there is no need to cover non-QemuOpts options in 
query-command-line-options.  libvirt can treat them as crystallized.

(b) documenting the schemata is not harder than what Amos proposed.

(c) schema inspection for objects remains a problem, but one that we 
need to solve anyway so it doesn't affect query-command-line-options.

Do you agree?

Paolo

Re: [Qemu-devel] fix/re-do query-command-line-options

[Markus Armbruster]

Paolo Bonzini <pbonzini@redhat.com> writes:

> Il 28/01/2014 10:36, Markus Armbruster ha scritto:
>> I think the data you can usefully collect with this approach is
>> approximately the data getopt_long()[*] gets: list of named command line
>> options, and whether they take an argument.
>>
>> You can use this data to fill in options not covered by QemuOpts.  This
>> is a definite improvement.
>>
>> It still falls short of fully solving the command line introspection
>> problem.
>>
>> However, I'm not into rejecting imperfect incremental improvements we
>> can have now in favor of perfect solutions we can maybe have some day.
>> Go right ahead with your incremental improvement!
>
> It depends.  If we can agree on the following:
>
> (a) do not add non-QemuOpts options (we haven't for a while)

That would mean we can't ever add an option that doesn't take an
argument again.

However, we need to somehow stuff those into QemuOpts anyway, so
-readconfig / -writeconfig can cover them.

> (b) document the QemuOpts schema for -acpitable, -smbios, -netdev,
> -net. These options validate the options with OptsVisitor, so we could
> do without QemuOpts schema, but we know the schema won't bitrot
> because we never remove suboptions.

-device?

> (c) do not add any more QemuOpts options without a schema, and use
> -object instead.
>
> Then:
>
> (a) there is no need to cover non-QemuOpts options in
> query-command-line-options.  libvirt can treat them as crystallized.

Some options are undef #ifdef.  That's actually a good idea, because it
permits finding out which options are available via command line
introspection.

Now, what if a non-QemuOpts option is under #ifdef?  I haven't
checked...  Even if there isn't one now, are we ready to give up the
ability to do that for good?

> (b) documenting the schemata is not harder than what Amos proposed.
>
> (c) schema inspection for objects remains a problem, but one that we
> need to solve anyway so it doesn't affect query-command-line-options.

As long as we don't have such schema inspection, I'm rather reluctant to
reject alternative means to solve problems people have *now*.

> Do you agree?

It depends :)

Re: [Qemu-devel] fix/re-do query-command-line-options

[Paolo Bonzini]

Il 28/01/2014 12:55, Markus Armbruster ha scritto:
> Paolo Bonzini <pbonzini@redhat.com> writes:
>
>> Il 28/01/2014 10:36, Markus Armbruster ha scritto:
>>> I think the data you can usefully collect with this approach is
>>> approximately the data getopt_long()[*] gets: list of named command line
>>> options, and whether they take an argument.
>>>
>>> You can use this data to fill in options not covered by QemuOpts.  This
>>> is a definite improvement.
>>>
>>> It still falls short of fully solving the command line introspection
>>> problem.
>>>
>>> However, I'm not into rejecting imperfect incremental improvements we
>>> can have now in favor of perfect solutions we can maybe have some day.
>>> Go right ahead with your incremental improvement!
>>
>> It depends.  If we can agree on the following:
>>
>> (a) do not add non-QemuOpts options (we haven't for a while)
>
> That would mean we can't ever add an option that doesn't take an
> argument again.

We can add it under an existing QemuOpts group or invent a new one (like 
we did for -rt or -msg).

> However, we need to somehow stuff those into QemuOpts anyway, so
> -readconfig / -writeconfig can cover them.

Yep.

>> (b) document the QemuOpts schema for -acpitable, -smbios, -netdev,
>> -net. These options validate the options with OptsVisitor, so we could
>> do without QemuOpts schema, but we know the schema won't bitrot
>> because we never remove suboptions.
>
> -device?

-device already provides its own introspection via "qom-list-types" and 
"-device driver,?".

-object doesn't have an equivalent of "-device driver,?", but that's a 
separate problem.

>> (a) there is no need to cover non-QemuOpts options in
>> query-command-line-options.  libvirt can treat them as crystallized.
>
> Some options are undef #ifdef.  That's actually a good idea, because it
> permits finding out which options are available via command line
> introspection.
>
> Now, what if a non-QemuOpts option is under #ifdef?  I haven't
> checked...  Even if there isn't one now, are we ready to give up the
> ability to do that for good?

There are some:

- legacy slirp options -tftp/-bootp/-redir/-smb

- -enable-fips is what triggered this discussion, but we found another 
solution in Libvirt

- -tpmdev is enabled only if CONFIG_TPM, but its presence can be queried 
via qom-list-types too.

>> (b) documenting the schemata is not harder than what Amos proposed.
>>
>> (c) schema inspection for objects remains a problem, but one that we
>> need to solve anyway so it doesn't affect query-command-line-options.
>
> As long as we don't have such schema inspection, I'm rather reluctant to
> reject alternative means to solve problems people have *now*.

Note the "doesn't affect query-command-line-options" part.  Amos's patch 
do not solve the problem of which classes can be instantiated with 
-object, or of which properties can be used.

Paolo

>> Do you agree?
>
> It depends :)
>

Re: [Qemu-devel] fix/re-do query-command-line-options

[Markus Armbruster]

[Adding Luiz...]

Paolo Bonzini <pbonzini@redhat.com> writes:

> Il 28/01/2014 12:55, Markus Armbruster ha scritto:
>> Paolo Bonzini <pbonzini@redhat.com> writes:
>>
>>> Il 28/01/2014 10:36, Markus Armbruster ha scritto:
>>>> I think the data you can usefully collect with this approach is
>>>> approximately the data getopt_long()[*] gets: list of named command line
>>>> options, and whether they take an argument.
>>>>
>>>> You can use this data to fill in options not covered by QemuOpts.  This
>>>> is a definite improvement.
>>>>
>>>> It still falls short of fully solving the command line introspection
>>>> problem.
>>>>
>>>> However, I'm not into rejecting imperfect incremental improvements we
>>>> can have now in favor of perfect solutions we can maybe have some day.
>>>> Go right ahead with your incremental improvement!
>>>
>>> It depends.  If we can agree on the following:
>>>
>>> (a) do not add non-QemuOpts options (we haven't for a while)
>>
>> That would mean we can't ever add an option that doesn't take an
>> argument again.
>
> We can add it under an existing QemuOpts group or invent a new one
> (like we did for -rt or -msg).

Do you mean -rtc?

-msg takes a timestamp=on/off argument.  I guess that doesn't feel too
forced, because we could conceivably have more settings related to error
reporting.

Do you mean to suggest new options should always be done in a way that
makes them fit into QemuOpts?

How would you add something like -S, -nodefaults, or -daemonize?

>> However, we need to somehow stuff those into QemuOpts anyway, so
>> -readconfig / -writeconfig can cover them.
>
> Yep.

The question is whether we should extend QemuOpts to cover options
without arguments, or change the options without arguments to fit into
existing QemuOpts, e.g. by making them all syntactic sugar for a
suitable QemuOpts-style option.

If we do the latter, we need to tell customers of command line
introspection to look only for the desugared forms.

>>> (b) document the QemuOpts schema for -acpitable, -smbios, -netdev,
>>> -net. These options validate the options with OptsVisitor, so we could
>>> do without QemuOpts schema, but we know the schema won't bitrot
>>> because we never remove suboptions.
>>
>> -device?
>
> -device already provides its own introspection via "qom-list-types"
> and "-device driver,?".

We're discussing introspection via QMP, so "-device driver,help" doesn't
count.

qom-list-types with argument "implements": "device" together with
device-list-properties indeed lets you introspect device models and
their properties.  You then need to to know how to translate that to
the command line.  For instance, you need to know that "type": "on/off"
in the former means "type": "boolean" in the latter, and so forth.

What I'd like to see is more unified introspection, not this smattering
of one-offs hacked up without too much thought to serve some immediate
need we have now.

We already have something that aspires to be our unified interface
description: QAPI schemata.  Perhaps we should make it our unified
introspection mechanism while we're at it.

> -object doesn't have an equivalent of "-device driver,?", but that's a
> separate problem.
>
>>> (a) there is no need to cover non-QemuOpts options in
>>> query-command-line-options.  libvirt can treat them as crystallized.
>>
>> Some options are undef #ifdef.  That's actually a good idea, because it
>> permits finding out which options are available via command line
>> introspection.
>>
>> Now, what if a non-QemuOpts option is under #ifdef?  I haven't
>> checked...  Even if there isn't one now, are we ready to give up the
>> ability to do that for good?
>
> There are some:
>
> - legacy slirp options -tftp/-bootp/-redir/-smb
>
> - -enable-fips is what triggered this discussion, but we found another
> solution in Libvirt

The irony is that -enable-fips should not exist :)

> - -tpmdev is enabled only if CONFIG_TPM, but its presence can be
> queried via qom-list-types too.
>
>>> (b) documenting the schemata is not harder than what Amos proposed.
>>>
>>> (c) schema inspection for objects remains a problem, but one that we
>>> need to solve anyway so it doesn't affect query-command-line-options.
>>
>> As long as we don't have such schema inspection, I'm rather reluctant to
>> reject alternative means to solve problems people have *now*.
>
> Note the "doesn't affect query-command-line-options" part.  Amos's
> patch do not solve the problem of which classes can be instantiated
> with -object, or of which properties can be used.

Possible misunderstanding on my part.  I was afraid you were arguing to
solve -object introspection *instead* of Amos's incremental improvement,
but that doesn't seem to be the case.

[...]

Re: [Qemu-devel] fix/re-do query-command-line-options

[Paolo Bonzini]

Il 28/01/2014 14:16, Markus Armbruster ha scritto:
>>> That would mean we can't ever add an option that doesn't take an
>>> argument again.
>>
>> We can add it under an existing QemuOpts group or invent a new one
>> (like we did for -rt or -msg).
>
> Do you mean -rtc?

I meant -rt, but -rtc applies just as well. :)

> -msg takes a timestamp=on/off argument.  I guess that doesn't feel too
> forced, because we could conceivably have more settings related to error
> reporting.
>
> Do you mean to suggest new options should always be done in a way that
> makes them fit into QemuOpts?

Yes.  Not into *existing* QemuOpts of course.

> How would you add something like -S, -nodefaults, or -daemonize?

I would add -nodefaults and -S to -machine.  I would deprecate 
-daemonize.  But that's just for the sake of -readconfig.  Consumers of 
introspection can just "know" that those options are there.

> The question is whether we should extend QemuOpts to cover options
> without arguments, or change the options without arguments to fit into
> existing QemuOpts, e.g. by making them all syntactic sugar for a
> suitable QemuOpts-style option.
>
> If we do the latter, we need to tell customers of command line
> introspection to look only for the desugared forms.

Yeah.

>>>> (b) document the QemuOpts schema for -acpitable, -smbios, -netdev,
>>>> -net. These options validate the options with OptsVisitor, so we could
>>>> do without QemuOpts schema, but we know the schema won't bitrot
>>>> because we never remove suboptions.
>>>
>>> -device?
>>
>> -device already provides its own introspection via "qom-list-types"
>> and "-device driver,?".
>
> We're discussing introspection via QMP, so "-device driver,help" doesn't
> count.
>
> qom-list-types with argument "implements": "device" together with
> device-list-properties indeed lets you introspect device models and
> their properties.  You then need to to know how to translate that to
> the command line.  For instance, you need to know that "type": "on/off"
> in the former means "type": "boolean" in the latter, and so forth.

Yes.  That's what I actually meant.  O:-)

> What I'd like to see is more unified introspection, not this smattering
> of one-offs hacked up without too much thought to serve some immediate
> need we have now.
>
> We already have something that aspires to be our unified interface
> description: QAPI schemata.  Perhaps we should make it our unified
> introspection mechanism while we're at it.

QOM introspection can use QAPI schemata.  Property types should be one of:

* child<ClassName>

* link<ClassName>

* QAPI scalar type

* QAPI compound type

I see QOM introspection as orthogonal to QAPI introspection, and 
QemuOpts introspection as complementary to both:

        command line introspection        QMP introspection
          |                    |            |        |
          |                    v            |        |
          v           QOM introspection  <--'        |
       QemuOpts                |                     |
     introspection             v                     |
                     QAPI introspection  <-----------'

Makes any sense? :)

>>>> (b) documenting the schemata is not harder than what Amos proposed.
>>>>
>>>> (c) schema inspection for objects remains a problem, but one that we
>>>> need to solve anyway so it doesn't affect query-command-line-options.
>>>
>>> As long as we don't have such schema inspection, I'm rather reluctant to
>>> reject alternative means to solve problems people have *now*.
>>
>> Note the "doesn't affect query-command-line-options" part.  Amos's
>> patch do not solve the problem of which classes can be instantiated
>> with -object, or of which properties can be used.
>
> Possible misunderstanding on my part.  I was afraid you were arguing to
> solve -object introspection *instead* of Amos's incremental improvement,
> but that doesn't seem to be the case.

Well, I was arguing against this series.  I think it provides little 
benefit and has a comparatively high cost in terms of future backwards 
compatibility.

I was arguing for ignoring non-QemuOpts options and focus on what really 
matters, which is QemuOpts, QOM and QAPI introspection.  The first is 
already there and can be fixed by adding schemata for -acpitable and 
friends.  QAPI introspection is already being tackled by Amos.

QOM/-object is indeed the elephant in the room, but luckily we have 
enough few users that I believe we can do it if we agree on the right 
design.

Paolo

Re: [Qemu-devel] fix/re-do query-command-line-options

[Markus Armbruster]

Paolo Bonzini <pbonzini@redhat.com> writes:

> Il 28/01/2014 14:16, Markus Armbruster ha scritto:
>>>> That would mean we can't ever add an option that doesn't take an
>>>> argument again.
>>>
>>> We can add it under an existing QemuOpts group or invent a new one
>>> (like we did for -rt or -msg).
>>
>> Do you mean -rtc?
>
> I meant -rt, but -rtc applies just as well. :)

Can't find -rt in output of -help.

>> -msg takes a timestamp=on/off argument.  I guess that doesn't feel too
>> forced, because we could conceivably have more settings related to error
>> reporting.
>>
>> Do you mean to suggest new options should always be done in a way that
>> makes them fit into QemuOpts?
>
> Yes.  Not into *existing* QemuOpts of course.

I meant QemuOpts-the-mechanism, so no misunderstanding here.

>> How would you add something like -S, -nodefaults, or -daemonize?
>
> I would add -nodefaults and -S to -machine.  I would deprecate
> -daemonize.  But that's just for the sake of -readconfig.  Consumers
> of introspection can just "know" that those options are there.

I'd be fine with a patch that makes all existing options that don't take
a QemuOpts argument sugar for options that do.

I'd rather not promise "all options will take an argument, or desugar
into some option that takes an argument, forever and ever".  But I don't
see the need for such a promise.  If we decide we want such options, we
just have to do the necessary work to make them visible in
introspection.

>> The question is whether we should extend QemuOpts to cover options
>> without arguments, or change the options without arguments to fit into
>> existing QemuOpts, e.g. by making them all syntactic sugar for a
>> suitable QemuOpts-style option.
>>
>> If we do the latter, we need to tell customers of command line
>> introspection to look only for the desugared forms.
>
> Yeah.

Documentation job.

However, desugaring the existing options could well be more work than
extending command line introspection and -readconfig/-writeconfig to
cover them.  Dunno.

>>>>> (b) document the QemuOpts schema for -acpitable, -smbios, -netdev,
>>>>> -net. These options validate the options with OptsVisitor, so we could
>>>>> do without QemuOpts schema, but we know the schema won't bitrot
>>>>> because we never remove suboptions.
>>>>
>>>> -device?
>>>
>>> -device already provides its own introspection via "qom-list-types"
>>> and "-device driver,?".
>>
>> We're discussing introspection via QMP, so "-device driver,help" doesn't
>> count.
>>
>> qom-list-types with argument "implements": "device" together with
>> device-list-properties indeed lets you introspect device models and
>> their properties.  You then need to to know how to translate that to
>> the command line.  For instance, you need to know that "type": "on/off"
>> in the former means "type": "boolean" in the latter, and so forth.
>
> Yes.  That's what I actually meant.  O:-)
>
>> What I'd like to see is more unified introspection, not this smattering
>> of one-offs hacked up without too much thought to serve some immediate
>> need we have now.
>>
>> We already have something that aspires to be our unified interface
>> description: QAPI schemata.  Perhaps we should make it our unified
>> introspection mechanism while we're at it.
>
> QOM introspection can use QAPI schemata.  Property types should be one of:
>
> * child<ClassName>
>
> * link<ClassName>
>
> * QAPI scalar type
>
> * QAPI compound type
>
> I see QOM introspection as orthogonal to QAPI introspection, and
> QemuOpts introspection as complementary to both:
>
>        command line introspection        QMP introspection
>          |                    |            |        |
>          |                    v            |        |
>          v           QOM introspection  <--'        |
>       QemuOpts                |                     |
>     introspection             v                     |
>                     QAPI introspection  <-----------'
>
> Makes any sense? :)

What do your arrows mean?

Is QemuOpts separate because it needs to be, or just because it exists,
and nobody can be bothered to fold it into one of the other things?

>>>>> (b) documenting the schemata is not harder than what Amos proposed.
>>>>>
>>>>> (c) schema inspection for objects remains a problem, but one that we
>>>>> need to solve anyway so it doesn't affect query-command-line-options.
>>>>
>>>> As long as we don't have such schema inspection, I'm rather reluctant to
>>>> reject alternative means to solve problems people have *now*.
>>>
>>> Note the "doesn't affect query-command-line-options" part.  Amos's
>>> patch do not solve the problem of which classes can be instantiated
>>> with -object, or of which properties can be used.
>>
>> Possible misunderstanding on my part.  I was afraid you were arguing to
>> solve -object introspection *instead* of Amos's incremental improvement,
>> but that doesn't seem to be the case.
>
> Well, I was arguing against this series.  I think it provides little
> benefit and has a comparatively high cost in terms of future backwards
> compatibility.

I'm can't yet see how making options without arguments visible in
query-command-line-options increases our compatibility liability, but
maybe you can enlighten me :)

Here's CommandLineOptionInfo's schema:

    ##
    # @CommandLineOptionInfo:
    #
    # Details about a command line option, including its list of parameter details
    #
    # @option: option name
    #
    # @parameters: an array of @CommandLineParameterInfo
    #
    # Since 1.5
    ##
    { 'type': 'CommandLineOptionInfo',
      'data': { 'option': 'str', 'parameters': ['CommandLineParameterInfo'] } }

Example: description of -msg, which takes a QemuOpts argument whose only
key is "timestamp":

    {"option": "msg"}, "parameters": [{"name": "timestamp", "type": "boolean"}]}

Example: description of -smbios, which takes a QemuOpts argument that
accepts any key=value:

    {"option": "smbios", "parameters": []}

Not the order we put them on the wire; I took the liberty to use a more
readable one.

An option without an argument fits right in, like this:

    {"option": "enable-fips"}

Less weird than the "parameters": [] we use for "anything goes".

> I was arguing for ignoring non-QemuOpts options and focus on what
> really matters, which is QemuOpts, QOM and QAPI introspection.

Point taken.

Can we really ignore non-QemuOpts, or do we have to make them sugar for
QemuOpts so they're visible in QemuOpts introspection?

>                                                                 The
> first is already there and can be fixed by adding schemata for
> -acpitable and friends.  QAPI introspection is already being tackled
> by Amos.
>
> QOM/-object is indeed the elephant in the room, but luckily we have
> enough few users that I believe we can do it if we agree on the right
> design.

Both QAPI introspection and QOM introspection are obviously valuable
jobs.

My questions are what is to be done to plug the gaps in command line
introspection short term, and how command line introspection should work
long term.

Re: [Qemu-devel] fix/re-do query-command-line-options

[Paolo Bonzini]

Il 28/01/2014 15:28, Markus Armbruster ha scritto:
> I'd rather not promise "all options will take an argument, or desugar
> into some option that takes an argument, forever and ever".  But I don't
> see the need for such a promise.  If we decide we want such options, we
> just have to do the necessary work to make them visible in
> introspection.

Agreed.

>>> The question is whether we should extend QemuOpts to cover options
>>> without arguments, or change the options without arguments to fit into
>>> existing QemuOpts, e.g. by making them all syntactic sugar for a
>>> suitable QemuOpts-style option.
>>>
>>> If we do the latter, we need to tell customers of command line
>>> introspection to look only for the desugared forms.
>>
>> Yeah.
>
> Documentation job.
>
> However, desugaring the existing options could well be more work than
> extending command line introspection and -readconfig/-writeconfig to
> cover them.  Dunno.

Definitely.  For example, -watchdog-action could become a -device 
option, which is nontrivial to implement.

"git log -p -SDEF\( -- qemu-options.hx" lets you find commits that added 
options.  The last non-QemuOpts option we added was -sandbox, in 1.2.0, 
shortly preceded by -no-user-config and -qtest.  So we can also tell 
customers of command line introspection "this is the set of non-sugar 
non-QemuOpts options as of 1.2".

>>        command line introspection        QMP introspection
>>          |                    |            |        |
>>          |                    v            |        |
>>          v           QOM introspection  <--'        |
>>       QemuOpts                |                     |
>>     introspection             v                     |
>>                     QAPI introspection  <-----------'
>>
>> Makes any sense? :)
>
> What do your arrows mean?

Nothing specific :)  Something like "requires".

> Is QemuOpts separate because it needs to be, or just because it exists,
> and nobody can be bothered to fold it into one of the other things?

There is some difference between QemuOpts and the others, in that it's 
out of necessity a human-oriented interface---or at least a "more" 
human-oriented interface.  So the optimistic me says it needs to be. 
But I'm not 100% sure.

> I'm can't yet see how making options without arguments visible in
> query-command-line-options increases our compatibility liability, but
> maybe you can enlighten me :)

Maybe there's none.

> Example: description of -smbios, which takes a QemuOpts argument that
> accepts any key=value:
>
>     {"option": "smbios", "parameters": []}
>
> Not the order we put them on the wire; I took the liberty to use a more
> readable one.
>
> An option without an argument fits right in, like this:
>
>     {"option": "enable-fips"}

Do we know for sure that libvirt or anyone else won't hiccup on the 
missing parameters?

> Can we really ignore non-QemuOpts, or do we have to make them sugar for
> QemuOpts so they're visible in QemuOpts introspection?

See above for my answer.

>> QOM/-object is indeed the elephant in the room, but luckily we have
>> enough few users that I believe we can do it if we agree on the right
>> design.
>
> Both QAPI introspection and QOM introspection are obviously valuable
> jobs.
>
> My questions are what is to be done to plug the gaps in command line
> introspection short term, and how command line introspection should work
> long term.

Short term, we definitely have to do something about the mismatches 
between option name and QemuOpts group name (boot-opts/smp-opts, resp. 
1.0 and 1.6.0).  And we have to document all the suboptions in the 
QemuOpts schema, with the exceptions of -device, -object (both to be 
satisfied with the QOM schema) and -blockdev (QAPI schema).  And we have 
to provide a way for libvirt to know that query-command-line-options is 
reliable.

Long term, I'm not sure we need more introspection than what we have. 
But regarding the command line, it would be definitely nice to move more 
stuff to QemuOpts, thus making -readconfig more complete.

Paolo