[Qemu-devel] RFC: -object vs -chardev creation order

[Qemu-devel] RFC: -object vs -chardev creation order

[Daniel P. Berrange]

The current QEMU startup code will create -chardev backends first, then
create -object backends, then -fsdev backends and so on, in some pretty
arbitrary order of types.

There is already a dependancy from the rng-egd object type, which has a
link to a chardev, which requires -chardev options be processed before
-object options.

In my work to introduce an object type for TLS credentials though, we
encounter exactly the opposite dependancy. A -chardev option will
require that -object options be processed first, so that the TLS
credantials can be created.

This is a no-win situation. Both orders of creation are wrong for some
set of use cases. This is only going to get worse over time, as more
types of user creatable objects are defined.

The application invoking QEMU could easily solve it, by putting the
args on the command line in the order in which they need to processed,
but then QEMU throws away this ordering information and creates them
based on type of backend.

I'm unclear on whether anyone has thought about ways to solve this yet,
but it is a blocker for my work now, so I need to figure out some kind
of solution asap, even if only a temporary workaround.

One option is to convert the -chardev backends so they become user
creatable objects, and thus -object can obsolete use of -chardev
for these. This sounds simple, but I fear it could be a rather large
yak shaving exercise to get that done.

Another idea is to perhaps change the way options are stored, so that
instead of having a separate QemuOptsList for each type of option, have
one single list and store the type with each entry in the list. Then we
can iterate over the single list creating things in exactly the same
order that they were passed on the command line.  I fear this may cause
regressions for apps using QEMU though - eg if some app is placing
'-object rng-egd' in the args, before the -chardev it would currently
work, but if we switch to honour ordering it would fail. We could
introduce a flag -strict-ordering but that feels gross in itself.

A third option is to not process -object args in one go, instead process
each type of object at a time. eg we'd first create all the
-object tls-credential instances, then create the -chardev instances,
then create the -object rng-egd instances. This is probably the least
amount of work in short term, but not all that scalable, unless we do
a catch-all default case, so we only need code up hacks for a few
particular object types.

Thus my gut feeling is to do option 3, but I'd like other opinions before
embarking on this....

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] RFC: -object vs -chardev creation order

[Paolo Bonzini]

On 17/03/2015 20:08, Daniel P. Berrange wrote:
> A third option is to not process -object args in one go, instead process
> each type of object at a time. eg we'd first create all the
> -object tls-credential instances, then create the -chardev instances,
> then create the -object rng-egd instances. This is probably the least
> amount of work in short term, but not all that scalable, unless we do
> a catch-all default case, so we only need code up hacks for a few
> particular object types.
> 
> Thus my gut feeling is to do option 3, but I'd like other opinions before
> embarking on this....

Yes, for now it is the best.

Another idea is to build objects lazily.  This requires adding a
"missing property" interface and making /objects implement it.  We can
think about it later.

Paolo

Re: [Qemu-devel] RFC: -object vs -chardev creation order

[Markus Armbruster]

"Daniel P. Berrange" <berrange@redhat.com> writes:

> The current QEMU startup code will create -chardev backends first, then
> create -object backends, then -fsdev backends and so on, in some pretty
> arbitrary order of types.
>
> There is already a dependancy from the rng-egd object type, which has a
> link to a chardev, which requires -chardev options be processed before
> -object options.
>
> In my work to introduce an object type for TLS credentials though, we
> encounter exactly the opposite dependancy. A -chardev option will
> require that -object options be processed first, so that the TLS
> credantials can be created.
>
> This is a no-win situation. Both orders of creation are wrong for some
> set of use cases. This is only going to get worse over time, as more
> types of user creatable objects are defined.
>
> The application invoking QEMU could easily solve it, by putting the
> args on the command line in the order in which they need to processed,
> but then QEMU throws away this ordering information and creates them
> based on type of backend.

Yes, and that's stupid.

> I'm unclear on whether anyone has thought about ways to solve this yet,
> but it is a blocker for my work now, so I need to figure out some kind
> of solution asap, even if only a temporary workaround.
>
> One option is to convert the -chardev backends so they become user
> creatable objects, and thus -object can obsolete use of -chardev
> for these. This sounds simple, but I fear it could be a rather large
> yak shaving exercise to get that done.
>
> Another idea is to perhaps change the way options are stored, so that
> instead of having a separate QemuOptsList for each type of option, have
> one single list and store the type with each entry in the list. Then we
> can iterate over the single list creating things in exactly the same
> order that they were passed on the command line.

It's how we should've done it, but:

>                                                   I fear this may cause
> regressions for apps using QEMU though - eg if some app is placing
> '-object rng-egd' in the args, before the -chardev it would currently
> work, but if we switch to honour ordering it would fail. We could
> introduce a flag -strict-ordering but that feels gross in itself.

Yes.

I'm afraid we'll have to break our byzantine command line at some
point...  I'd rather not break it now, though.

> A third option is to not process -object args in one go, instead process
> each type of object at a time. eg we'd first create all the
> -object tls-credential instances, then create the -chardev instances,
> then create the -object rng-egd instances. This is probably the least
> amount of work in short term, but not all that scalable, unless we do
> a catch-all default case, so we only need code up hacks for a few
> particular object types.
>
> Thus my gut feeling is to do option 3, but I'd like other opinions before
> embarking on this....

Piling yet another hack on top isn't great, put telling you "no more
hacks" after everybody and his dog already got to pile on some doesn't
feel fair.

That said: can we sort the creation work topologically?  Requires
identifying references.  In the QemuOpts world, a reference is a
string-valued parameter that is interpreted as ID in a well-known
QemuOptsList.  Adding the necessary information to QemuOptDesc desc[]
shouldn't be too hard.  The ones with empty desc[] could be troublesome,
as usual.

Re: [Qemu-devel] RFC: -object vs -chardev creation order

[Daniel P. Berrange]

On Wed, Mar 18, 2015 at 10:43:25AM +0100, Markus Armbruster wrote:
> "Daniel P. Berrange" <berrange@redhat.com> writes:
> > A third option is to not process -object args in one go, instead process
> > each type of object at a time. eg we'd first create all the
> > -object tls-credential instances, then create the -chardev instances,
> > then create the -object rng-egd instances. This is probably the least
> > amount of work in short term, but not all that scalable, unless we do
> > a catch-all default case, so we only need code up hacks for a few
> > particular object types.
> >
> > Thus my gut feeling is to do option 3, but I'd like other opinions before
> > embarking on this....
> 
> Piling yet another hack on top isn't great, put telling you "no more
> hacks" after everybody and his dog already got to pile on some doesn't
> feel fair.

I think I'll do this right now, since it should be a fast fix.

> That said: can we sort the creation work topologically?  Requires
> identifying references.  In the QemuOpts world, a reference is a
> string-valued parameter that is interpreted as ID in a well-known
> QemuOptsList.  Adding the necessary information to QemuOptDesc desc[]
> shouldn't be too hard.  The ones with empty desc[] could be troublesome,
> as usual.

This sounds intriguing though - we should have enough info around to
do a topological sort - its a shame that properties are only registered
at time the object is created, rather than being registered against the
classes, as then we wouldn't even need to add more info to the
QemuOptsList.  I'll have a think about this approach though and see if
I can come up with something that works

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|