Re: [PATCH] PoC: Rust binding for QAPI (qemu-ga only, for now)

[Markus Armbruster <armbru@redhat.com>]

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> Hi
>
> On Mon, Sep 21, 2020 at 1:16 PM Markus Armbruster <armbru@redhat.com> wrote:
>>
>> marcandre.lureau@redhat.com writes:
>>
>> > From: Marc-André Lureau <marcandre.lureau@redhat.com>
>> >
>> > Hi,
>> >
>> > Among the QEMU developers, there is a desire to use Rust. (see previous
>> > thread from Stefan "Why QEMU should move from C to Rust", the rust-vmm
>> > related projects and other experiments).
>> >
>> > Thanks to our QAPI type system and the associate code generator, it is
>> > relatively straightforward to create Rust bindings for the generated C
>> > types (also called sys/ffi binding) and functions. (rust-bindgen could
>> > probably do a similar job, but it would probably bring other issues).
>> > This provides an important internal API already.
>> >
>> > Slightly more complicated is to expose a Rust API for those, and provide
>> > convenient conversions C<->Rust. Taking inspiration from glib-rs
>> > binding, I implemented a simplified version of the FromGlib/ToGlib
>> > traits, with simpler ownership model, sufficient for QAPI needs.
>> >
>> > The usage is relatively simple:
>> >
>> > - from_qemu_none(ptr: *const sys::P) -> T
>> >   Return a Rust type T for a const ffi pointer P.
>> >
>> > - from_qemu_full(ptr: *mut sys::P) -> T
>> >   Return a Rust type T for a ffi pointer P, taking ownership.
>> >
>> > - T::to_qemu_none() -> Stash<P>
>> >   Returns a borrowed ffi pointer P (using a Stash to destroy "glue"
>> >   storage data, if any).
>> >
>> > - T::to_qemu_full() -> P
>> >   Returns a ffi pointer P. (P resources are leaked/passed to C/ffi)
>> >
>> > With those traits, it's relatively easy to implement the QMP callbacks.
>> > With enough interest, we could eventually start rewriting QGA in
>> > Rust, as it is a simple service. See qga/qmp.rs for some examples.
>> > We could also try to tackle qemu itself.
>>
>> Up to here, you're talking about *internal* interfaces.  Correct?
>>
>> Your motivation is enabling use of Rust in QEMU.  Correct?
>
> That's the first motivation, indeed.

Sounds useful.

>> > Finally, given that the QAPI types are easy to serialize, it was simple
>> > to use "serde" on them, and provide a D-Bus interface for QMP with zbus.
>> > (a similar approach could probably be taken for other protocols, that
>> > could be dynamically loaded... anyone like protobuf better?)
>>
>> QMP is an *external* interface.
>>
>> It supports compatible evolution: we can make certain kinds of changes
>> without affecting clients.  These include:
>>
>> * Adding optional arguments
>
> This would change the signature of the function, and would need an
> interface version bump.
>
> Alternative: pass optional arguments as an extra dictionary. This is a
> common idiom in D-Bus (the "a{sv}" type that maps strings to generic
> values)
>
> Potentially, use gvariant serialization format, which has maybe type.
> But gvariant isn't implemented by most D-Bus libraries (that was the
> plan long time ago, but it didn't happen as people lost interest).
>
>> * Adding results
>
> Also change the signature of the function.
>
> However, since messages have boundaries, it is easy to ignore return values.

I'm not sure I understand this.

The compatible change I have in mind is adding members to the complex
type returned by a command.

>> * Adding values to an enumeration type, branches to a union or
>>   alternate
>>
>
> As long as the discriminant is represented as a string, it should be fine.
>
>> * Reordering members of enumerations, structs, unions
>
> Again, if the discriminant is a string, it should be the same as with json.
>
> For the members, the usage of dictionaries is required in this case
> (else the type signature would change).
>
>> * Turning an argument type into an alternate with the old type as branch
>
> That would also change the function signature.
>
> There isn't much solution I can think of, unless we have an implicit
> tagged enum for every argument, which would be quite nasty.
>
>>
>> We've made use of this extensively.  See also
>> docs/devel/qapi-code-gen.txt section "Compatibility considerations."
>>
>> How do such changes affect clients of the proposed D-Bus interface?
>
> The introspection XML will always reflect the expected signature. You
> should bump your interface version whenever you make incompatible
> changes.

How do "interface versions" work?  Client's and server's version need to
match, or else no go?

> If this happens too often, we could also introduce a D-Bus override
> mechanism to do manual translations from external interface to
> internal.

Greek to me :)