Re: [PATCH preview 0/3] reviving minimal QAPI generation from 2021

[Markus Armbruster <armbru@redhat.com>]

Paolo Bonzini <pbonzini@redhat.com> writes:

> Il mer 18 giu 2025, 16:25 Markus Armbruster <armbru@redhat.com> ha scritto:
>
>> I don't know enough about Rust/serde to give advice.  I do know how to
>> make a fool of myself by asking dumb questions.
>>
>
> No dumb questions, only dumb answers.
>
>> For commands this is not a problem because the real underlying
>> > transformation is QObject->QObject and the intermediate steps (to and
>> > from QObject) can use serde.
>>
>> Are you talking about commands implemented in Rust?
>>
>
> Yes. I will intersperse your text with the corresponding Rust/serde
> implementation.
>
>> The existing data flow is roughly like this (I'm simplifying):
>>
>> 1. Parse JSON text into request QObject, pass to QMP core
>>
>> 2. Extract command name string and argument QDict
>>
>> 3. Look up generated command marshaller / unmarshaller, pass argument
>>    QDict to it
>>
>
> Same so far since this is C code.
>
>> 4. Unmarshall argument QDict with the QObject input visitor and
>>    generated visit_type_ARG()
>
> Unmarshall with QObject Deserializer, which talks to a serde-generated
> Deserialize implementation.
>
>> 5. Pass the C arguments to the handwritten command handler, receive the
>>    C return value
>>
>
> Same.
>
>> 6. Marshall the return value into a QObject with the QObject output
>>    visitor and generated visit_type_RET(), return it to QMP core
>>
>
> Marshall with the QObject Serializer, which talks to a serde-generated
> Serialize implementation.
>
>> 7. Insert it into a response QObject
>>
>> 8. Unparse response QObject into JSON text
>>
>
> Same.
>>
>> How would a Serde flow look like?
>>
>
> As described above, visitors are bypassed and the marshalling/unmarshalling
> works directly at the QObject level.

We use the same code (handwritten QMP core) up to QObject and from
QObject on.

In between, we use QObject input/output visitors with generated C data
types for commands implemented in C, and Serde-generated
deserializers/serializers with generated Rust data types for commands
implemented in Rust.

Correct?

> Implementation-wise the main difference is that 1) the type code
> (Serialize/Deserialize) is not the same for serialization and
> desetialization, unlike visit_type_*() 2) the code generation is done by
> serde instead of qapi-gen and we'd be mostly oblivious to how it works.
>
> The Serializer and Deserializer should be about 1500 lines of Rust + tests,
> and they would do the functionality of the QObject input and output
> visitors.

We end up with two separate ways to do the same job, which is kind of
sad.

I gather the alternative would be to use generated QAPI visitors for the
generated Rust data types instead of Serde.  This would be unusual for
Rust code: Serde exists and works, so reinvent it would be wasteful and
dumb.

Gut feeling: sticking to how things are usually done (here: with Serde)
is the more prudent choice.

>> > However, QOM property getters/setters (especially, but not
>> > exclusively, for properties with compound types) remain a problem
>> > since these use callbacks with a Visitor* argument.
>>
>> object_property_set() takes the new property value wrapped in an input
>> visitor.  The property setter extracts it using visit_type_FOOs() with
>> this input visitor as it sees fit.  Ideally, it uses exactly
>> visit_type_PROPTYPE().
>>
>> object_property_get() takes an output visitor to be wrapped it around
>> the property value.  The property getter inserts it using
>> visit_type_FOOs() with this output visitor as it sees fit.  Ideally, it
>> uses exactly visit_type_PROPTYPE().
>>
>> We sometimes use a QObject input / output visitor, and sometimes a
>> string input / output visitor.  The latter come with restrictions, and
>> are evolutionary dead ends.
>>
>> The QObject visitors wrap a QObject, the string visitors wrap a string
>> (d'oh).
>>
>
> Yep. The string visitor is why we cannot just change getters and setters to
> use QObject.

The string visitors have long been a thorn in my side.

I wish QOM wasn't so annoyingly flexible.  I wish a property had to be
of QAPI type (possibly complex).  Less headaches.  Less boilerplate,
too.

Almost all QOM properties are.  And the few that aren't feel like bad
ideas to me.

> In this case, without writing a visit_type_*() implementation that can
> write to a Rust struct, an intermediate QObject would be the only way to
> turn a Visitor into a Rust data type. So I can imagine three ways to
> operate:
>
> * Keep using serde like for commands: in the callback that is invoked by
> object_property_set() do Visitor->QObject->setter (yes that means double
> conversion when the source visitor is and QObject visitor) or for the
> getter case, getter->QObject->Visitor. This has the minimum amount of code
> added to qapi-gen.

Recall the callback gets the new property value wrapped in an input
visitor.  Whereas a C setter extracts it into some C variable(s), a Rust
setter extracts it into a QObject, which it then passes to Serde for
deserialization into Rust variables.  Correct?

> * Generate a visit_type_*() implementation that emits a Rust struct (i.e.
> one that maps for example 'str' to a String and not a *mut c_char) and
> forgo serde completely. Use this generated implementation everywhere: QOM
> getters and setters, as well as QMP commands. This is how C code works.

This is the alternative mentioned above.

> * Generate rust->C (e.g. String->*mut c_char) and C->rust converters from
> qapi-gen; use the existing C visit_type_*() to extract data from visitors
> and then apply said converters to turn the data into a Rust struct, and
> likewise in the other direction. This was the way Marc-André's prototype
> worked.

Converters between C and Rust data types let us cut at the C data type
instead of at QObject.

We use the same code up to QAPI-generated C data type and from
QAPI-generated C data type on.

In between, we work directly with the C data type for properties
implemented in C, and convert to and from the Rust data type for
properties implemented in Rust.

Correct?

The simplest such converters convert via QObject.  Grossly inefficient
:)

Marc-André's prototype demonstrates efficient converters can be had.
The question is whether they're worth their keep.

>> I'm afraid this is too terse for ignorant me.
>>
>
> I tried to translate that. :)

Thank you!