Should we replace QAPI? (was: [PATCH v14 02/14] fixup! migration: New QAPI type 'MigrateAddress')

[Markus Armbruster <armbru@redhat.com>]

Daniel P. Berrangé <berrange@redhat.com> writes:

> On Fri, Oct 20, 2023 at 02:37:16PM +0200, Markus Armbruster wrote:
>> Fabiano Rosas <farosas@suse.de> writes:
>> 
>> > Markus Armbruster <armbru@redhat.com> writes:
>> >
>> >> Fabiano Rosas <farosas@suse.de> writes:
>> >>
>> >>> Signed-off-by: Fabiano Rosas <farosas@suse.de>
>> >
>> > Side question: are we using valid JSON at all? I threw this in a random
>> > online linter and it complains about the single quotes. We could have a
>> > proper tool doing the validation in CI.
>> 
>> You've come a sad, sad place.
>> 
>> docs/devel/qapi-code-gen.rst:
>> 
>>     Schema syntax
>>     -------------
>> 
>>     Syntax is loosely based on `JSON <http://www.ietf.org/rfc/rfc8259.txt>`_.
>>     Differences:
>> 
>>     * Comments: start with a hash character (``#``) that is not part of a
>>       string, and extend to the end of the line.
>> 
>>     * Strings are enclosed in ``'single quotes'``, not ``"double quotes"``.
>> 
>>     * Strings are restricted to printable ASCII, and escape sequences to
>>       just ``\\``.
>> 
>>     * Numbers and ``null`` are not supported.
>> 
>> If your reaction to item 2 is "this is stupid", you'd be exactly right.
>> 
>> Here's the conclusion of a discussion on possible improvements we had in
>> 2020:
>> https://lore.kernel.org/qemu-devel/877dt5ofoi.fsf@dusky.pond.sub.org/
>
> Looking at those options again I so strongly want to be able to
> say "none of the above".
>
> We have a need to describe data structures, and generate code for
> serializing/deserializing them on the wire. We created a language
> for this and wrote our own C code generator, our own docs generator,
> own our json parser & formatter, and now are also writing our own
> Go code generator (Python, Rust too ?).
>
> IMHO this is way too much NiH for my likely, and creates a maint
> burden for ourselves that we could do without.
>
> <open-can-of-worms>
> At the point in time we invented QAPI this was perhaps justifiable,
> though even back then I thought we should have done a binary format
> and used XDR to describe it, as a standard pre-existing language and
> toolset.

Path dependence...

I wasn't involved in the decisions that led to QAPI, nor its initial
design and implementation.

Its design was substantially constrained by QMP, which predates QAPI by
almost two years.  I was involved in QMP's design for a bit.  We argued
back and forth, and I eventually stepped aside to let the guys doing the
actual work make the decisions.

QAPI was further constrained by the desire to use it with QOM.  The two
are joined at the hip by visitors.  Nevertheless, QOM and QAPI are
integrated poorly.

My general attitude towards generating source code is "don't; use a more
powerful programming language".  Instead of creating yet another pillar
supporting Greenspun's Tenth Rule, I feel we should have embedded a
sufficiently powerful programming language, and used it for the control
plane.  My choice would've been Lisp.

The maintenance burden is modest, but real.  The QAPI generator has
gotten ~1.6 patches per week for the last five years, trending down.  In
the last year, it's been 68 patches, 437 insertions, 338 deletions
total, much of it in docs/devel/qapi-code-gen.rst.  Meanwhile, the
entire project has had 130 times as many patches, 620 times as many
insertions, and 400 times as many deletions.

QAPI infrastructure maintenance is dwarved several times over by QAPI
schema maintenance.  Chiefly patch review.

To be fair, adding language bindings will take a non-trivial one time
investment plus ongoing maintenance for each language.

> Today I wouldn't suggest XDR, but would be inclined to suggest
> Protobuf. This already has code & docs generators for every
> programming language we would ever need.
>
> Protobuf has a defined serialization format too though which is
> binary based IIUC, so distinct from our JSON wire format.
>
> The interesting question though is whether it would be feasible to
> transition to this today, despite our historical baggage ? With
> recent interest in accessing QAPI for many more languages, it

Go, Rust, and what else?

> is timely to consider if we can adopt a standardized toolset to
> reduce this burden of talking to QEMU from other languages.
>
> A transition would need several big ticket items
>
>  * A QAPI visitor impl that can spit out the protobuf document.
>    ie to translate all our existing QAPI formats into protobuf,
>    as doing this manually would be madness. This is probably
>    the easy bit.

This is about machine-assisted translation of the QAPI schema to
protobuf, isn't it?

>  * A custom protobuf visitor impl that can spit out C code
>    that has the same API as our QAPI C generator. ie so we can
>    avoid a big bang to convert all of QEMU internals. I suspect
>    this is quite alot of work.

The alternative is a big bang to convert, which could be less work or
more.

>  * A custom protobuf serializer compatible with our current
>    JSON format. ie so that QEMU can continue to expose the
>    current QMP protocol to apps for legacy compat for some
>    period of time, while also exposing a new native binary
>    protobuf protocol for future usage. Also probably quite
>    alot of work.

No alternative.

> That's all certainly quite alot of work and probably things I have
> forgotten.

QOM's use of QAPI might add problems.

>            Above all it does assume it is possible to define a
> loss-less mapping from QAPI -> Protobuf language. I've not proved
> this is possible, but my inclination is that it probably should be,
> and if not, we could likely enable it by strategically deprecated
> and then deleting troublesome bits prior to a conversion.
> </open-can-of-worms>

While I share your distaste for the massive NIH QAPI has become, I'm not
sure replacing it now is economical.  It requires a massive one-time
investment, offset by saving us one-time investments into QAPI bindings
for other languages of interest.  Whether it can actually reduce ongoing
maintenance appreciably after the replacement is unclear.

Pretty much the same for QOM.