Re: [PATCH v4 00/11]

["Daniel P. Berrangé" <berrange@redhat.com>]

On Fri, Feb 14, 2025 at 09:29:33PM +0100, Victor Toso wrote:
> Hi again,
> 
> This patch series intent is to introduce a generator that produces a Go
> module for Go applications to interact over QMP with QEMU.
> 
> Previous version (10 Jan 2025)
>     https://lists.gnu.org/archive/html/qemu-devel/2025-01/msg01530.html
> 
> The generated code was mostly tested using existing examples in the QAPI
> documentation, 192 instances that might have multiple QMP messages each.

snip

> 1. Daniel wrote a demo on top of v3 and proposed changes that would
>    result in more interesting module to build on top:
>    https://lists.gnu.org/archive/html/qemu-devel/2025-01/msg03052.html
>    
>    I've implemented all the suggestions that are relevant for this
>    introductory series, they are:
> 
>   a. New struct type Message, that shall be used for a 1st level
>      unmarshalling of the JSON message.
>   b. Removal of Marshal/Unmarshal code in both Events and Comands,
>      together with utility code that is not relevant anymore.
>   c. Declaration of 3 new interfaces:
>     i.   Events
>     ii.  Commands
>     iii. CommandsAsync
> 
> 2. I've moved the code to a new folder: scripts/qapi/golang. This
>    allowed me to move templates out of golang.py, keeping go related
>    code self-contained in the new directory.

When I think about the code generator and how this will all
evolve over time, I have a strong feeling that none of this
should be in qemu.git.

Taking those three interfaces in (1)(c), when we come to actually
generate implementations of them, the generated code is going to
be intimately tied to the client/server API framework we need to
plumb into.

IMHO qemu.git should not have any knowledge of this, as it will
create a bidirectional dependency betweeen qemu.git and the
qemu-go.git repo, requiring them to evolve in lockstep.

We'll need 3 releases of the Go code per year, to match the
introduction the new QAPI schema versions, but outside that I
think there ought to be the freedom to have other Go releases
independently.

The need for this to be part of qemu.git is fairly narrow. It
provides a new hook into the QAPI generator code, and the QAPI
generator is not installed by 'make install', it is in-tree
only.

Can we solve this linkage ? We would need to be able to:

 * Have a mechanism for the QAPI code generator to load
   new genrator backends
 * Have a mechanism to tell the QAPI code generator to
   only run a single backend.
 * Have a mechanism to consume the QAPI code genrfator
   from outside qemu.git

The first point there could be addressable using the python "entry-points"
concept

  https://packaging.python.org/en/latest/specifications/entry-points/
  https://setuptools.pypa.io/en/latest/pkg_resources.html#entry-points

Or, alternatively by passing the name of a python module on the CLI.

The second point is just a bit of glue code to wire up a new CLI arg

The third point either involves having 'make install' put the QAPI
code generator into /usr/share/qemu, or /usr/lib/python3.x. The
latter would be if QAPI code generator were a formally release python
module on pypi. The former would be if we're just trying a QAPI code
generator as a local tool, not for pypi. A fallback would be to
consume qemu.git as a git submodule from qapi-go.git. The gitmodule
commit would not have to match the version of the QAPI schema we are
generating code for.

The least effort approach would be making QAPI code genrator accept
a python module name, and call it via a git submodule. This would
avoid declaring a long term support policy for the QAPI code gen
python APIs as a blocker.

> 
> 3. As mentioned in (2), created protocol.go and utils.go that are 100%
>    hand generated Go code. Message mentioned in (1a) is under
>    protocol.go
> 
> 4. Defined license using SPDX-License-Identifier.
>   a. Every Go source code written by hand is 100% MIT-0
>   b. Every Go source code generated is dual licensed as MIT-0 and
>      GPL-2.0-or-later
>   c. The binary code is expected to be MIT-0 only but not really
>      relevant for this series.
> 
>   If you want more information, please check the thread:
>   https://lists.gnu.org/archive/html/qemu-devel/2024-11/msg01621.html
> 
> 5. I've renamed the generated files.
>   a. Any type related file is now prefixed with "gen_type_"
>   b. Any interface related file is prefixed as "gen_iface_"
> 
> 6. Relevant changes were made to the doc but it is not complete. I plan
>    that follow-up proposals would add to the documentation.
> 
> 7. Improvements to the generator were made to.
> 
> 8. Also worth to mention that resulting generated code does not have any
>    diff with gofmt and goimport tools, as requested in the past.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|