Re: [PATCH v3 0/8] qapi-go: add generator for Golang interfaces

["Daniel P. Berrangé" <berrange@redhat.com>]

On Mon, Jan 13, 2025 at 01:52:25PM +0100, Markus Armbruster wrote:
> Victor Toso <victortoso@redhat.com> writes:
> 
> > This patch series intent is to introduce a generator that produces a Go
> > module for Go applications to interact over QMP with QEMU.
> >
> > The initial Goal is to have a Go module that works as intended and can
> > be improved upon. I'd consider initial releases to be alpha while we
> > work with utilities tools and libraries on top of this.
> >
> > The generated code should reside in a separated Git repository, similar
> > to python-qemu-qmp.
> >
> > Applications should be able to consume this under qemu.org
> > namespace (e.g: import "qemu.org/go/qemu"), see Daniel's suggestion:
> > https://lists.gnu.org/archive/html/qemu-devel/2023-09/msg07024.html
> >
> > This is the third iteration:
> > v2: https://lists.gnu.org/archive/html/qemu-devel/2023-10/msg04785.html
> >
> > I've pushed this series in my gitlab fork:
> > https://gitlab.com/victortoso/qapi-go/
> >
> > The fork contains some tests, including tests that were generated from
> > QAPI's own examples from another generator created for testing, if you
> > are interested in it:
> > https://lists.gnu.org/archive/html/qemu-devel/2023-09/msg04946.html
> >
> > I've generated the qapi-go module over each commit of this series, see:
> > https://gitlab.com/victortoso/qapi-go/-/commits/qapi-golang-v3-by-patch
> >
> > I've also generated the qapi-go module over QEMU tags: v9.1.0, v9.2.0:
> > https://gitlab.com/victortoso/qapi-go/-/commits/qapi-golang-v3-by-tags
> >
> > --
> >
> > Sorry that its been awhile between v2 and v3, I had to prioritize other
> > things. I hope to get this back on track in 2025.
> >
> > Cheers,
> > Victor
> >
> > * Changes:
> >
> > On generated go:
> >  - the output should be formatted as gofmt/goimports tools (Daniel)
> >
> >  - Included QAPI's documentation too (Daniel), see:
> >    https://lists.gnu.org/archive/html/qemu-devel/2024-11/msg01621.html
> >     
> >  - Commands and Events should Marshal directly (Andrea)
> >
> > On python script:
> >  - rebased: now uses QAPISchemaBranches, QAPISchemaAlternatives
> >
> >  - use textwrap as much as possible (Andrea)
> >
> >  - lots of changes to make the output like gofmt does
> >
> > Victor Toso (8):
> >   qapi: golang: Generate enum type
> >   qapi: golang: Generate alternate types
> >   qapi: golang: Generate struct types
> >   qapi: golang: structs: Address nullable members
> >   qapi: golang: Generate union type
> >   qapi: golang: Generate event type
> >   qapi: golang: Generate command type
> >   docs: add notes on Golang code generator
> >
> >  docs/devel/index-build.rst          |    1 +
> >  docs/devel/qapi-golang-code-gen.rst |  548 +++++++++
> >  scripts/qapi/golang.py              | 1645 +++++++++++++++++++++++++++
> >  scripts/qapi/main.py                |    3 +
> >  4 files changed, 2197 insertions(+)
> >  create mode 100644 docs/devel/qapi-golang-code-gen.rst
> >  create mode 100644 scripts/qapi/golang.py
> 
> This is series adds a backend that slots in cleanly, i.e. without any
> changes to the core.  That makes it as low-risk to merge as it gets.
> 
> I'd like an Acked-by for the generated Go from someone familiar the kind
> of software that could use it.

In my other (huge) reply to this thread I tried to provide that
analysis by imagining how I would want to consume a QEMU API in
Go, creating an example application and trying to see how well
it fits with this design.

> The -* files are all one pair of files per module (the things pulled in
> with include directives), if any.  We do this to avoid "touch the QAPI
> schema, recompile the world."
> 
> The generated Go is monolithic.  No "recompile the world" problem with
> Go?

IIUC, the intent is that we dn't generate the go code as part of the
regular QEMU build. IOW, most of the time it would be generate once
against a release tag. If someone was actively working on QAPI schema
on git master, at the same time as developing a Go application, they'll
have a bit more of a repeated compile penalty. I'm not convinced that's
a big enough common case to worry about modularizing though.

> golang.py is somewhat big.  Whether splitting it up along the lines of
> the C backend would improve things I can't say.  No need to worry about
> that now.

With the monolothic go code, generated once per release tag, Go is
going to be reusing cached previously compiled objects most of the
time. I think that's likely to be good enough.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|