Re: [PATCH v3 00/15] python: drop qemu.qmp from qemu.git tree

["Daniel P. Berrangé" <berrange@redhat.com>]

On Fri, Dec 05, 2025 at 01:00:42AM -0500, John Snow wrote:
> Hello!
> 
> This series drops the in-tree version of our python-qemu-qmp package
> ("qemu.qmp") in favor of the version hosted on PyPI, whose repository is
> located at https://gitlab.com/qemu-project/python-qemu-qmp.
> 
> GitLab CI: https://gitlab.com/jsnow/qemu/-/pipelines/2197613036
>        (FreeBSD isn't my fault...!)
> 
> The major effects of this patch series are:
> 
> 1. qemu.qmp will be installed from PyPI or vendored packages instead of
>    being utilized directly from the qemu.git tree.

This is not getting installed in enough scenarios IMHO.

My current workflow is commonly

  $ ./configure --target-list=x86_64-softmmu
  $ make
  $ ./scripts/qmp/qmp-shell-wrap 
   /var/home/berrange/src/virt/qemu/build
  Traceback (most recent call last):
    File "/var/home/berrange/src/virt/qemu/scripts/qmp/qmp-shell-wrap", line 7, in <module>
      from qemu.qmp import qmp_shell
  ModuleNotFoundError: No module named 'qemu.qmp'

Even if I add in a call to 'source build/pyvenv/bin/activate'
after 'make', I'm still missing the qemu.qmp python module.

AFAICT, qemu.qmp only gets installed in the venv if you run
'make check', and IMHO that should not be expected for the
above usage scenario.

Likewise I can no longer run any of the test programs directly
without first having run 'make check'. eg what I would currently
do is:

   $ ./configure --target-list=x86_64-softmmu
   $ make
   $ cd build/tests/qemu-iotests
   $ ./check 300

I don't generally run 'make check' as frequently as I
'configure && make' as it adds alot of time which is
not needed usually.

In general I find it tedious having to remember to run
'source build/pyvenv/bin/activate' in every terminal
window where I might want to run a QEMU script that relies
on python. It is also fairly undesirable too, as I use the
same terminal window for various purposes, that activating
the QEMU venv can interfere with non-QEMU python usage.

I would find it helpful if we provided a "run" script in the
root dir of the build directory that contained the following

  $ cat build/run
  #!/bin/sh

  set -e

  HERE=$(realpath $(dirname $0))
  source $HERE/pyvenv/bin/activate
  exec "$@"

Which would be used as a wrapper to execute scripts with the
right environment. That would let us keep the simpler workflow,
and avoid polluting the global terminal environment with the
qemu venv.

eg to be used as

   $ ./configure --target-list=x86_64-softmmu
   $ make
   $ cd build/tests/qemu-iotests
   $ ../../run ./check 300

or

  $ ./configure --target-list=x86_64-softmmu
  $ make
  $ ./build/run ./scripts/qmp/qmp-shell-wrap 



> 2. There are no new python dependencies checked or installed during
>    configure.  All test dependencies are installed post-hoc on an
>    as-needed basis.
> 3. "make check-venv" is no longer required to be run manually before any
>    test that is integrated with meson; this includes "make check" and
>    "make check-functional".
> 4. "make check-venv" no longer installs functional test dependencies: it
>    installs only the core suite of python test dependencies.
> 5. "make check-venv" is now required as a pre-requisite for running
>    device-crash-test and manually executed iotests.
> 6. Unfortunately, python3-wheel and python3-setuptools are now required
>    on the host system if tests are to be executed and >= Python 3.13 is
>    used.
> 7. An awful lot of deleted lines of code, and a lot fewer headaches
>    managing two nearly-identical copies of this source code. O:-)
> 
> Patches 1-5 are build system focused; they set up new pythondeps.toml,
> mkvenv, and meson systems in preparation for relying on an external
> qemu.qmp library, but does not yet make the switch.
> 
> Patches 6-9 are testing and CI focused; they add necessary preparation
> steps to keep tests running happily once the in-tree qemu.qmp library is
> removed.
> 
> Patches 10-15 are build system focused again; they implement the actual
> switchover to the external qemu.qmp library.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|