qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v3 00/15] python: drop qemu.qmp from qemu.git tree
@ 2025-12-05  6:00 John Snow
  2025-12-05  6:00 ` [PATCH v3 01/15] python/mkvenv: create timestamp file for each group "ensured" John Snow
                   ` (16 more replies)
  0 siblings, 17 replies; 26+ messages in thread
From: John Snow @ 2025-12-05  6:00 UTC (permalink / raw)
  To: qemu-devel
  Cc: Cleber Rosa, Marc-André Lureau, Paolo Bonzini, Thomas Huth,
	Li-Wen Hsu, Kevin Wolf, qemu-block, Michael Roth, John Snow,
	Markus Armbruster, Hanna Reitz, Alex Bennée, Ed Maste,
	Daniel P. Berrangé, Philippe Mathieu-Daudé

Hello!

This series drops the in-tree version of our python-qemu-qmp package
("qemu.qmp") in favor of the version hosted on PyPI, whose repository is
located at https://gitlab.com/qemu-project/python-qemu-qmp.

GitLab CI: https://gitlab.com/jsnow/qemu/-/pipelines/2197613036
       (FreeBSD isn't my fault...!)

The major effects of this patch series are:

1. qemu.qmp will be installed from PyPI or vendored packages instead of
   being utilized directly from the qemu.git tree.
2. There are no new python dependencies checked or installed during
   configure.  All test dependencies are installed post-hoc on an
   as-needed basis.
3. "make check-venv" is no longer required to be run manually before any
   test that is integrated with meson; this includes "make check" and
   "make check-functional".
4. "make check-venv" no longer installs functional test dependencies: it
   installs only the core suite of python test dependencies.
5. "make check-venv" is now required as a pre-requisite for running
   device-crash-test and manually executed iotests.
6. Unfortunately, python3-wheel and python3-setuptools are now required
   on the host system if tests are to be executed and >= Python 3.13 is
   used.
7. An awful lot of deleted lines of code, and a lot fewer headaches
   managing two nearly-identical copies of this source code. O:-)

Patches 1-5 are build system focused; they set up new pythondeps.toml,
mkvenv, and meson systems in preparation for relying on an external
qemu.qmp library, but does not yet make the switch.

Patches 6-9 are testing and CI focused; they add necessary preparation
steps to keep tests running happily once the in-tree qemu.qmp library is
removed.

Patches 10-15 are build system focused again; they implement the actual
switchover to the external qemu.qmp library.

v3:

 - Fixed FreeBSD vm test
 - Fixed Cirrus macOS test (Needed setuptools as a dep, "file:///" had
   one too many forward slashes, which works fine for older
   pip/setuptools, but was incompatibly fixed recently.)
 - Fixed an issue that prevented installing the local python packages in
   an offline, isolated build environment (need --no-isolated-build and
   pip environment variable flags)
 - Added the python3-wheel and python3-setuptools build dependencies to
   lcitool, owing to the above issues
 - Ensured that the $download flag (for PyPI access) from configure is
   carried over into pytest/meson.build invocations.  This defaults to
   "enabled", so test dependencies should not have any issue being
   fetched on most developer workstations and build environments.
 - Fixed several bugs in the mkvenv.py local package installation mechanism.
 - Flags and workarounds added for installing local packages offline now
   apply *only* to local package installations, and not "normal"
   depspecs.
 - Reordered patches into more logical "phases" for easier review (I
   hope...?)

Known issues: "make check-build" still installs all of the test
dependencies to the pyvenv. I'm not sure if this is desired or not, but
I ran into problems preventing it from doing this without breaking a ton
of CI tests. I'll continue to investigate this, but wanted to send this
v3 out.

v2:

 - move "make check-venv" earlier in GitLab CI/CD pipeline, to avoid
   re-running configure
 - Fix functional tests not using PyPI to fetch pygdbmi package
 - Remove pre-requisites which are now merged

--js

John Snow (15):
  python/mkvenv: create timestamp file for each group "ensured"
  python/mkvenv: bump 'qemu.qmp' dependency for testdeps
  python/mkvenv: add 'checktests' and 'functests' dependency groups
  python/mkvenv: add mechanism to install local package(s)
  meson, mkvenv: add checktests and functests custom targets
  tests: Use configured python to run GitLab iotests
  tests: run "make check-venv" before running iotests
  tests: ensure "make check-venv" is run for crash tests
  tests/lcitool: add python3 wheel and setuptools deps for qemu
  python: add vendored qemu.qmp package
  meson, mkvenv: make iotests depend on checktests group
  meson, mkvenv: make functional tests depend on functests group
  meson, mkvenv: add qemu.git/python/qemu package to pythondeps.toml
  tests: replace old "check-venv" target with meson target
  python: delete qemu.qmp

 meson.build                                   |    1 +
 .gitlab-ci.d/buildtest.yml                    |   11 +-
 .gitlab-ci.d/cirrus/freebsd-14.vars           |    2 +-
 .gitlab-ci.d/cirrus/macos-14.vars             |    2 +-
 python/qemu/qmp/__init__.py                   |   60 -
 python/qemu/qmp/error.py                      |   53 -
 python/qemu/qmp/events.py                     |  751 -----------
 python/qemu/qmp/legacy.py                     |  339 -----
 python/qemu/qmp/message.py                    |  217 ----
 python/qemu/qmp/models.py                     |  146 ---
 python/qemu/qmp/protocol.py                   | 1101 -----------------
 python/qemu/qmp/py.typed                      |    0
 python/qemu/qmp/qmp_client.py                 |  732 -----------
 python/qemu/qmp/qmp_shell.py                  |  689 -----------
 python/qemu/qmp/qmp_tui.py                    |  665 ----------
 python/qemu/qmp/util.py                       |  150 ---
 python/qemu/utils/qom_fuse.py                 |    1 -
 python/scripts/mkvenv.py                      |   44 +-
 python/scripts/vendor.py                      |    2 +
 python/setup.cfg                              |   31 +-
 python/tests/minreqs.txt                      |    8 +-
 python/tests/protocol.py                      |  596 ---------
 python/wheels/qemu_qmp-0.0.5-py3-none-any.whl |  Bin 0 -> 72263 bytes
 pythondeps.toml                               |   13 +-
 pyvenv/meson.build                            |   40 +
 tests/Makefile.include                        |   25 +-
 tests/docker/dockerfiles/alpine.docker        |    2 +
 tests/docker/dockerfiles/centos9.docker       |    2 +
 tests/functional/meson.build                  |    7 +-
 tests/lcitool/projects/qemu.yml               |    2 +
 tests/qemu-iotests/meson.build                |    2 +-
 tests/vm/generated/freebsd.json               |    2 +
 32 files changed, 126 insertions(+), 5570 deletions(-)
 delete mode 100644 python/qemu/qmp/__init__.py
 delete mode 100644 python/qemu/qmp/error.py
 delete mode 100644 python/qemu/qmp/events.py
 delete mode 100644 python/qemu/qmp/legacy.py
 delete mode 100644 python/qemu/qmp/message.py
 delete mode 100644 python/qemu/qmp/models.py
 delete mode 100644 python/qemu/qmp/protocol.py
 delete mode 100644 python/qemu/qmp/py.typed
 delete mode 100644 python/qemu/qmp/qmp_client.py
 delete mode 100644 python/qemu/qmp/qmp_shell.py
 delete mode 100644 python/qemu/qmp/qmp_tui.py
 delete mode 100644 python/qemu/qmp/util.py
 delete mode 100644 python/tests/protocol.py
 create mode 100644 python/wheels/qemu_qmp-0.0.5-py3-none-any.whl
 create mode 100644 pyvenv/meson.build

-- 
2.51.1




^ permalink raw reply	[flat|nested] 26+ messages in thread

end of thread, other threads:[~2025-12-10 16:26 UTC | newest]

Thread overview: 26+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-12-05  6:00 [PATCH v3 00/15] python: drop qemu.qmp from qemu.git tree John Snow
2025-12-05  6:00 ` [PATCH v3 01/15] python/mkvenv: create timestamp file for each group "ensured" John Snow
2025-12-05  8:32   ` Thomas Huth
2025-12-05  6:00 ` [PATCH v3 02/15] python/mkvenv: bump 'qemu.qmp' dependency for testdeps John Snow
2025-12-05  6:00 ` [PATCH v3 03/15] python/mkvenv: add 'checktests' and 'functests' dependency groups John Snow
2025-12-05  8:47   ` Thomas Huth
2025-12-05  6:00 ` [PATCH v3 04/15] python/mkvenv: add mechanism to install local package(s) John Snow
2025-12-05  6:00 ` [PATCH v3 05/15] meson, mkvenv: add checktests and functests custom targets John Snow
2025-12-05  6:00 ` [PATCH v3 06/15] tests: Use configured python to run GitLab iotests John Snow
2025-12-05  6:00 ` [PATCH v3 07/15] tests: run "make check-venv" before running iotests John Snow
2025-12-05  6:00 ` [PATCH v3 08/15] tests: ensure "make check-venv" is run for crash tests John Snow
2025-12-05  8:53   ` Thomas Huth
2025-12-05  6:00 ` [PATCH v3 09/15] tests/lcitool: add python3 wheel and setuptools deps for qemu John Snow
2025-12-05  6:00 ` [PATCH v3 10/15] python: add vendored qemu.qmp package John Snow
2025-12-05  6:00 ` [PATCH v3 11/15] meson, mkvenv: make iotests depend on checktests group John Snow
2025-12-05  6:00 ` [PATCH v3 12/15] meson, mkvenv: make functional tests depend on functests group John Snow
2025-12-05  6:00 ` [PATCH v3 13/15] meson, mkvenv: add qemu.git/python/qemu package to pythondeps.toml John Snow
2025-12-05  6:00 ` [PATCH v3 14/15] tests: replace old "check-venv" target with meson target John Snow
2025-12-05  6:00 ` [PATCH v3 15/15] python: delete qemu.qmp John Snow
2025-12-09 16:35 ` [PATCH v3 00/15] python: drop qemu.qmp from qemu.git tree John Snow
2025-12-09 17:00 ` Daniel P. Berrangé
2025-12-09 17:23   ` John Snow
2025-12-09 19:43     ` John Snow
2025-12-09 19:46     ` Daniel P. Berrangé
2025-12-09 20:05       ` John Snow
2025-12-10 16:25         ` Daniel P. Berrangé

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).