All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v4 00/17] python: drop qemu.qmp from qemu.git tree
@ 2026-01-19 21:27 John Snow
  2026-01-19 21:27 ` [PATCH v4 01/17] python/mkvenv: create timestamp file for each group "ensured" John Snow
                   ` (17 more replies)
  0 siblings, 18 replies; 40+ messages in thread
From: John Snow @ 2026-01-19 21:27 UTC (permalink / raw)
  To: qemu-devel
  Cc: Marc-André Lureau, Vladimir Sementsov-Ogievskiy, Kevin Wolf,
	Maksim Davydov, Daniel P. Berrangé, Li-Wen Hsu,
	Markus Armbruster, Thomas Huth, John Snow, Mauro Carvalho Chehab,
	Hanna Reitz, Cleber Rosa, Michael Roth, Yonggang Luo, Ed Maste,
	Peter Xu, Fabiano Rosas, Paolo Bonzini, qemu-block,
	Philippe Mathieu-Daudé, Alex Bennée

Hello!

This series drops the in-tree version of our python-qemu-qmp package
("qemu.qmp") in favor of the version hosted on PyPI, whose repository is
located at https://gitlab.com/qemu-project/python-qemu-qmp.

GitLab CI: https://gitlab.com/jsnow/qemu/-/pipelines/2272170300

This version reflects Dan's suggestion to always install the in-tree
Python packages by default.

The major effects of this patch series are:

1. qemu.qmp and the python/ directory will be installed to the
   configure-time venv (build/pyvenv) by default.
2. qemu.qmp will be installed from PyPI or vendored packages instead of
   being utilized directly from the qemu.git tree.
3. "make check-venv" is removed as it is no longer needed.
4. functional test dependencies are installed as-needed when running
   those tests.
5. Unfortunately, python3-wheel and python3-setuptools are now required
   on the host system if tests are to be executed and >= Python 3.13 is
   used.
6. An awful lot of deleted lines of code, and a lot fewer headaches
   managing two nearly-identical copies of this source code. O:-)

v4:

 - Rename "checktests" group to "tooling", install it by default
   alongside meson at configure-time.
 - Remove "check-venv" entirely.
 - Update python/README.rst to reflect the new paradigm
 - Update many scripts that import qemu.qmp to give new errors when they
   are run outside of the configure venv

KNOWN ISSUES IN V4:

 - The last patch is an RFC based on Thomas' observations, I don't have
   a strong opinion on if we want it or not.
 - Removing "make check-venv" causes the VM tests to rebuild more often;
   I was not sure how to replace the old conditional behavior in an
   equivalent manner. Suggestions, anyone...?

v3:

 - Fixed FreeBSD vm test
 - Fixed Cirrus macOS test (Needed setuptools as a dep, "file:///" had
   one too many forward slashes, which works fine for older
   pip/setuptools, but was incompatibly fixed recently.)
 - Fixed an issue that prevented installing the local python packages in
   an offline, isolated build environment (need --no-isolated-build and
   pip environment variable flags)
 - Added the python3-wheel and python3-setuptools build dependencies to
   lcitool, owing to the above issues
 - Ensured that the $download flag (for PyPI access) from configure is
   carried over into pytest/meson.build invocations.  This defaults to
   "enabled", so test dependencies should not have any issue being
   fetched on most developer workstations and build environments.
 - Fixed several bugs in the mkvenv.py local package installation mechanism.
 - Flags and workarounds added for installing local packages offline now
   apply *only* to local package installations, and not "normal"
   depspecs.
 - Reordered patches into more logical "phases" for easier review (I
   hope...?)

Known issues: "make check-build" still installs all of the test
dependencies to the pyvenv. I'm not sure if this is desired or not, but
I ran into problems preventing it from doing this without breaking a ton
of CI tests. I'll continue to investigate this, but wanted to send this
v3 out.

v2:

 - move "make check-venv" earlier in GitLab CI/CD pipeline, to avoid
   re-running configure
 - Fix functional tests not using PyPI to fetch pygdbmi package
 - Remove pre-requisites which are now merged

--js

John Snow (17):
  python/mkvenv: create timestamp file for each group "ensured"
  python/mkvenv: bump 'qemu.qmp' dependency for testdeps
  python/mkvenv: add 'tooling' and 'functests' dependency groups
  python/mkvenv: add mechanism to install local package(s)
  meson, mkvenv: add functests custom target
  tests: Use configured python to run GitLab iotests
  tests: use "run" script to execute device-crash-test
  tests/lcitool: add python3 wheel and setuptools deps for qemu
  python: add vendored qemu.qmp package
  meson, mkvenv: make functional tests depend on functests group
  meson, mkvenv: add qemu.git/python/qemu package to pythondeps.toml
  configure: unconditionally install "tooling" group
  tests: remove "make check-venv" target
  scripts: nudge users to use 'run' script for scripts that import
    qemu.qmp
  python: delete qemu.qmp
  python: update README.rst to reflect qemu.qmp's removal
  RFC: exclude pyvenv targets from "make check-build"

 python/README.rst                             |   50 +-
 configure                                     |    2 +-
 meson.build                                   |    1 +
 .gitlab-ci.d/buildtest.yml                    |   26 +-
 .gitlab-ci.d/cirrus/freebsd-14.vars           |    2 +-
 .gitlab-ci.d/cirrus/macos-14.vars             |    2 +-
 .gitlab-ci.d/windows.yml                      |    2 +
 python/qemu/qmp/__init__.py                   |   60 -
 python/qemu/qmp/error.py                      |   53 -
 python/qemu/qmp/events.py                     |  751 -----------
 python/qemu/qmp/legacy.py                     |  339 -----
 python/qemu/qmp/message.py                    |  217 ----
 python/qemu/qmp/models.py                     |  146 ---
 python/qemu/qmp/protocol.py                   | 1101 -----------------
 python/qemu/qmp/py.typed                      |    0
 python/qemu/qmp/qmp_client.py                 |  732 -----------
 python/qemu/qmp/qmp_shell.py                  |  689 -----------
 python/qemu/qmp/qmp_tui.py                    |  665 ----------
 python/qemu/qmp/util.py                       |  150 ---
 python/qemu/utils/qom_fuse.py                 |    1 -
 python/scripts/mkvenv.py                      |   44 +-
 python/scripts/vendor.py                      |    2 +
 python/setup.cfg                              |   31 +-
 python/tests/minreqs.txt                      |    8 +-
 python/tests/protocol.py                      |  596 ---------
 python/wheels/qemu_qmp-0.0.5-py3-none-any.whl |  Bin 0 -> 72263 bytes
 pythondeps.toml                               |   13 +-
 pyvenv/meson.build                            |   28 +
 scripts/compare-machine-types.py              |    7 +-
 scripts/device-crash-test                     |    5 +-
 scripts/mtest2make.py                         |    4 +-
 scripts/qmp/qemu-ga-client                    |   13 +-
 scripts/qmp/qmp-shell                         |   13 +-
 scripts/qmp/qmp-shell-wrap                    |   13 +-
 scripts/qmp/qom-fuse                          |   13 +-
 scripts/qmp/qom-get                           |   13 +-
 scripts/qmp/qom-list                          |   13 +-
 scripts/qmp/qom-set                           |   13 +-
 scripts/qmp/qom-tree                          |   13 +-
 scripts/qmp_helper.py                         |    9 +-
 scripts/render_block_graph.py                 |   10 +-
 scripts/simplebench/bench_block_job.py        |   10 +-
 tests/Makefile.include                        |   22 +-
 tests/docker/dockerfiles/alpine.docker        |    2 +
 tests/docker/dockerfiles/centos9.docker       |    2 +
 tests/functional/meson.build                  |    5 +-
 tests/lcitool/projects/qemu.yml               |    2 +
 tests/migration-stress/guestperf/engine.py    |   15 +-
 tests/vm/Makefile.include                     |   24 +-
 tests/vm/generated/freebsd.json               |    2 +
 50 files changed, 250 insertions(+), 5684 deletions(-)
 delete mode 100644 python/qemu/qmp/__init__.py
 delete mode 100644 python/qemu/qmp/error.py
 delete mode 100644 python/qemu/qmp/events.py
 delete mode 100644 python/qemu/qmp/legacy.py
 delete mode 100644 python/qemu/qmp/message.py
 delete mode 100644 python/qemu/qmp/models.py
 delete mode 100644 python/qemu/qmp/protocol.py
 delete mode 100644 python/qemu/qmp/py.typed
 delete mode 100644 python/qemu/qmp/qmp_client.py
 delete mode 100644 python/qemu/qmp/qmp_shell.py
 delete mode 100644 python/qemu/qmp/qmp_tui.py
 delete mode 100644 python/qemu/qmp/util.py
 delete mode 100644 python/tests/protocol.py
 create mode 100644 python/wheels/qemu_qmp-0.0.5-py3-none-any.whl
 create mode 100644 pyvenv/meson.build

-- 
2.52.0




^ permalink raw reply	[flat|nested] 40+ messages in thread

end of thread, other threads:[~2026-01-20 18:28 UTC | newest]

Thread overview: 40+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-01-19 21:27 [PATCH v4 00/17] python: drop qemu.qmp from qemu.git tree John Snow
2026-01-19 21:27 ` [PATCH v4 01/17] python/mkvenv: create timestamp file for each group "ensured" John Snow
2026-01-19 21:27 ` [PATCH v4 02/17] python/mkvenv: bump 'qemu.qmp' dependency for testdeps John Snow
2026-01-19 21:27 ` [PATCH v4 03/17] python/mkvenv: add 'tooling' and 'functests' dependency groups John Snow
2026-01-20  6:05   ` Vladimir Sementsov-Ogievskiy
2026-01-20  8:41   ` Thomas Huth
2026-01-20  8:47   ` Daniel P. Berrangé
2026-01-19 21:27 ` [PATCH v4 04/17] python/mkvenv: add mechanism to install local package(s) John Snow
2026-01-19 21:27 ` [PATCH v4 05/17] meson, mkvenv: add functests custom target John Snow
2026-01-20  8:53   ` Daniel P. Berrangé
2026-01-20 17:27     ` John Snow
2026-01-20 17:33       ` Daniel P. Berrangé
2026-01-20 17:37         ` John Snow
2026-01-19 21:27 ` [PATCH v4 06/17] tests: Use configured python to run GitLab iotests John Snow
2026-01-20  8:45   ` Thomas Huth
2026-01-20  8:54   ` Daniel P. Berrangé
2026-01-19 21:27 ` [PATCH v4 07/17] tests: use "run" script to execute device-crash-test John Snow
2026-01-20  8:46   ` Thomas Huth
2026-01-20  8:54   ` Daniel P. Berrangé
2026-01-19 21:27 ` [PATCH v4 08/17] tests/lcitool: add python3 wheel and setuptools deps for qemu John Snow
2026-01-20  8:52   ` Thomas Huth
2026-01-20  8:55   ` Daniel P. Berrangé
2026-01-19 21:27 ` [PATCH v4 09/17] python: add vendored qemu.qmp package John Snow
2026-01-19 21:27 ` [PATCH v4 10/17] meson, mkvenv: make functional tests depend on functests group John Snow
2026-01-20  8:57   ` Daniel P. Berrangé
2026-01-20  8:57   ` Thomas Huth
2026-01-19 21:27 ` [PATCH v4 11/17] meson, mkvenv: add qemu.git/python/qemu package to pythondeps.toml John Snow
2026-01-19 21:27 ` [PATCH v4 12/17] configure: unconditionally install "tooling" group John Snow
2026-01-20  8:59   ` Daniel P. Berrangé
2026-01-20  9:00   ` Thomas Huth
2026-01-19 21:27 ` [PATCH v4 13/17] tests: remove "make check-venv" target John Snow
2026-01-20  9:01   ` Daniel P. Berrangé
2026-01-19 21:27 ` [PATCH v4 14/17] scripts: nudge users to use 'run' script for scripts that import qemu.qmp John Snow
2026-01-20  9:02   ` Daniel P. Berrangé
2026-01-19 21:27 ` [PATCH v4 15/17] python: delete qemu.qmp John Snow
2026-01-19 21:27 ` [PATCH v4 16/17] python: update README.rst to reflect qemu.qmp's removal John Snow
2026-01-20  9:05   ` Daniel P. Berrangé
2026-01-20 18:08     ` John Snow
2026-01-19 21:27 ` [PATCH v4 17/17] RFC: exclude pyvenv targets from "make check-build" John Snow
2026-01-20 18:27 ` [PATCH v4 00/17] python: drop qemu.qmp from qemu.git tree John Snow

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.