qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: John Snow <jsnow@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Philippe Mathieu-Daudé" <philmd@linaro.org>,
	"John Snow" <jsnow@redhat.com>,
	"Wainer dos Santos Moschetta" <wainersm@redhat.com>,
	"Paolo Bonzini" <pbonzini@redhat.com>,
	"Alex Bennée" <alex.bennee@linaro.org>,
	"Beraldo Leal" <bleal@redhat.com>,
	"Thomas Huth" <thuth@redhat.com>,
	"Cleber Rosa" <crosa@redhat.com>
Subject: [RFC PATCH 0/3] configure: create a python venv and install meson
Date: Tue, 28 Mar 2023 17:11:16 -0400	[thread overview]
Message-ID: <20230328211119.2748442-1-jsnow@redhat.com> (raw)

This patch series creates a mandatory venv during configure time and
uses it to install meson.

The eventual point of this is to ensure that the Python used to run
meson is the same Python used to run Sphinx, tests, and any build-time
scripting we have. As it stands, meson and sphinx (and their extensions)
may run in a different Python environment than the one configured and
chosen by the user at configure/build time.

My goals for this series and future work are:

- Unconditionally create a venv to be used for both building and testing
  at configure time
- The python interpreter used by this venv is unconditionally the one
  specified by configure
- *all* python scripts in qemu.git executed as part of the build system,
   tests, or CI are *always* executed in the context of this venv.
- It is possible to build and test fully offline with sufficient
  preparation by installing appropriate system packages.
- Missing depdencies, when possible, are fetched and installed
  automatically to make developer environments "just work".
- Minimize cost during configure time whenever possible.

This series is in RFC state; some notable things:
- So far, only meson is included in this setup.
- There's a ton of debugging prints everywhere. It's extremely chatty right now.
- Sphinx and qemu.qmp are not yet included in this venv.
  (qemu.qmp isn't able to be removed from the tree yet.)
- Testing isn't yet *fully* switched over.
- There is no online functionality yet, this series is *100% offline* -
  it's the harder option, so I tackled it first.

Some known bugs as of now:
- venv-in-venv setups are not yet handled.
- python3.7 setups without setuptools/pip in the host environment may be
  unable to generate script file shims; a workaround is in development
  but wasn't ready today. I decided to exclude it.

The good news:
- Works for Python 3.7 and up, on Fedora, OpenSuSE, Red Hat, CentOS, Alpine,
  Debian, Ubuntu, NetBSD, OpenBSD, and hopefully everywhere
- Startup cost is only milliseconds on the fast path
- No new dependencies (...for most platforms; the asterisk is Debian.)
- No new configure flags (...yet.)

John Snow (3):
  python: add mkvenv.py
  tests: add python3-venv dependency
  configure: install meson to a python virtual environment

 configure                                     | 155 ++++--
 .gitlab-ci.d/buildtest-template.yml           |   2 +-
 python/scripts/mkvenv.py                      | 445 ++++++++++++++++++
 .../dockerfiles/debian-all-test-cross.docker  |   3 +-
 .../dockerfiles/debian-hexagon-cross.docker   |   3 +-
 .../dockerfiles/debian-riscv64-cross.docker   |   3 +-
 .../dockerfiles/debian-tricore-cross.docker   |   3 +-
 7 files changed, 561 insertions(+), 53 deletions(-)
 create mode 100644 python/scripts/mkvenv.py

-- 
2.39.2




             reply	other threads:[~2023-03-28 21:12 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-03-28 21:11 John Snow [this message]
2023-03-28 21:11 ` [RFC PATCH 1/3] python: add mkvenv.py John Snow
2023-03-29 12:56   ` Paolo Bonzini
2023-03-30 14:00     ` John Snow
2023-03-31  8:44       ` Paolo Bonzini
2023-03-31 10:01         ` Paolo Bonzini
2023-04-13 16:10     ` John Snow
2023-04-13 16:26     ` John Snow
2023-03-28 21:11 ` [RFC PATCH 2/3] tests: add python3-venv dependency John Snow
2023-03-28 21:11 ` [RFC PATCH 3/3] configure: install meson to a python virtual environment John Snow
2023-03-29 12:32   ` Paolo Bonzini
2023-03-29 12:53   ` Daniel P. Berrangé
2023-03-29 13:27     ` Paolo Bonzini
2023-03-29 13:31       ` Daniel P. Berrangé
2023-04-12  1:27         ` John Snow
2023-03-29 13:02 ` [RFC PATCH 0/3] configure: create a python venv and install meson Paolo Bonzini
2023-03-30 14:11   ` John Snow
2023-03-31  8:40     ` Paolo Bonzini

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230328211119.2748442-1-jsnow@redhat.com \
    --to=jsnow@redhat.com \
    --cc=alex.bennee@linaro.org \
    --cc=bleal@redhat.com \
    --cc=crosa@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=philmd@linaro.org \
    --cc=qemu-devel@nongnu.org \
    --cc=thuth@redhat.com \
    --cc=wainersm@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).