[PATCH] docs: document what configure does with virtual environments

[Paolo Bonzini <pbonzini@redhat.com>]

Given the recent confusion around how QEMU detects the system
Meson installation, and/or decides to install its own, it is
time to fill in the "Python virtual environments and the QEMU
build system" section of the documentation.

As a curiosity, a first and partial draft of the text was generated
by an LLM[1].  It required quite a bit of editing and probably did not
save much time, but some expressions do remain in the finished text.

[1] https://chat.openai.com/share/42c1500d-71c1-480b-bab9-7ccc2c155365

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 docs/devel/build-system.rst | 54 ++++++++++++++++++++++++++++++++++---
 pythondeps.toml             |  3 ++-
 2 files changed, 53 insertions(+), 4 deletions(-)

diff --git a/docs/devel/build-system.rst b/docs/devel/build-system.rst
index 21f78da7d1d..4def2e55e73 100644
--- a/docs/devel/build-system.rst
+++ b/docs/devel/build-system.rst
@@ -122,10 +122,49 @@ functioning.  These are performed using a few more helper functions:
    indicated by $TMPC.
 
 
-Python virtual environments and the QEMU build system
------------------------------------------------------
+Python virtual environments and the build process
+-------------------------------------------------
+
+An important part of configure is to create a Python virtual environment
+(venv) during the configuration phase, using the Python interpreter that
+``configure`` identified, or that was requested via the ``--python``
+command line option and the ``$PYTHON`` variable from the environment.
+The venv resides in the ``pyvenv`` directory in the build tree,
+and provides consistency in how the build process runs Python code.
+In particular it avoids a potential mismatch, where Meson and Sphinx
+binaries on the PATH might operate in a different Python environment
+than the one chosen by the user during the build process.
+
+At this stage, ``configure`` also queries the chosen Python interpreter
+about QEMU's build dependencies.  ``configure`` does  *not*
+pick the ``meson``, ``sphinx-build`` or ``avocado`` binaries in the PATH;
+likewise, there are no options such as ``--meson`` or ``sphinx-build``.
+If QEMU does not find a dependency, check that it was installed in the
+right ``site-packages`` directory or with the right ``pip`` program.
+
+If the package is available as a system package for the chosen
+interpreter, ``configure`` prepares a small script that invokes it
+from the venv itself.  If not, ``configure`` can also optionally
+install dependencies in the virtual environment with ``pip``.
+Downloading is triggered only when a ``configure`` option (currently,
+only ``--enable-docs``) is explicitly enabled but the dependencies are
+not present, and can also be disabled with ``--disable-download``.[#pip]_
+
+.. [#pip_] Avocado can also be installed with ``pip`` in the virtual
+           environment when running ``make check-avocado``.  In this
+           case, it is not currently possible to block the downloading.
+
+The required versions of the packages are stored in a configuration file
+``pythondeps.toml``.  The format is custom to QEMU, but it is documented
+at the top of the file itself and it should be easy to understand.  The
+requirements should make it possible to use the version that is packaged
+that is provided by supported distros.
+
+When dependencies are downloaded, instead, ``configure`` uses a "known
+good" version that is also listed in ``pythondeps.toml``.  In this
+scenario, ``pythondeps.toml`` behaves like the "lock file" used by
+``cargo``, ``poetry`` or other dependency management systems.
 
-TBD
 
 Stage 2: Meson
 ==============
@@ -376,6 +415,15 @@ This is needed to obey the --python= option passed to the configure
 script, which may point to something other than the first python3
 binary on the path.
 
+By the time Meson runs, Python dependencies are available in the virtual
+environment and should be invoked though the scripts that ``configure``
+places under ``pyvenv``.  One way to do so is as follows, using Meson's
+``find_program`` function::
+
+  sphinx_build = find_program(
+       fs.parent(python.full_path()) / 'sphinx-build',
+       required: get_option('docs'))
+
 
 Stage 3: Make
 =============
diff --git a/pythondeps.toml b/pythondeps.toml
index 0a35ebcf9f0..4beadfd96f5 100644
--- a/pythondeps.toml
+++ b/pythondeps.toml
@@ -10,7 +10,8 @@
 # - accepted: accepted versions when using a system package
 # - installed: fixed version to install in the virtual environment
 #              if a system package is not found; if not specified,
-#              the minimum and maximum
+#              defaults to the same as "accepted" or, if also missing,
+#              to the newest version available on PyPI.
 # - canary: if specified, use this program name to present more
 #           precise error diagnostics to the user.  For example,
 #           'sphinx-build' can be used as a bellwether for the
-- 
2.41.0