Re: Build platform guarantees, docs, tests, and snakes in the garden

[Markus Armbruster <armbru@redhat.com>]

John Snow <jsnow@redhat.com> writes:

> Hi, I've long been a little confused about the specifics of our build
> platform guarantee and how it applies to documentation and testing.

"Guarantee" feels too strong.  See below.

> My *current* understanding is that our build platform guarantee applies to
> both unit tests and building documentation, but that this requirement may
> not be as absolute as I imagine it.
>
> The way I have endeavored to manage the Python tooling in our tree so far
> is to preserve, without fail, our ability to perform fully offline builds
> on all supported platforms (provided that the right distro repo packages
> are available).

Relevant part of docs/about/build-platforms.rst:

    Some build dependencies may follow less conservative rules:

    Python runtime
      Distributions with long-term support often provide multiple versions
      of the Python runtime.  While QEMU will initially aim to support the
      distribution's default runtime, it may later increase its minimum version
      to any newer python that is available as an option from the vendor.
      In this case, it will be necessary to use the ``--python`` command line
      option of the ``configure`` script to point QEMU to a supported
      version of the Python runtime.

      As of QEMU |version|, the minimum supported version of Python is 3.9.

    Python build dependencies
      Some of QEMU's build dependencies are written in Python.  Usually these
      are only packaged by distributions for the default Python runtime.
      If QEMU bumps its minimum Python version and a non-default runtime is
      required, it may be necessary to fetch python modules from the Python
      Package Index (PyPI) via ``pip``, in order to build QEMU.

We "initially aim to support the distribution's default runtime".  Once
we don't, fetching from PyPI "may be necessary".  This seems to imply
that such fetching will be necessary when we use the default runtime.

I read "aim" as "make an effort".  This isn't exactly a "guarantee".
It's much, much weaker than "to preserve, without fail, our ability to
perform fully offline builds on all supported platforms".

Keeping fully offline builds working is certainly desirable, but not
regardless of cost.

>                 The Python virtual environment created at configure time
> bends over backwards to use system packages *whenever possible*, and the
> list of exceptions - notably Meson itself - uses vendored packages only in
> very specific cases where it is possible to vendor such packages. Fetching
> packages from PyPI is generally offered only as a convenience for developer
> workstations to, in general, save developers from having to know anything
> about Python. (I think I've done a good job there, to be honest!)

You have: few people have noticed your work.

> (Notably: Meson is pure python and has no dependencies, so it is possible
> to vendor it for offline builds. Tools like Sphinx, however, have many
> dependencies and are not so easily vendored. Thus, we have created a
> tenuous arrangement where we are allowed to use versions of Meson that
> otherwise would break our build platform guarantee.)
>
> Lately, we've had some issues with the wide range of Sphinx versions we
> support presenting various cross-platform difficulties. In particular,
> Akihiko Odaki has sent patches to bump our Sphinx version to at least
> 6.2.1, because platforms with Python 3.13.1 can no longer run Sphinx 3.x at
> all, so having that be our "default install version" causes issues on newer
> platforms.
>
> However, if we take as iron-clad our commitment to the build platform
> promise -- *and* guarantee offline/tarball builds as well -- then Debian 12

I do not think such a commitment exists; see my reading of
build-platforms.rst above.

Even if it did, treating it as iron-clad would be foolish.  We need to
consider cost vs. reward, always.

> (as an example) only offers Sphinx 5.3.0 and not newer unless we allow
> internet access to fetch Sphinx 6.2.1. This is not a problem for developer
> workstations at all, but I am unclear on what problems this may cause for
> tarball releases and downstream offline/isolated/reproducible builds, if
> any.
>
> In this case, we can (probably) "fix" the issue by continuing to allow
> older Sphinx while preferring a newer Sphinx version when it is missing,
> but then we lose the ability to make code cleanups and drop a lot of
> back-compat crud. If memory serves, there were other issues recently where
> older versions of Sphinx behaved differently from newer versions, causing
> intermittent failures that were hard to track down.
>
> What I'd like to know is: what precisely are our options in this scenario?
> Do we consider it acceptable for some platforms to be unable to build docs
> offline? How highly do we value the ability to locally build docs for any
> given release?

I believe the value of fully offline builds goes down as the build
platform ages.

Initially, the distribution will (hopefully!) want to package the then
current version of QEMU.  We want to make that easy.  Enabling fully
offline builds are part of that.

Perhaps the distribution later wants to package / build newer versions
of QEMU fully offline on the now aged release.  I'd expect this to
happen less and less as the distribution ages.  Making it easy is still
desirable, but worth less and less trouble to us.

> Before I throw my weight behind any given option, I just want to know what
> we consider our non-negotiable obligations to be.

In my opinion: none.  We should consider the tradeoffs, and make
pragmatic decisions.