Re: Minimum sphinx version?

[Markus Armbruster <armbru@redhat.com>]

John Snow <jsnow@redhat.com> writes:

> Hi, following up from my KVM forum talk (
> https://www.youtube.com/watch?v=GGbz3WwvqLw ) I wanted to ask about what
> version of Sphinx we could use as a minimum version to help ease the burden
> of modernizing our QMP docs without a long backwards compatibility tail
> (and hundreds of lines of compatibility goop to make it happen.)

Let me add some perspective...

Current docs/sphinx/qapidoc.py is 370 SLOC.  In my experience, it
resists improvement pretty hard, by Sphinx throwing impenetrable stack
traces at the slightest provocation.

Your replacement, which generated *much* superior rendered documentation
is how much, not counting compatibility goop?

The compatibility goop adds how much?

As far as I know, the replacement proper is nice, maintainable code,
while the goop is, well, goop.  Anything breaks there, we run for the
hills and wait for the John Snow cavalry to save the day.

> First, some necessary background on the current state of our Python
> environment and build system:
>
> Paolo and I worked on a Python virtual environment tied to the QEMU build
> system (python/scripts/mkvenv.py) which performs various setup actions
> necessary to instantiate python *stuff* to ensure the build works and is
> always using the correct, configured Python interpreter.
>
> Currently, configure will invoke mkvenv in such a fashion that it prefers
> any local packages you have installed: notably, meson, sphinx, and
> sphinx-rtd-theme if you already have them installed from your distro's
> repository will be used *if they use the version of python you specified to
> configure* (or they match the version that configure guessed if you didn't
> specify one.)
>
> For meson, if you don't meet the minimum version, we will install a version
> for you from our vendored whl files in the repository/tarball.
>
> For sphinx, if you don't meet the minimum version and you have specified
> --enable-docs, we'll install a version from the internet, from PyPI. If you
> did not specify --enable-docs, we disable docs.

Double-checking I understand what you wrote:

* We support Python 3.8 and later.

  3.8 was released in 2019, upstream full support ended in 2021, and
  security fixes are about to end.

  docs/about/build-platforms.rst claims 3.7.  I'll fix it.

* We support Meson 1.1.0 and later.

  If the system doesn't provide, we fall back to bundled Meson 1.2.3.

* We support Sphinx 3.4.3 or later.

  If the system doesn't provide, we pull one from PyPI with
  --enable-docs, else disable docs.  Which version do we pull?

  There's a check for >=1.7.0 in docs/meson.build.  Could that go?

* We support sphinx-rtd-theme.

  Does it come with Sphinx?

> We did it this way because:
> - It's *really fast* if you already have the dependencies you need
> - meson is easy to vendor as a .whl because it's pure python, has *zero*
> dependencies, and it's absolutely required for the build.
> - docs are not *required* to build QEMU
> - We went out of our way to ensure that QEMU could be built in an offline,
> isolated environment on all of our supported platforms.

Loaded question, but I know you can answer this better than I could: how
common is it to support a wide range of versions for Python tools?

> That being said:
>
> The current reality is that Sphinx 3.4.3 is our minimum because RHEL 9
> offers that as the distro package and I have not dared bump our version
> beyond that for fear of disrupting our ability to build docs on RHEL 9
> without internet.
>
> What I'd like to ask is: How adamant are we that we can build docs on older
> platforms? Do we consider it part of our platform promise? Can we bump
> Sphinx to a slightly newer version at the expense of offline doc builds for
> RHEL 9?
>
> To reiterate: interactive developer builds from git or source tarballs
> would continue to work (and be able to build docs just fine) so long as
> --enable-docs is passed, but will require an internet connection to set up
> the python environment with a new-enough sphinx. RHEL 9 rpm builds of
> bleeding edge QEMU may be cut off from building docs in isolated RPMBUILD
> environments unless that repository backports a newer sphinx that can be
> identified as a dependency in the specfile.
>
> How much of a problem do we consider that to be?
>
> CCing the usual suspects.
>
> --js