Re: [PATCH v2 6/7] CI: Stop building docs on centos8

[Markus Armbruster <armbru@redhat.com>]

Kevin Wolf <kwolf@redhat.com> writes:

> Am 14.02.2023 um 08:40 hat Markus Armbruster geschrieben:
>> I read this on Friday, and decided to let it sit until after the
>> weekend.  Well, it's now Tuesday, and to be frank, it's still as
>> offensively flippant as it was on Friday.  It shows either ignorance of
>> or cavalier disregard for the sheer amount of work some of us have had
>> to put into keeping old versions of Python viable.
>> 
>> The latter would be quite unlike you, so it must be the former.
>
> Honest question, Markus, because I haven't been following as much what
> is happening in Python recently: What are the biggest pain points in
> this context?
>
> Has Python started removing features from new versions more aggressively
> so that we have to update the code so it can run on newer versions, and
> still keep compatibility paths for older versions that don't have the
> replacement yet?

A caveat before I answer: I'm aware of four non-trivial uses of Python,
(QAPI generator, Avocado, the qemu/qmp package, meson), but my direct
experience is limited to just QAPI.

And another preliminary remark: I think there's confusion about the
actual impact of dropping 3.6.  But attempting to clarify that is a
topic for another message (Message-ID: <87v8k2ycjb.fsf@pond.sub.org>,
just sent).

I've personally observed roughly three kinds of pain.  In order of
(sharply) increasing intensity:

1. Python language and core libraries

   Python still evolves at a good pace.

   We can sidestep new features.  We can't sidestep bugs.  In either
   case, we need to become aware of the issue first.

   For features, diligent reading of documentation suffices, but for
   bugs, it's testing or bust.

   Things have exploded into John's face often enough to "motivate" him
   to spend a huge chunk of his time (man-months) on building and
   maintaining infrastructure for testing with all the Python versions.
   Naturally, the wider the version range, the bigger the test matrix,
   and the deeper the time sink.

   The range of versions we're trying to target is actually wider than
   Python's deprecation grace period.  This means we sometimes get to do
   things in both the old and the new way (because neither works across
   the whole range of versions), or look for ways to sidestep the mess
   somehow.

   Python programs are nicely portable between host systems.  Between
   more than a few Python versions, not so much.

2. Python tooling and non-core libraries

   Much of these don't give a rat's ass for anything but the latest few
   versions, let alone for EOLed ones like 3.6.  Likely because they
   decided it would be a fool's errand.

   For instance, Python packaging has changed beyond recognition since
   3.6.  There is no single way to package that spans the range of
   versions we try to target.  Bites the qemu/qmp package.

   Linters are also a moving target, and keeping lint tests pass for a
   wide range of linters version involves lots of largely pointless
   fiddling.  Another dimension in the test matrix.

3. Python typing is too immature in anything but latest versions

   This is kind of a special case of 2., but it's a big one for *me*, so
   I'm making it an item of its own.

   Type hints have been evolving substantially, and it doesn't look like
   this is going to stop any time soon.

   Trying to keep typing tests pass for a wide range of mypy versions is
   even less practical than for other linters.

   Can we simply skip type checking unless we have a sufficiently recent
   mypy?  After all, type hints are designed to be transparent at
   runtime, so code with new type hints should still run on old Python,
   shouldn't it?  Nope.  Doesn't.

   Kevin and others pointed out that the QAPI generator code is harder
   to work with than it should be, in good part because they have to
   spend too much time understanding possible types.  In response to
   this valid complaint, we embarked on the known non-trivial project to
   add type hints to the QAPI code generator.  This may have been a
   mistake.  We had no idea just how much trouble type hints would give
   us when combined with a range of Python versions this wide.  But here
   we are, 2+ years older and somewhat better informed, and our informed
   desire is to narrow the version range as much as practical.

   Ditching 3.6 feels eminently practical.  Let's do it.