Re: [PATCH v2 6/7] CI: Stop building docs on centos8

[Markus Armbruster <armbru@redhat.com>]

John Snow <jsnow@redhat.com> writes:

> On Tue, Feb 14, 2023 at 12:26 PM Kevin Wolf <kwolf@redhat.com> wrote:
>>
>> Am 14.02.2023 um 15:03 hat Paolo Bonzini geschrieben:
>> > In the case of Python the issue is not the interpreter per se, though
>> > there are a couple new feature in Python 3.7 that are quite nice (for
>> > example improved data classes[1] or context variables[2]). The main
>> > problem as far as I understood (and have seen in my experience) is
>> > linting tools. New versions fix bugs that caused false positives, but
>> > also become more strict at the same time. The newer versions at the
>> > same time are very quick at dropping support for old versions of
>> > Python; while older versions sometimes throw deprecation warnings on
>> > new versions of Python. This makes it very hard to support a single
>> > version of, say, mypy that works on all versions from RHEL8 and SLE15
>> > to Fedora 38 and Ubuntu 23.04.
>>
>> Why do we have to support a single version of mypy? What is wrong with
>> running an old mypy version with old Python version, and a newer mypy
>> with newer Python versions?
>
> Well, the problem is, ...
>
>>
>> Sure, they will complain about different things, but it doesn't feel
>> that different from supporting multiple C compilers in various versions.

Different C compilers often produce different, useful warnings, so
compiling with several of them can get you more useful warnings.

However, you get pretty much all the useful warnings just from the
latest versions, at least in my experience.  You keep the old ones
working not for getting additional useful warnings, but to compile on
additional hosts.  As the version range widens, warnings from old
versions might even become such an annoyance that switching them off
becomes sensible.

> ...well, it's this.
>
> The first dimension of the test matrix is the version of mypy itself.
> The second dimension of the test matrix is the version of Python mypy
> runs under. A given version of mypy can run under multiple versions of
> Python and may indeed have different behaviors under each.
> The third dimension of this test matrix is the version(s) of Python
> our code is targeting; for instance we configure mypy to understand
> that we require Python 3.6.
>
> Trying to cast the net wide on *all of these* gets tough; the very
> latest versions of mypy don't support 3.6 at all, so you'll have cases
> where the mypy that just so happens to come with your Fedora
> installation just won't work properly with our code anymore, which has
> to test under 3.6 appropriately.
>
> In general, the majority of python package upstreams I am aware of
> simply pin a python version and a mypy version and leave it at that.

When pretty much everybody else doesn't do something we're doing, there
are two obvious explanations: (1) we're so smart we discovered something
worth doing pretty much first, or (2) the thing we're doing is a bad
idea in general.

In this case, my money is on (2).  It may still be a good idea for us
(because "in general").  But that notion needs to be supported with
concrete reasons, such as ...

> Compatibility concerns for most of the upstreams just do not really
> ever consider that you'd be running *and* testing against a large
> spread of versions. I've gone the extra mile and I run mypy and pylint
> under all versions of python from 3.6 to 3.11 to ensure that developer
> workstations can run the same linting tests and feel assured that if
> it passes locally, it will pass on the CI and vice-versa. Our matrix
> is generally quite a bit larger than most upstreams, so I am playing a
> lot of whack-a-mole to keep things functioning consistently across the
> versions. I will probably even add support for Python 3.12 alpha
> *soon* because it's already available in the Fedora repo, and it will
> be good to find any compatibility issues before that version is
> officially released. (I'll need to do this for the qemu.qmp package,
> which should have support for 3.12 on the day 3.12 releases and not
> sometime afterwards.)
>
> I know the "check-tox" test was annoying upstream as it sometimes
> turned yellow because a new pylint version was released, but that's
> how I stay ahead of breaking changes before they're on local
> workstations.
>
> We could avoid at least one of the reasons for dropping 3.6 support by
> saying "Tough cookies, you'll use precisely this version of mypy and
> precisely this python interpreter, or you'll get nothing" and that
> does relieve a huge amount of pressure as-is.

... this one:

>                                               But, as a courtesy, I do
> go out of my way -- where possible -- to ensure that developers can
> use whichever versions of tools their distro is providing them so that
> they don't have to mess around with that stuff. Unfortunately, that
> means when upstreams start dropping support for older things, it gets
> hairier and quite a bit more painful.

I appreciate the courtesy.  But it feels like it's gotten rather costly,
and getting costlier.

> I think 3.6 being the first version that offers full-throated type
> hint support has unique pain in this circumstance because a lot of the
> details did not get hammered out until 3.7 or later; in general the
> amount of workarounds I have had to apply because something type
> checks only in 3.7+ but not 3.6 is fairly high compared to the number
> of times I've found that a different version was the odd one out. I
> don't expect this to become a recurring desire where I start to whine
> about old Python versions before our support window would otherwise
> let me drop them; I think this is actually just a unique pain point of
> this one version if we keep static typing.
>
> That's the tooling issue, anyway.

Perhaps dropping just 3.6 from the test matrix will get us back to a
reasonable cost / benefit ratio.  But I'd like to encourage you to have
a sober look at the test matrix, and not shy away from cutting more.