Re: [RFC PATCH 0/9] tests: run python tests under the build/tests/venv environment

[John Snow <jsnow@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 2807 bytes --]

On Fri, May 13, 2022, 12:50 PM Paolo Bonzini <pbonzini@redhat.com> wrote:

> On 5/13/22 18:07, Daniel P. Berrangé wrote:
> >> e.g. what if I want to require mypy >= 0.900 for testing, but you have a
> >> system package that requires mypy < 0.700?
> > I would expect us to not require packages that are not present in
> > the distros implied by
> >
> >    https://www.qemu.org/docs/master/about/build-platforms.html
> >
> > if that was absolutely a must have, then gracefully skip tests
> > if the system version wasn't new enough. The user could always
> > pass --python-env=pip if they want to force new enough
> >
>
> Consider that e.g. RHEL RPMs do not do mypy or pylint in %check, because
> the version of the linters in RHEL is usually older than what the
> upstream packages expect.
>
> I don't think it's a good idea for QEMU to support what Red Hat
> packagers decided was a bad idea to support.
>
> Paolo
>

Yeah, I have to insist that due to the way these packages are developed
upstream that it is simply not reasonable to expect that the local package
version will work. pylint changes behavior virtually every single release.
This series itself even has a patch that is playing whackamole to support a
mypy that's brand new while supporting older mypy versions.

It's a huge overhead for little gain.

Far preferable to just say "Oh, your linter version is too old, we can't
run this test locally."

the qemu (and qemu.qmp) packages do not express a runtime/install
dependency on mypy/pylint/isort/flake8/avocado/tox. These packages only get
pulled in for the [devel] option group, and for good reason.

What is really the outlier here is iotest 297, which brings a kind of
meta-test into the same category as functional/regression tests. Supporting
this on default system packages is not on my personal todo list. Moving
this test off to a "make check-python" and deleting iotest 297 might be an
option. This is a test that simply might need to be skipped by an SRPM
build. (it already isn't run, so there's no additional harm by continuing
to not run it.)

If we are running a test suite and we allow pypi via the config, then I
believe we ought to allow the pypi versions to supersede the system ones -
*iff* the system ones are insufficient. I will continue to endeavor to test
a very wide matrix of dependencies, I just have to be honest that I don't
think I will reasonably achieve the full breadth you are asking for here.

For no-internet configs, we may have to accept that some platforms simply
don't have new enough dependencies to run some tests. I don't see this as
violating our build platform promise. I don't believe the build platform
promise ever reasonably extended to a "development platform promise".

--js

>

[-- Attachment #2: Type: text/html, Size: 3869 bytes --]