Re: [PATCH v2 19/22] tests/qtest/migration: Add migration-test-smoke

["Daniel P. Berrangé" <berrange@redhat.com>]

On Fri, Dec 20, 2024 at 10:18:37AM -0500, Peter Xu wrote:
> On Thu, Dec 19, 2024 at 04:31:04PM -0300, Fabiano Rosas wrote:
> > We shouldn't change stuff that's also used by the rest of the
> > community. People know about QEMU_TEST_FLAKY_TESTS and -m slow. These
> > must continue to work the same.
> 
> I see what I overlook; it's used much more than I thought in qtest and we
> also have a CI for it.. So ok, let's keep at least QEMU_TEST_FLAKY_TESTS.
> 
> But again, I don't think it matters much even if we rename it, it means the
> flaky CI test won't run these two migration tests, but that's not the end
> of the world either, if you see what I meant.  CI relies on the normal
> tests rather than flaky tests to present.
> 
> We should be able to move in / take out FLAKY tests at will, as that's not
> what CI is really relying on.  Here renaming the macro in migration test
> almost means we take both ou.t

Side-note - QEMU_TEST_FLAKY_TESTS is something we should apply
consistently across all types of tests - unit, qtest, functional,
and across all environments - CI and local developer execution.

In recent changes to functional testing, I've set the expectation[1]
that any use of QEMU_TEST_FLAKY_TESTS *must* be accompanied by a
link to the gitlab.com/qemu/qemu-project  issue that describes
the flaky behaviour seen. We've got too many places with flaky
tests where we don't quiet remember what was flaky, so don't
know if we can remove it or not

With regards,
Daniel

[1]https://gitlab.com/qemu-project/qemu/-/blob/master/tests/functional/qemu_test/decorators.py?ref_type=heads#L42

-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|