From: Cornelia Huck <cohuck@redhat.com>
To: "Philippe Mathieu-Daudé" <philmd@redhat.com>
Cc: Eduardo Habkost <ehabkost@redhat.com>,
Denis Plotnikov <dplotnikov@virtuozzo.com>,
qemu-devel@nongnu.org,
Wainer dos Santos Moschetta <wainersm@redhat.com>,
Cleber Rosa <crosa@redhat.com>
Subject: Re: [PATCH v2 04/29] tests/acceptance: Use 'version-min' tag to verify QEMU binary version
Date: Thu, 30 Jan 2020 18:08:59 +0100 [thread overview]
Message-ID: <20200130180859.43954c22.cohuck@redhat.com> (raw)
In-Reply-To: <ee877d12-c0a4-4f82-5d59-2e0235158479@redhat.com>
On Thu, 30 Jan 2020 14:28:19 +0100
Philippe Mathieu-Daudé <philmd@redhat.com> wrote:
> On 1/30/20 1:26 PM, Cornelia Huck wrote:
> > On Wed, 29 Jan 2020 22:23:20 +0100
> > Philippe Mathieu-Daudé <philmd@redhat.com> wrote:
> >
> >> Introduce the 'version-min' tag.
> >> Tests can set it to the minimum version of QEMU they require.
> >> If QEMU is older, the tests will be cancelled (skipped):
> >>
> >> $ python -m avocado --show=app run tests/acceptance/x86_cpu_model_versions.py
> >> (01/11) X86CPUModelAliases.test_4_0_alias_compatibility: CANCEL: Test expects version '4.1' but QEMU binary is '3.1.1' (0.10 s)
> >> (02/11) X86CPUModelAliases.test_4_1_alias: CANCEL: Test expects version '4.1' but QEMU binary is '3.1.1' (0.10 s)
> >> (03/11) X86CPUModelAliases.test_none_alias: CANCEL: Test expects version '4.1' but QEMU binary is '3.1.1' (0.10 s)
> >> ...
> >>
> >> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> >> ---
> >> tests/acceptance/avocado_qemu/__init__.py | 18 ++++++++++++++++++
> >> 1 file changed, 18 insertions(+)
> >
> > Hm... this might cause headaches if someone tries to backport tests on
> > distro versions; but we probably won't be able to test for the feature
> > in many cases anyway?
> >
>
> Are you saying a distro version might not match upstream version when a
> feature is introduced? Isn't it the point of versioning? If you are
> thinking of some case you point me at it so I can improve this patch?
Something like the following:
- a feature together with a test is introduced in QEMU version n
- a distro is using m (where m < n), but wants the new feature and
backports it
If that distro now wants to include the acceptance test as well (to run
in their internal testing), they need to adapt the version check as
well, or the test simply won't run. Depending on how this is reported,
it might not be easily noticed. If the test depends on another feature
(but the feature does not), it might get a bit more complicated.
It's probably not a big problem, but I wanted to point it out.
next prev parent reply other threads:[~2020-01-30 17:10 UTC|newest]
Thread overview: 71+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-29 21:23 [PATCH v2 00/29] tests/acceptance/virtio_seg_max_adjust: Restrict it to Linux/X86 Philippe Mathieu-Daudé
2020-01-29 21:23 ` [PATCH v2 01/29] hw/core: Allow setting 'virtio-blk-device.scsi' property on OSX host Philippe Mathieu-Daudé
2020-01-30 12:20 ` Cornelia Huck
2020-01-30 13:23 ` Philippe Mathieu-Daudé
2020-01-29 21:23 ` [PATCH v2 02/29] accel/accel: Make TYPE_ACCEL abstract Philippe Mathieu-Daudé
2020-01-29 21:31 ` Philippe Mathieu-Daudé
2020-01-30 12:22 ` Cornelia Huck
2020-01-29 21:23 ` [PATCH v2 03/29] python/qemu: Add binutils::binary_get_version() Philippe Mathieu-Daudé
2020-02-03 19:17 ` Wainer dos Santos Moschetta
2020-02-06 16:39 ` Philippe Mathieu-Daudé
2020-01-29 21:23 ` [PATCH v2 04/29] tests/acceptance: Use 'version-min' tag to verify QEMU binary version Philippe Mathieu-Daudé
2020-01-30 12:26 ` Cornelia Huck
2020-01-30 13:28 ` Philippe Mathieu-Daudé
2020-01-30 17:08 ` Cornelia Huck [this message]
2020-01-30 17:18 ` Philippe Mathieu-Daudé
2020-01-30 20:40 ` Eduardo Habkost
2020-01-29 21:23 ` [PATCH v2 05/29] tests/acceptance: Restrict X86CPUModelAliases test to QEMU >= 4.1 Philippe Mathieu-Daudé
2020-01-29 21:23 ` [PATCH v2 06/29] python/qemu: Add binutils::binary_get_arch() Philippe Mathieu-Daudé
2020-01-31 9:49 ` Cornelia Huck
2020-01-29 21:23 ` [PATCH v2 07/29] tests/acceptance: Use the 'arch' tag to verify QEMU binary target Philippe Mathieu-Daudé
2020-01-31 9:51 ` Cornelia Huck
2020-01-29 21:23 ` [PATCH v2 08/29] python/qemu: Add binutils::binary_get_machines() Philippe Mathieu-Daudé
2020-01-31 9:53 ` Cornelia Huck
2020-01-29 21:23 ` [PATCH v2 09/29] tests/acceptance: Use 'machine' tag to check if available in QEMU binary Philippe Mathieu-Daudé
2020-01-31 10:00 ` Cornelia Huck
2020-02-06 18:17 ` Liam Merwick
2020-02-26 17:34 ` Liam Merwick
2020-01-29 21:23 ` [PATCH v2 10/29] python/qemu: Add binutils::binary_get_qom_implementations() Philippe Mathieu-Daudé
2020-02-07 14:28 ` Liam Merwick
2020-02-07 14:33 ` Philippe Mathieu-Daudé
2020-01-29 21:23 ` [PATCH v2 11/29] python/qemu: Add binutils::binary_get_accels() Philippe Mathieu-Daudé
2020-02-07 14:35 ` Liam Merwick
2020-01-29 21:23 ` [PATCH v2 12/29] python/qemu/accel: Use binutils::binary_get_accels() Philippe Mathieu-Daudé
2020-02-07 14:37 ` Liam Merwick
2020-01-29 21:23 ` [PATCH v2 13/29] python/qemu: Add binutils::binary_get_devices() Philippe Mathieu-Daudé
2020-02-07 15:02 ` Liam Merwick
2020-01-29 21:23 ` [RFC PATCH v2 14/29] tests/acceptance: Use 'device' tags to check availability in QEMU binary Philippe Mathieu-Daudé
2020-01-29 21:23 ` [PATCH v2 15/29] tests: rename virtio_seg_max_adjust to virtio_check_params Philippe Mathieu-Daudé
2020-01-30 12:32 ` Cornelia Huck
2020-01-31 0:05 ` Philippe Mathieu-Daudé
2020-01-29 21:23 ` [PATCH v2 16/29] tests/acceptance/virtio_check_params: Only remove listed machines Philippe Mathieu-Daudé
2020-01-29 21:23 ` [PATCH v2 17/29] tests/acceptance/virtio_check_params: Improve exception logging Philippe Mathieu-Daudé
2020-02-06 19:54 ` Eduardo Habkost
2020-02-06 20:36 ` Philippe Mathieu-Daudé
2020-01-29 21:23 ` [PATCH v2 18/29] tests/acceptance/virtio_check_params: List machine being tested Philippe Mathieu-Daudé
2020-01-30 12:35 ` Cornelia Huck
2020-01-31 0:08 ` Philippe Mathieu-Daudé
2020-01-29 21:23 ` [PATCH v2 19/29] tests/acceptance/virtio_check_params: Only test Xen as superuser Philippe Mathieu-Daudé
2020-01-30 12:37 ` Cornelia Huck
2020-01-30 13:29 ` Philippe Mathieu-Daudé
2020-01-29 21:23 ` [PATCH v2 20/29] tests/acceptance/virtio_check_params: Skip test if arch is not supported Philippe Mathieu-Daudé
2020-01-29 21:23 ` [RFC PATCH v2 21/29] tests/acceptance/virtio_check_params: Kludge to skip tests on MIPS Philippe Mathieu-Daudé
2020-01-29 21:23 ` [PATCH v2 22/29] tests/acceptance/virtio_check_params: Support the s390x architecture Philippe Mathieu-Daudé
2020-01-29 21:23 ` [PATCH v2 23/29] tests/acceptance/virtio_check_params: Support the ppc architecture Philippe Mathieu-Daudé
2020-01-29 21:23 ` [PATCH v2 24/29] tests/acceptance/virtio_check_params: Default to -nodefaults Philippe Mathieu-Daudé
2020-02-06 16:01 ` Thomas Huth
2020-01-29 21:23 ` [PATCH v2 25/29] tests/acceptance/virtio_check_params: Require a virtio-scsi-pci device Philippe Mathieu-Daudé
2020-01-29 21:23 ` [PATCH v2 26/29] tests/acceptance/boot_linux_console: Do not use VGA on Clipper machine Philippe Mathieu-Daudé
2020-02-06 15:35 ` Thomas Huth
2020-01-29 21:23 ` [PATCH v2 27/29] tests/acceptance/migration: Default to -nodefaults Philippe Mathieu-Daudé
2020-02-06 16:03 ` Thomas Huth
2020-02-07 11:57 ` Wainer dos Santos Moschetta
2020-01-29 21:23 ` [PATCH v2 28/29] tests/acceptance/version: " Philippe Mathieu-Daudé
2020-02-06 15:34 ` Thomas Huth
2020-02-07 12:05 ` Wainer dos Santos Moschetta
2020-01-29 21:23 ` [PATCH v2 29/29] MAINTAINERS: Add Acceptance tests reviewers Philippe Mathieu-Daudé
2020-01-30 8:41 ` Thomas Huth
2020-01-31 13:46 ` Wainer dos Santos Moschetta
2020-01-29 21:28 ` [PATCH v2 00/29] tests/acceptance/virtio_seg_max_adjust: Restrict it to Linux/X86 Philippe Mathieu-Daudé
2020-02-06 16:48 ` Philippe Mathieu-Daudé
2020-06-04 8:19 ` python/qemu: Refactor QemuBinaryInfo Philippe Mathieu-Daudé
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200130180859.43954c22.cohuck@redhat.com \
--to=cohuck@redhat.com \
--cc=crosa@redhat.com \
--cc=dplotnikov@virtuozzo.com \
--cc=ehabkost@redhat.com \
--cc=philmd@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=wainersm@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).