Re: [PATCH v3 4/4] [NOT FOR MERGE] tests/qtest/migration: Adapt tests to use older QEMUs

[Fabiano Rosas <farosas@suse.de>]

Peter Xu <peterx@redhat.com> writes:

> On Fri, Jan 05, 2024 at 03:04:49PM -0300, Fabiano Rosas wrote:
>> [This patch is not necessary anymore after 8.2 has been released]
>> 
>> Add the 'since' annotations to recently added tests and adapt the
>> postcopy test to use the older "uri" API when needed.
>> 
>> Signed-off-by: Fabiano Rosas <farosas@suse.de>
>
> You marked this as not-for-merge.  Would something like this still be
> useful in the future?  IIUC it's a matter of whether we'd still want to
> test those old binaries.
>

Technically yes, but I fail to see what benefit testing old binaries
would bring us. I'm thinking maybe it could be useful for bisecting
compatibility issues, but I can't think of a scenario where we'd like to
change the older QEMU instead of the newer.

I'm of course open to suggestions if you or anyone else has an use case
that you'd like to keep viable.

So far, my idea is that once a new QEMU is released, all the "since:"
annotations become obsolete. We could even remove them. This series is
just infrastructure to make our life easier if a change is ever
introduced that is incompatible with the n-1 QEMU. IMO we cannot have
compatibility testing if a random change might break a test and make it
more difficult to run the remaining tests. So we'd use 'since' or the
vercmp function to skip/adapt the offending tests until the next QEMU is
released.

I'm basing myself on this loosely worded support statement from our
docs:

  "In general QEMU tries to maintain forward migration compatibility
  (i.e. migrating from QEMU n->n+1) and there are users who benefit from
  backward compatibility as well."