Re: [PULL 00/17] CI job updates, header cleanups and other misc patches

[Fabiano Rosas <farosas@suse.de>]

Daniel P. Berrangé <berrange@redhat.com> writes:

> On Thu, Apr 25, 2024 at 07:11:41AM +0200, Thomas Huth wrote:
>> On 24/04/2024 18.21, Richard Henderson wrote:
>> > On 4/24/24 00:57, Thomas Huth wrote:
>> > > The following changes since commit 13b1e9667737132440f4d500c31cb69320c6b15a:
>> > > 
>> > >    Merge tag 'for-upstream' of https://gitlab.com/bonzini/qemu into
>> > > staging (2024-04-23 17:35:57 -0700)
>> > > 
>> > > are available in the Git repository at:
>> > > 
>> > >    https://gitlab.com/thuth/qemu.git tags/pull-request-2024-04-24
>> > > 
>> > > for you to fetch changes up to 8f29bab03ea22694a127ee33edeb4ce99eeb124e:
>> > > 
>> > >    target/s390x: Remove KVM stubs in cpu_models.h (2024-04-24 09:45:02 +0200)
>> > > 
>> > > ----------------------------------------------------------------
>> > > * Update OpenBSD CI image to 7.5
>> > > * Update/remove Ubuntu 20.04 CI jobs
>> > > * Update CentOS 8 CI jobs to CentOS 9
>> > > * Some clean-ups and improvements to travis.yml
>> > > * Minor test fixes
>> > > * s390x header clean-ups
>> > > * Doc updates
>> > 
>> > This introduces a failure in the migration-compat-x86_64 job:
>> > 
>> > https://gitlab.com/qemu-project/qemu/-/jobs/6707154868
>> 
>> It wasn't failing for me:
>> 
>>  https://gitlab.com/thuth/qemu/-/jobs/6702058896
>> 
>> And according to the diffstat of my pull request, it's only touching test
>> files, docs, and s390x stuff, so I somehow fail to see how it could
>> influence x86 migration at a first glance. It also looks like the job is
>> running on opensuse, and not on CentOS or Ubuntu, so it should likely not be
>> influenced by the changes in this PR.
>> 
>> Could you please hit the re-run button of that job? If it then passes, we're
>> likely rather facing an intermitted failure that might have been introduced
>> earlier already...
>
> The specific job that's failiing is the multifd cancellation test.
> This is exactly the kind of functional area of migration where
> non-deterministic failures are likely to appear. So one for the
> migration maintainers to look at most likely.
>

Yes, that's the test that catches any race conditions on
error/cleanup. I agree that the PR here doesn't seem to interact with
any of that. The previous PR from Peter Xu has code that changes error
paths, so it's more likely to be it. The recent changes to the migration
tests are out of the picture as well because that CI job uses the
previous QEMU version's tests (i.e. 9.0).

I tried to reproduce it and no luck so far. Usually these bugs reproduce
well on my machine. I'll probably need to leave it running overnight to
find the race.