Re: [RFC PATCH-for-8.2] .gitlab-ci.d/cirrus.yml: Promote NetBSD job as gating

["Daniel P. Berrangé" <berrange@redhat.com>]

On Fri, Nov 10, 2023 at 10:30:26AM +0100, Thomas Huth wrote:
> On 10/11/2023 10.22, Daniel P. Berrangé wrote:
> > On Thu, Nov 09, 2023 at 06:15:51PM +0100, Thomas Huth wrote:
> > > On 09/11/2023 17.58, Daniel P. Berrangé wrote:
> > > > On Thu, Nov 09, 2023 at 04:35:56PM +0100, Philippe Mathieu-Daudé wrote:
> > > > > On 9/11/23 16:35, Philippe Mathieu-Daudé wrote:
> > > > > > This Cirrus-CI based job takes ~12min, similarly to macOS job.
> > > > > > 
> > > > > > Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> > > > > > ---
> > > > > > Based-on: <20231109150900.91186-1-philmd@linaro.org>
> > > > > >              "tests/vm/netbsd: Use Python v3.11"
> > > > > > ---
> > > > > >     .gitlab-ci.d/cirrus.yml | 3 +--
> > > > > >     1 file changed, 1 insertion(+), 2 deletions(-)
> > > > > > 
> > > > > > diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
> > > > > > index e7f1f83c2c..7b01acb104 100644
> > > > > > --- a/.gitlab-ci.d/cirrus.yml
> > > > > > +++ b/.gitlab-ci.d/cirrus.yml
> > > > > > @@ -94,8 +94,6 @@ aarch64-macos-12-base-build:
> > > > > >         - cirrus-run -v --show-build-log always .gitlab-ci.d/cirrus/$NAME.yml
> > > > > >       variables:
> > > > > >         QEMU_JOB_CIRRUS: 1
> > > > > > -    QEMU_JOB_OPTIONAL: 1
> > > > > > -
> > > > > >     x86-netbsd:
> > > > > >       extends: .cirrus_kvm_job
> > > > > > @@ -110,3 +108,4 @@ x86-openbsd:
> > > > > >         NAME: openbsd
> > > > > >         CONFIGURE_ARGS: --target-list=i386-softmmu,riscv64-softmmu,mips64-softmmu
> > > > > >         TEST_TARGETS: check
> > > > > > +    QEMU_JOB_OPTIONAL: 1
> > > > > 
> > > > > BTW OpenBSD works for me, but takes ~20min (similar to the FreeBSD job).
> > > ...
> > > > I could have sworn our cirrus jobs were much slower in the past (around
> > > > the 40 min mark), as we had to cut down what we were running to avoid
> > > > frequent timeouts.
> > > 
> > > You're right, Daniel. Seems like both, the Cirrus netbsd and the openbsd job
> > > are currently broken and only output some help text instead of compiling
> > > QEMU:
> > > 
> > >   https://gitlab.com/philmd/qemu/-/jobs/5497861511#L6834
> > > 
> > > ... that's why the finish so fast.
> > > 
> > > IIRC last time I've seen them "working", they were running into the 80
> > > minute timeout again.
> > > 
> > > So the netbsd and openbsd job are indeed not very useful anymore. I think we
> > > should rather remove them and add a proper job via our own custom
> > > KVM-capable runners instead.
> > 
> > The CI job isn't the issue though - it is merely a sign of brokeness
> > elsewhere.  Either tests/vm/{netbsd,openbsd} are broken, or our entire
> > build process for those platforms is broken.
> > 
> > We need to root cause this, rather than hide it further by dropping
> > the CI jobs.
> 
> "make vm-build-netbsd" locally just works fine (as soon as Philippe's python
> fix gets merged). I just had another try with the cirrus-ci job, but it
> indeeds run into timeout issues again:
> 
>  https://gitlab.com/thuth/qemu/-/jobs/5501021556
> 
> I guess we could cut it down again by e.g. removing aarch64-softmmu from the
> target list ... but we then still have the problem that we can not run it by
> default due to the limitations of cirrus-ci only allowing to run 2 jobs in
> parallel. And as long as we don't run things by default, they apparently
> tend to bit-rot quite fast...

Right, even if we drop 1 target, with the other jobs we need to run in
Cirrus, it is still going to be too long. We would need it to be in the
30 min range maximum, to be viable running it by default I tink.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|