[RFC PATCH-for-8.2] .gitlab-ci.d/cirrus.yml: Promote NetBSD job as gating

[RFC PATCH-for-8.2] .gitlab-ci.d/cirrus.yml: Promote NetBSD job as gating

[Philippe Mathieu-Daudé]

This Cirrus-CI based job takes ~12min, similarly to macOS job.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
Based-on: <20231109150900.91186-1-philmd@linaro.org>
          "tests/vm/netbsd: Use Python v3.11"
---
 .gitlab-ci.d/cirrus.yml | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
index e7f1f83c2c..7b01acb104 100644
--- a/.gitlab-ci.d/cirrus.yml
+++ b/.gitlab-ci.d/cirrus.yml
@@ -94,8 +94,6 @@ aarch64-macos-12-base-build:
     - cirrus-run -v --show-build-log always .gitlab-ci.d/cirrus/$NAME.yml
   variables:
     QEMU_JOB_CIRRUS: 1
-    QEMU_JOB_OPTIONAL: 1
-
 
 x86-netbsd:
   extends: .cirrus_kvm_job
@@ -110,3 +108,4 @@ x86-openbsd:
     NAME: openbsd
     CONFIGURE_ARGS: --target-list=i386-softmmu,riscv64-softmmu,mips64-softmmu
     TEST_TARGETS: check
+    QEMU_JOB_OPTIONAL: 1
-- 
2.41.0

Re: [RFC PATCH-for-8.2] .gitlab-ci.d/cirrus.yml: Promote NetBSD job as gating

[Philippe Mathieu-Daudé]

On 9/11/23 16:35, Philippe Mathieu-Daudé wrote:
> This Cirrus-CI based job takes ~12min, similarly to macOS job.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
> Based-on: <20231109150900.91186-1-philmd@linaro.org>
>            "tests/vm/netbsd: Use Python v3.11"
> ---
>   .gitlab-ci.d/cirrus.yml | 3 +--
>   1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
> index e7f1f83c2c..7b01acb104 100644
> --- a/.gitlab-ci.d/cirrus.yml
> +++ b/.gitlab-ci.d/cirrus.yml
> @@ -94,8 +94,6 @@ aarch64-macos-12-base-build:
>       - cirrus-run -v --show-build-log always .gitlab-ci.d/cirrus/$NAME.yml
>     variables:
>       QEMU_JOB_CIRRUS: 1
> -    QEMU_JOB_OPTIONAL: 1
> -
>   
>   x86-netbsd:
>     extends: .cirrus_kvm_job
> @@ -110,3 +108,4 @@ x86-openbsd:
>       NAME: openbsd
>       CONFIGURE_ARGS: --target-list=i386-softmmu,riscv64-softmmu,mips64-softmmu
>       TEST_TARGETS: check
> +    QEMU_JOB_OPTIONAL: 1

BTW OpenBSD works for me, but takes ~20min (similar to the FreeBSD job).

Re: [RFC PATCH-for-8.2] .gitlab-ci.d/cirrus.yml: Promote NetBSD job as gating

[Daniel P. Berrangé]

On Thu, Nov 09, 2023 at 04:35:56PM +0100, Philippe Mathieu-Daudé wrote:
> On 9/11/23 16:35, Philippe Mathieu-Daudé wrote:
> > This Cirrus-CI based job takes ~12min, similarly to macOS job.
> > 
> > Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> > ---
> > Based-on: <20231109150900.91186-1-philmd@linaro.org>
> >            "tests/vm/netbsd: Use Python v3.11"
> > ---
> >   .gitlab-ci.d/cirrus.yml | 3 +--
> >   1 file changed, 1 insertion(+), 2 deletions(-)
> > 
> > diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
> > index e7f1f83c2c..7b01acb104 100644
> > --- a/.gitlab-ci.d/cirrus.yml
> > +++ b/.gitlab-ci.d/cirrus.yml
> > @@ -94,8 +94,6 @@ aarch64-macos-12-base-build:
> >       - cirrus-run -v --show-build-log always .gitlab-ci.d/cirrus/$NAME.yml
> >     variables:
> >       QEMU_JOB_CIRRUS: 1
> > -    QEMU_JOB_OPTIONAL: 1
> > -
> >   x86-netbsd:
> >     extends: .cirrus_kvm_job
> > @@ -110,3 +108,4 @@ x86-openbsd:
> >       NAME: openbsd
> >       CONFIGURE_ARGS: --target-list=i386-softmmu,riscv64-softmmu,mips64-softmmu
> >       TEST_TARGETS: check
> > +    QEMU_JOB_OPTIONAL: 1
> 
> BTW OpenBSD works for me, but takes ~20min (similar to the FreeBSD job).

In the most recent pipeline FreeBSD too 22 mins, macOS 14 mins.

The key factor is that the Cirrus job needs to complete before the
GitLab job times out. We have a 1 hr 20 limit on GitLab jobs, and
Cirrus CI allows 2 jobs in parallel.

So in the worst case where the OpenBSD job was blocked until FreeBSD
job finished, we would be waiting about 45 mins for completion.

In the best case we would be waiting 36 mins.

Well technically the worst case would be no parallelization at all,
which is 70 mins serialized execution time, which is kinda close to
the 1hr20 limit. This does sometimes happen, but I don't know how
often.

Sometimes Cirrus CI can also delay the jobs starting for a while
due to lack of runners.

IOW, in normal times we could afford to enable all these jobs, but
if Cirrus CI is under heavy load we increase chance of timeouts.

I could have sworn our cirrus jobs were much slower in the past (around
the 40 min mark), as we had to cut down what we were running to avoid
frequent timeouts.

I'd say lets wait until this release is done though. Enable OpenBSD/NetBSD
when the new dev cycle opens up, so we can watch their stablility for a
bit, and not impact stability of the stable branch for this forthcoming
release.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [RFC PATCH-for-8.2] .gitlab-ci.d/cirrus.yml: Promote NetBSD job as gating

[Thomas Huth]

On 09/11/2023 17.58, Daniel P. Berrangé wrote:
> On Thu, Nov 09, 2023 at 04:35:56PM +0100, Philippe Mathieu-Daudé wrote:
>> On 9/11/23 16:35, Philippe Mathieu-Daudé wrote:
>>> This Cirrus-CI based job takes ~12min, similarly to macOS job.
>>>
>>> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
>>> ---
>>> Based-on: <20231109150900.91186-1-philmd@linaro.org>
>>>             "tests/vm/netbsd: Use Python v3.11"
>>> ---
>>>    .gitlab-ci.d/cirrus.yml | 3 +--
>>>    1 file changed, 1 insertion(+), 2 deletions(-)
>>>
>>> diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
>>> index e7f1f83c2c..7b01acb104 100644
>>> --- a/.gitlab-ci.d/cirrus.yml
>>> +++ b/.gitlab-ci.d/cirrus.yml
>>> @@ -94,8 +94,6 @@ aarch64-macos-12-base-build:
>>>        - cirrus-run -v --show-build-log always .gitlab-ci.d/cirrus/$NAME.yml
>>>      variables:
>>>        QEMU_JOB_CIRRUS: 1
>>> -    QEMU_JOB_OPTIONAL: 1
>>> -
>>>    x86-netbsd:
>>>      extends: .cirrus_kvm_job
>>> @@ -110,3 +108,4 @@ x86-openbsd:
>>>        NAME: openbsd
>>>        CONFIGURE_ARGS: --target-list=i386-softmmu,riscv64-softmmu,mips64-softmmu
>>>        TEST_TARGETS: check
>>> +    QEMU_JOB_OPTIONAL: 1
>>
>> BTW OpenBSD works for me, but takes ~20min (similar to the FreeBSD job).
...
> I could have sworn our cirrus jobs were much slower in the past (around
> the 40 min mark), as we had to cut down what we were running to avoid
> frequent timeouts.

You're right, Daniel. Seems like both, the Cirrus netbsd and the openbsd job 
are currently broken and only output some help text instead of compiling QEMU:

  https://gitlab.com/philmd/qemu/-/jobs/5497861511#L6834

... that's why the finish so fast.

IIRC last time I've seen them "working", they were running into the 80 
minute timeout again.

So the netbsd and openbsd job are indeed not very useful anymore. I think we 
should rather remove them and add a proper job via our own custom 
KVM-capable runners instead.

  Thomas

Re: [RFC PATCH-for-8.2] .gitlab-ci.d/cirrus.yml: Promote NetBSD job as gating

[Philippe Mathieu-Daudé]

On 9/11/23 18:15, Thomas Huth wrote:
> On 09/11/2023 17.58, Daniel P. Berrangé wrote:
>> On Thu, Nov 09, 2023 at 04:35:56PM +0100, Philippe Mathieu-Daudé wrote:
>>> On 9/11/23 16:35, Philippe Mathieu-Daudé wrote:
>>>> This Cirrus-CI based job takes ~12min, similarly to macOS job.
>>>>
>>>> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
>>>> ---
>>>> Based-on: <20231109150900.91186-1-philmd@linaro.org>
>>>>             "tests/vm/netbsd: Use Python v3.11"
>>>> ---
>>>>    .gitlab-ci.d/cirrus.yml | 3 +--
>>>>    1 file changed, 1 insertion(+), 2 deletions(-)
>>>>
>>>> diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
>>>> index e7f1f83c2c..7b01acb104 100644
>>>> --- a/.gitlab-ci.d/cirrus.yml
>>>> +++ b/.gitlab-ci.d/cirrus.yml
>>>> @@ -94,8 +94,6 @@ aarch64-macos-12-base-build:
>>>>        - cirrus-run -v --show-build-log always 
>>>> .gitlab-ci.d/cirrus/$NAME.yml
>>>>      variables:
>>>>        QEMU_JOB_CIRRUS: 1
>>>> -    QEMU_JOB_OPTIONAL: 1
>>>> -
>>>>    x86-netbsd:
>>>>      extends: .cirrus_kvm_job
>>>> @@ -110,3 +108,4 @@ x86-openbsd:
>>>>        NAME: openbsd
>>>>        CONFIGURE_ARGS: 
>>>> --target-list=i386-softmmu,riscv64-softmmu,mips64-softmmu
>>>>        TEST_TARGETS: check
>>>> +    QEMU_JOB_OPTIONAL: 1
>>>
>>> BTW OpenBSD works for me, but takes ~20min (similar to the FreeBSD job).
> ...
>> I could have sworn our cirrus jobs were much slower in the past (around
>> the 40 min mark), as we had to cut down what we were running to avoid
>> frequent timeouts.
> 
> You're right, Daniel. Seems like both, the Cirrus netbsd and the openbsd 
> job are currently broken and only output some help text instead of 
> compiling QEMU:
> 
>   https://gitlab.com/philmd/qemu/-/jobs/5497861511#L6834
> 
> ... that's why the finish so fast.

Doh, I didn't noticed...

> IIRC last time I've seen them "working", they were running into the 80 
> minute timeout again.
> 
> So the netbsd and openbsd job are indeed not very useful anymore. I 
> think we should rather remove them and add a proper job via our own 
> custom KVM-capable runners instead.

Warner, Richard, OK with that?

Re: [RFC PATCH-for-8.2] .gitlab-ci.d/cirrus.yml: Promote NetBSD job as gating

[Daniel P. Berrangé]

On Thu, Nov 09, 2023 at 06:15:51PM +0100, Thomas Huth wrote:
> On 09/11/2023 17.58, Daniel P. Berrangé wrote:
> > On Thu, Nov 09, 2023 at 04:35:56PM +0100, Philippe Mathieu-Daudé wrote:
> > > On 9/11/23 16:35, Philippe Mathieu-Daudé wrote:
> > > > This Cirrus-CI based job takes ~12min, similarly to macOS job.
> > > > 
> > > > Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> > > > ---
> > > > Based-on: <20231109150900.91186-1-philmd@linaro.org>
> > > >             "tests/vm/netbsd: Use Python v3.11"
> > > > ---
> > > >    .gitlab-ci.d/cirrus.yml | 3 +--
> > > >    1 file changed, 1 insertion(+), 2 deletions(-)
> > > > 
> > > > diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
> > > > index e7f1f83c2c..7b01acb104 100644
> > > > --- a/.gitlab-ci.d/cirrus.yml
> > > > +++ b/.gitlab-ci.d/cirrus.yml
> > > > @@ -94,8 +94,6 @@ aarch64-macos-12-base-build:
> > > >        - cirrus-run -v --show-build-log always .gitlab-ci.d/cirrus/$NAME.yml
> > > >      variables:
> > > >        QEMU_JOB_CIRRUS: 1
> > > > -    QEMU_JOB_OPTIONAL: 1
> > > > -
> > > >    x86-netbsd:
> > > >      extends: .cirrus_kvm_job
> > > > @@ -110,3 +108,4 @@ x86-openbsd:
> > > >        NAME: openbsd
> > > >        CONFIGURE_ARGS: --target-list=i386-softmmu,riscv64-softmmu,mips64-softmmu
> > > >        TEST_TARGETS: check
> > > > +    QEMU_JOB_OPTIONAL: 1
> > > 
> > > BTW OpenBSD works for me, but takes ~20min (similar to the FreeBSD job).
> ...
> > I could have sworn our cirrus jobs were much slower in the past (around
> > the 40 min mark), as we had to cut down what we were running to avoid
> > frequent timeouts.
> 
> You're right, Daniel. Seems like both, the Cirrus netbsd and the openbsd job
> are currently broken and only output some help text instead of compiling
> QEMU:
> 
>  https://gitlab.com/philmd/qemu/-/jobs/5497861511#L6834
> 
> ... that's why the finish so fast.
> 
> IIRC last time I've seen them "working", they were running into the 80
> minute timeout again.
> 
> So the netbsd and openbsd job are indeed not very useful anymore. I think we
> should rather remove them and add a proper job via our own custom
> KVM-capable runners instead.

The CI job isn't the issue though - it is merely a sign of brokeness
elsewhere.  Either tests/vm/{netbsd,openbsd} are broken, or our entire
build process for those platforms is broken.

We need to root cause this, rather than hide it further by dropping
the CI jobs.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [RFC PATCH-for-8.2] .gitlab-ci.d/cirrus.yml: Promote NetBSD job as gating

[Thomas Huth]

On 10/11/2023 10.22, Daniel P. Berrangé wrote:
> On Thu, Nov 09, 2023 at 06:15:51PM +0100, Thomas Huth wrote:
>> On 09/11/2023 17.58, Daniel P. Berrangé wrote:
>>> On Thu, Nov 09, 2023 at 04:35:56PM +0100, Philippe Mathieu-Daudé wrote:
>>>> On 9/11/23 16:35, Philippe Mathieu-Daudé wrote:
>>>>> This Cirrus-CI based job takes ~12min, similarly to macOS job.
>>>>>
>>>>> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
>>>>> ---
>>>>> Based-on: <20231109150900.91186-1-philmd@linaro.org>
>>>>>              "tests/vm/netbsd: Use Python v3.11"
>>>>> ---
>>>>>     .gitlab-ci.d/cirrus.yml | 3 +--
>>>>>     1 file changed, 1 insertion(+), 2 deletions(-)
>>>>>
>>>>> diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
>>>>> index e7f1f83c2c..7b01acb104 100644
>>>>> --- a/.gitlab-ci.d/cirrus.yml
>>>>> +++ b/.gitlab-ci.d/cirrus.yml
>>>>> @@ -94,8 +94,6 @@ aarch64-macos-12-base-build:
>>>>>         - cirrus-run -v --show-build-log always .gitlab-ci.d/cirrus/$NAME.yml
>>>>>       variables:
>>>>>         QEMU_JOB_CIRRUS: 1
>>>>> -    QEMU_JOB_OPTIONAL: 1
>>>>> -
>>>>>     x86-netbsd:
>>>>>       extends: .cirrus_kvm_job
>>>>> @@ -110,3 +108,4 @@ x86-openbsd:
>>>>>         NAME: openbsd
>>>>>         CONFIGURE_ARGS: --target-list=i386-softmmu,riscv64-softmmu,mips64-softmmu
>>>>>         TEST_TARGETS: check
>>>>> +    QEMU_JOB_OPTIONAL: 1
>>>>
>>>> BTW OpenBSD works for me, but takes ~20min (similar to the FreeBSD job).
>> ...
>>> I could have sworn our cirrus jobs were much slower in the past (around
>>> the 40 min mark), as we had to cut down what we were running to avoid
>>> frequent timeouts.
>>
>> You're right, Daniel. Seems like both, the Cirrus netbsd and the openbsd job
>> are currently broken and only output some help text instead of compiling
>> QEMU:
>>
>>   https://gitlab.com/philmd/qemu/-/jobs/5497861511#L6834
>>
>> ... that's why the finish so fast.
>>
>> IIRC last time I've seen them "working", they were running into the 80
>> minute timeout again.
>>
>> So the netbsd and openbsd job are indeed not very useful anymore. I think we
>> should rather remove them and add a proper job via our own custom
>> KVM-capable runners instead.
> 
> The CI job isn't the issue though - it is merely a sign of brokeness
> elsewhere.  Either tests/vm/{netbsd,openbsd} are broken, or our entire
> build process for those platforms is broken.
> 
> We need to root cause this, rather than hide it further by dropping
> the CI jobs.

"make vm-build-netbsd" locally just works fine (as soon as Philippe's python 
fix gets merged). I just had another try with the cirrus-ci job, but it 
indeeds run into timeout issues again:

  https://gitlab.com/thuth/qemu/-/jobs/5501021556

I guess we could cut it down again by e.g. removing aarch64-softmmu from the 
target list ... but we then still have the problem that we can not run it by 
default due to the limitations of cirrus-ci only allowing to run 2 jobs in 
parallel. And as long as we don't run things by default, they apparently 
tend to bit-rot quite fast...

  Thomas

Re: [RFC PATCH-for-8.2] .gitlab-ci.d/cirrus.yml: Promote NetBSD job as gating

[Daniel P. Berrangé]

On Fri, Nov 10, 2023 at 10:30:26AM +0100, Thomas Huth wrote:
> On 10/11/2023 10.22, Daniel P. Berrangé wrote:
> > On Thu, Nov 09, 2023 at 06:15:51PM +0100, Thomas Huth wrote:
> > > On 09/11/2023 17.58, Daniel P. Berrangé wrote:
> > > > On Thu, Nov 09, 2023 at 04:35:56PM +0100, Philippe Mathieu-Daudé wrote:
> > > > > On 9/11/23 16:35, Philippe Mathieu-Daudé wrote:
> > > > > > This Cirrus-CI based job takes ~12min, similarly to macOS job.
> > > > > > 
> > > > > > Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> > > > > > ---
> > > > > > Based-on: <20231109150900.91186-1-philmd@linaro.org>
> > > > > >              "tests/vm/netbsd: Use Python v3.11"
> > > > > > ---
> > > > > >     .gitlab-ci.d/cirrus.yml | 3 +--
> > > > > >     1 file changed, 1 insertion(+), 2 deletions(-)
> > > > > > 
> > > > > > diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
> > > > > > index e7f1f83c2c..7b01acb104 100644
> > > > > > --- a/.gitlab-ci.d/cirrus.yml
> > > > > > +++ b/.gitlab-ci.d/cirrus.yml
> > > > > > @@ -94,8 +94,6 @@ aarch64-macos-12-base-build:
> > > > > >         - cirrus-run -v --show-build-log always .gitlab-ci.d/cirrus/$NAME.yml
> > > > > >       variables:
> > > > > >         QEMU_JOB_CIRRUS: 1
> > > > > > -    QEMU_JOB_OPTIONAL: 1
> > > > > > -
> > > > > >     x86-netbsd:
> > > > > >       extends: .cirrus_kvm_job
> > > > > > @@ -110,3 +108,4 @@ x86-openbsd:
> > > > > >         NAME: openbsd
> > > > > >         CONFIGURE_ARGS: --target-list=i386-softmmu,riscv64-softmmu,mips64-softmmu
> > > > > >         TEST_TARGETS: check
> > > > > > +    QEMU_JOB_OPTIONAL: 1
> > > > > 
> > > > > BTW OpenBSD works for me, but takes ~20min (similar to the FreeBSD job).
> > > ...
> > > > I could have sworn our cirrus jobs were much slower in the past (around
> > > > the 40 min mark), as we had to cut down what we were running to avoid
> > > > frequent timeouts.
> > > 
> > > You're right, Daniel. Seems like both, the Cirrus netbsd and the openbsd job
> > > are currently broken and only output some help text instead of compiling
> > > QEMU:
> > > 
> > >   https://gitlab.com/philmd/qemu/-/jobs/5497861511#L6834
> > > 
> > > ... that's why the finish so fast.
> > > 
> > > IIRC last time I've seen them "working", they were running into the 80
> > > minute timeout again.
> > > 
> > > So the netbsd and openbsd job are indeed not very useful anymore. I think we
> > > should rather remove them and add a proper job via our own custom
> > > KVM-capable runners instead.
> > 
> > The CI job isn't the issue though - it is merely a sign of brokeness
> > elsewhere.  Either tests/vm/{netbsd,openbsd} are broken, or our entire
> > build process for those platforms is broken.
> > 
> > We need to root cause this, rather than hide it further by dropping
> > the CI jobs.
> 
> "make vm-build-netbsd" locally just works fine (as soon as Philippe's python
> fix gets merged). I just had another try with the cirrus-ci job, but it
> indeeds run into timeout issues again:
> 
>  https://gitlab.com/thuth/qemu/-/jobs/5501021556
> 
> I guess we could cut it down again by e.g. removing aarch64-softmmu from the
> target list ... but we then still have the problem that we can not run it by
> default due to the limitations of cirrus-ci only allowing to run 2 jobs in
> parallel. And as long as we don't run things by default, they apparently
> tend to bit-rot quite fast...

Right, even if we drop 1 target, with the other jobs we need to run in
Cirrus, it is still going to be too long. We would need it to be in the
30 min range maximum, to be viable running it by default I tink.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [RFC PATCH-for-8.2] .gitlab-ci.d/cirrus.yml: Promote NetBSD job as gating

[Reinoud Zandijk]

On Thu, Nov 09, 2023 at 06:15:51PM +0100, Thomas Huth wrote:
> On 09/11/2023 17.58, Daniel P. Berrangé wrote:
> > On Thu, Nov 09, 2023 at 04:35:56PM +0100, Philippe Mathieu-Daudé wrote:
> > > On 9/11/23 16:35, Philippe Mathieu-Daudé wrote:
> > > > This Cirrus-CI based job takes ~12min, similarly to macOS job.
> > > > 
> > > > Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> > > > ---
> > > > Based-on: <20231109150900.91186-1-philmd@linaro.org>
> > > >             "tests/vm/netbsd: Use Python v3.11"
> > > > ---
> > > >    .gitlab-ci.d/cirrus.yml | 3 +--
> > > >    1 file changed, 1 insertion(+), 2 deletions(-)
> > > > 
> > > > diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
> > > > index e7f1f83c2c..7b01acb104 100644
> > > > --- a/.gitlab-ci.d/cirrus.yml
> > > > +++ b/.gitlab-ci.d/cirrus.yml
> > > > @@ -94,8 +94,6 @@ aarch64-macos-12-base-build:
> > > >        - cirrus-run -v --show-build-log always .gitlab-ci.d/cirrus/$NAME.yml
> > > >      variables:
> > > >        QEMU_JOB_CIRRUS: 1
> > > > -    QEMU_JOB_OPTIONAL: 1
> > > > -
> > > >    x86-netbsd:
> > > >      extends: .cirrus_kvm_job
> > > > @@ -110,3 +108,4 @@ x86-openbsd:
> > > >        NAME: openbsd
> > > >        CONFIGURE_ARGS: --target-list=i386-softmmu,riscv64-softmmu,mips64-softmmu
> > > >        TEST_TARGETS: check
> > > > +    QEMU_JOB_OPTIONAL: 1
> > > 
> > > BTW OpenBSD works for me, but takes ~20min (similar to the FreeBSD job).
> ...
> > I could have sworn our cirrus jobs were much slower in the past (around
> > the 40 min mark), as we had to cut down what we were running to avoid
> > frequent timeouts.
> 
> You're right, Daniel. Seems like both, the Cirrus netbsd and the openbsd job
> are currently broken and only output some help text instead of compiling
> QEMU:
> 
>  https://gitlab.com/philmd/qemu/-/jobs/5497861511#L6834
> 
> ... that's why the finish so fast.
> 
> IIRC last time I've seen them "working", they were running into the 80
> minute timeout again.
> 
> So the netbsd and openbsd job are indeed not very useful anymore. I think we
> should rather remove them and add a proper job via our own custom
> KVM-capable runners instead.

Even though I am a co-maintainer of the NetBSD support for Qemu I am not quite
sure what testcase this is. Is this a regression test of installing NetBSD
from an ISO? That somehow times out? Where can I find the resulting console
output? Maybe the installer changed?

With regards,
Reinoud

Re: [RFC PATCH-for-8.2] .gitlab-ci.d/cirrus.yml: Promote NetBSD job as gating

[Reinoud Zandijk]

On Fri, Nov 10, 2023 at 10:12:38PM +0100, Reinoud Zandijk wrote:
> On Thu, Nov 09, 2023 at 06:15:51PM +0100, Thomas Huth wrote:
> > On 09/11/2023 17.58, Daniel P. Berrangé wrote:
> > > On Thu, Nov 09, 2023 at 04:35:56PM +0100, Philippe Mathieu-Daudé wrote:
...
> > You're right, Daniel. Seems like both, the Cirrus netbsd and the openbsd job
> > are currently broken and only output some help text instead of compiling
> > QEMU:
> > 
> >  https://gitlab.com/philmd/qemu/-/jobs/5497861511#L6834
> > 
> > ... that's why the finish so fast.
> > 
> > IIRC last time I've seen them "working", they were running into the 80
> > minute timeout again.
> > 
> > So the netbsd and openbsd job are indeed not very useful anymore. I think we
> > should rather remove them and add a proper job via our own custom
> > KVM-capable runners instead.
> 
> Even though I am a co-maintainer of the NetBSD support for Qemu I am not quite
> sure what testcase this is. Is this a regression test of installing NetBSD
> from an ISO? That somehow times out? Where can I find the resulting console
> output? Maybe the installer changed?

Re-reading the thread its about compiling Qemu on NetBSD. Doh. I am a novice
to the test kit you use so please forgive me if I don't make sense. Am I right
that it does install NetBSD OK, it then comes up and then tries to compile
Qemu on it but it fails due to some Python errors in the test script? Does it
use NetBSDs pkgsrc with its patches or has it its own method of dealing with
them?

With regards,
Reinoud

Re: [RFC PATCH-for-8.2] .gitlab-ci.d/cirrus.yml: Promote NetBSD job as gating

[Thomas Huth]

On 11/11/2023 18.33, Reinoud Zandijk wrote:
> On Fri, Nov 10, 2023 at 10:12:38PM +0100, Reinoud Zandijk wrote:
>> On Thu, Nov 09, 2023 at 06:15:51PM +0100, Thomas Huth wrote:
>>> On 09/11/2023 17.58, Daniel P. Berrangé wrote:
>>>> On Thu, Nov 09, 2023 at 04:35:56PM +0100, Philippe Mathieu-Daudé wrote:
> ...
>>> You're right, Daniel. Seems like both, the Cirrus netbsd and the openbsd job
>>> are currently broken and only output some help text instead of compiling
>>> QEMU:
>>>
>>>   https://gitlab.com/philmd/qemu/-/jobs/5497861511#L6834
>>>
>>> ... that's why the finish so fast.
>>>
>>> IIRC last time I've seen them "working", they were running into the 80
>>> minute timeout again.
>>>
>>> So the netbsd and openbsd job are indeed not very useful anymore. I think we
>>> should rather remove them and add a proper job via our own custom
>>> KVM-capable runners instead.
>>
>> Even though I am a co-maintainer of the NetBSD support for Qemu I am not quite
>> sure what testcase this is. Is this a regression test of installing NetBSD
>> from an ISO? That somehow times out? Where can I find the resulting console
>> output? Maybe the installer changed?
> 
> Re-reading the thread its about compiling Qemu on NetBSD. Doh. I am a novice
> to the test kit you use so please forgive me if I don't make sense. Am I right
> that it does install NetBSD OK, it then comes up and then tries to compile
> Qemu on it but it fails due to some Python errors in the test script? Does it
> use NetBSDs pkgsrc with its patches or has it its own method of dealing with
> them?

No worries, the "make vm-build-netbsd" test itself is just working fine 
(after applying Philippe's fix from here: 
https://lore.kernel.org/qemu-devel/20231109150900.91186-1-philmd@linaro.org/ ).

This thread here is about the CI job that could run in the gitlab-CI... it's 
got a quite complicated setup - NetBSD is running as KVM guest on a runner 
on cirrus-ci.com, and that whole thing is triggered by a gitlab-CI job - so 
this setup is often broken and thus does not run by default in the CI.
That's why I suggested to remove the job and replace it by a job that 
directly runs in a KVM-capable runner on the gitlab-CI instead of taking the 
detour via cirrus-ci.com.

  Thomas