Re: [PATCH for-7.0] gitlab-ci: Add cirrus-ci based tests for NetBSD and OpenBSD

[Thomas Huth <thuth@redhat.com>]

On 10/12/2021 13.10, Daniel P. Berrangé wrote:
> On Thu, Dec 09, 2021 at 11:31:24AM +0100, Thomas Huth wrote:
>> Cirrus-CI provides KVM in their Linux containers, so we can also run
>> our VM-based NetBSD and OpenBSD build jobs there.
>> Since the VM installation might take a while, we only run the "help"
>> target on the first invocation to avoid timeouts, and then only check
>> the build during the next run, once the base image has been cached.
>> For the the build tests, we also only use very a limited set of target
>> CPUs since compiling in these VMs is not very fast (especially the
>> build on OpenBSD seems to be incredibly slow).
>> For the time being, the jobs are also marked as manually only, since
>> this double-indirect setup (with the cirrus-run script and VMs in
>> the Cirrus-CI containers) might fail more often than the other jobs.
> 
> I think they'll have to be manual forever basically unless
> something changes in cirrus.
> 
> Historically we've had trouble with the cirrus jobs timing out.
> This was ultimately a result of the fact that only 2 cirrus jobs
> can run concurrently, and we had duplicate jobs being scheduled
> on 'master' and 'staging'. This resulted in 4 jobs being queued
> and most of the time, and because each job took > 30 minutes,
> two of them would frequently hit the gitlab job 1 hour timeout.
> 
> Unless we can ensure that /all/ our cirrus jobs will reliably
> completed in about 20 minutes in normal case (30 mins if
> cirrus is being slow), then we can't have more than 2 cirrus
> jobs as one or more will end up going over the 1 hour cutoff.
> 
> The idea of having NetBSD/OpenBSD jobs is good, but I think
> it feels like a case where we're going to need to look at
> using custom runners if we want them trigger on 'staging'.
> 
> Manual jobs could be ok for contributors forks at most.

Agreed. I'll rephrase the last paragraph a little bit:

The jobs are marked as "manual" only, since this double-indirect setup (with 
the cirrus-run script and VMs in the Cirrus-CI containers) might fail more 
often than the other jobs, and since we can trigger a limited amount of 
Cirrus-CI jobs at a time anyway (due to the restrictions in the free tier of 
Cirrus). Thus these jobs are rather added as convenience for contributors 
who would like to run the NetBSD/OpenBSD tests without the need of 
downloading and installing the corresponding VM images on their local machines.

  Thomas