[PATCH] .gitlab-ci.d/cirrus: Remove the netbsd and openbsd jobs

[PATCH] .gitlab-ci.d/cirrus: Remove the netbsd and openbsd jobs

[Thomas Huth]

During the past months, the netbsd and openbsd jobs in the Cirrus-CI
were broken most of the time - the setup to run a BSD in KVM on Cirrus-CI
from gitlab via the cirrus-run script was very fragile, and since the
jobs were not run by default, it used to bitrot very fast.

Now Cirrus-CI also introduce a limit on the amount of free CI minutes
that you get there, so it is not appealing at all anymore to run
these BSDs in this setup - it's better to run the checks locally via
"make vm-build-openbsd" and "make vm-build-netbsd" instead. Thus let's
remove these CI jobs now.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.d/cirrus.yml           | 37 -------------------------------
 .gitlab-ci.d/cirrus/kvm-build.yml | 31 --------------------------
 2 files changed, 68 deletions(-)
 delete mode 100644 .gitlab-ci.d/cirrus/kvm-build.yml

diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
index 4671f069c3..74de2edbb4 100644
--- a/.gitlab-ci.d/cirrus.yml
+++ b/.gitlab-ci.d/cirrus.yml
@@ -89,40 +89,3 @@ aarch64-macos-14-base-build:
     PKG_CONFIG_PATH: /opt/homebrew/curl/lib/pkgconfig:/opt/homebrew/ncurses/lib/pkgconfig:/opt/homebrew/readline/lib/pkgconfig
     TEST_TARGETS: check-unit check-block check-qapi-schema check-softfloat check-qtest-x86_64
     QEMU_JOB_OPTIONAL: 1
-
-
-# The following jobs run VM-based tests via KVM on a Linux-based Cirrus-CI job
-.cirrus_kvm_job:
-  extends: .base_job_template
-  stage: build
-  image: registry.gitlab.com/libvirt/libvirt-ci/cirrus-run:master
-  needs: []
-  timeout: 80m
-  script:
-    - sed -e "s|[@]CI_REPOSITORY_URL@|$CI_REPOSITORY_URL|g"
-          -e "s|[@]CI_COMMIT_REF_NAME@|$CI_COMMIT_REF_NAME|g"
-          -e "s|[@]CI_COMMIT_SHA@|$CI_COMMIT_SHA|g"
-          -e "s|[@]NAME@|$NAME|g"
-          -e "s|[@]CONFIGURE_ARGS@|$CONFIGURE_ARGS|g"
-          -e "s|[@]TEST_TARGETS@|$TEST_TARGETS|g"
-      <.gitlab-ci.d/cirrus/kvm-build.yml >.gitlab-ci.d/cirrus/$NAME.yml
-    - cat .gitlab-ci.d/cirrus/$NAME.yml
-    - cirrus-run -v --show-build-log always .gitlab-ci.d/cirrus/$NAME.yml
-  variables:
-    QEMU_JOB_CIRRUS: 1
-    QEMU_JOB_OPTIONAL: 1
-
-
-x86-netbsd:
-  extends: .cirrus_kvm_job
-  variables:
-    NAME: netbsd
-    CONFIGURE_ARGS: --target-list=x86_64-softmmu,ppc64-softmmu,aarch64-softmmu
-    TEST_TARGETS: check
-
-x86-openbsd:
-  extends: .cirrus_kvm_job
-  variables:
-    NAME: openbsd
-    CONFIGURE_ARGS: --target-list=i386-softmmu,riscv64-softmmu,mips64-softmmu
-    TEST_TARGETS: check
diff --git a/.gitlab-ci.d/cirrus/kvm-build.yml b/.gitlab-ci.d/cirrus/kvm-build.yml
deleted file mode 100644
index a93881aa8b..0000000000
--- a/.gitlab-ci.d/cirrus/kvm-build.yml
+++ /dev/null
@@ -1,31 +0,0 @@
-container:
-  image: fedora:35
-  cpu: 4
-  memory: 8Gb
-  kvm: true
-
-env:
-  CIRRUS_CLONE_DEPTH: 1
-  CI_REPOSITORY_URL: "@CI_REPOSITORY_URL@"
-  CI_COMMIT_REF_NAME: "@CI_COMMIT_REF_NAME@"
-  CI_COMMIT_SHA: "@CI_COMMIT_SHA@"
-
-@NAME@_task:
-  @NAME@_vm_cache:
-    folder: $HOME/.cache/qemu-vm
-  install_script:
-    - dnf update -y
-    - dnf install -y git make openssh-clients qemu-img qemu-system-x86 wget meson
-  clone_script:
-    - git clone --depth 100 "$CI_REPOSITORY_URL" .
-    - git fetch origin "$CI_COMMIT_REF_NAME"
-    - git reset --hard "$CI_COMMIT_SHA"
-  build_script:
-    - if [ -f $HOME/.cache/qemu-vm/images/@NAME@.img ]; then
-        make vm-build-@NAME@ J=$(getconf _NPROCESSORS_ONLN)
-          EXTRA_CONFIGURE_OPTS="@CONFIGURE_ARGS@"
-          BUILD_TARGET="@TEST_TARGETS@" ;
-      else
-        make vm-build-@NAME@ J=$(getconf _NPROCESSORS_ONLN) BUILD_TARGET=help
-          EXTRA_CONFIGURE_OPTS="--disable-system --disable-user --disable-tools" ;
-      fi
-- 
2.44.0

Re: [PATCH] .gitlab-ci.d/cirrus: Remove the netbsd and openbsd jobs

[Peter Maydell]

On Fri, 26 Apr 2024 at 12:38, Thomas Huth <thuth@redhat.com> wrote:
>
> During the past months, the netbsd and openbsd jobs in the Cirrus-CI
> were broken most of the time - the setup to run a BSD in KVM on Cirrus-CI
> from gitlab via the cirrus-run script was very fragile, and since the
> jobs were not run by default, it used to bitrot very fast.
>
> Now Cirrus-CI also introduce a limit on the amount of free CI minutes
> that you get there, so it is not appealing at all anymore to run
> these BSDs in this setup - it's better to run the checks locally via
> "make vm-build-openbsd" and "make vm-build-netbsd" instead. Thus let's
> remove these CI jobs now.

So what's the plan to keep BSD CI coverage? This seems
like a step backwards towards "the person handling the
pullreq merges has to do some local private ad-hoc testing
too" :-(

thanks
-- PMM

Re: [PATCH] .gitlab-ci.d/cirrus: Remove the netbsd and openbsd jobs

[Thomas Huth]

On 26/04/2024 14.30, Peter Maydell wrote:
> On Fri, 26 Apr 2024 at 12:38, Thomas Huth <thuth@redhat.com> wrote:
>>
>> During the past months, the netbsd and openbsd jobs in the Cirrus-CI
>> were broken most of the time - the setup to run a BSD in KVM on Cirrus-CI
>> from gitlab via the cirrus-run script was very fragile, and since the
>> jobs were not run by default, it used to bitrot very fast.
>>
>> Now Cirrus-CI also introduce a limit on the amount of free CI minutes
>> that you get there, so it is not appealing at all anymore to run
>> these BSDs in this setup - it's better to run the checks locally via
>> "make vm-build-openbsd" and "make vm-build-netbsd" instead. Thus let's
>> remove these CI jobs now.
> 
> So what's the plan to keep BSD CI coverage? This seems
> like a step backwards towards "the person handling the
> pullreq merges has to do some local private ad-hoc testing
> too" :-(

With regards to NetBSD and OpenBSD, this is not a step backward since these 
gitlab jobs were never run anyway (they could only be triggered manually, 
but hardly anybody did that AFAIK).

If we want to have proper support for those OSes, I think somebody would 
need to set up a custom runner on a beefy KVM-capable server somewhere where 
we could run the "make vm-build-*bsd" commands. By the way, are Eldon's CI 
runners still around? IIRC they were capable of running KVM ?

  Thomas

Re: [PATCH] .gitlab-ci.d/cirrus: Remove the netbsd and openbsd jobs

[Eldon Stegall]

On Fri, Apr 26, 2024 at 02:47:20PM +0200, Thomas Huth wrote:
> With regards to NetBSD and OpenBSD, this is not a step backward since these 
> gitlab jobs were never run anyway (they could only be triggered manually, 
> but hardly anybody did that AFAIK).
> 
> If we want to have proper support for those OSes, I think somebody would 
> need to set up a custom runner on a beefy KVM-capable server somewhere where 
> we could run the "make vm-build-*bsd" commands. By the way, are Eldon's CI 
> runners still around? IIRC they were capable of running KVM ?

My datacenter had a power outage recently, so I disable my runner, and
haven't prioritized bringing it back up until now. I am glad to get this
going again, I'll look at it this weekend. 

It is a dedicated machine (R610 or Cisco UCS C220, I don't quite recall)
with I believe ~2X cores and 100+ GB memory. I hit a couple of minor
hurdles when CI started utilizing kubernetes runners, so one question I
had was whether I should install a lightweight kubernetes system and run
the gitlab kubernetes CI operator or the normal gitlab runner service. 

There should also be plenty of space to build *bsd VM's. Do pre-existing
upstream BSD images have an nocloud support so that we can build from a
stable updated base? Sorry I'm not super familiar with the BSD
ecosystems, but happy to try to fill in the gaps.

Thanks,
Eldon

Re: [PATCH] .gitlab-ci.d/cirrus: Remove the netbsd and openbsd jobs

[Thomas Huth]

On 26/04/2024 15.46, Eldon Stegall wrote:
> On Fri, Apr 26, 2024 at 02:47:20PM +0200, Thomas Huth wrote:
>> With regards to NetBSD and OpenBSD, this is not a step backward since these
>> gitlab jobs were never run anyway (they could only be triggered manually,
>> but hardly anybody did that AFAIK).
>>
>> If we want to have proper support for those OSes, I think somebody would
>> need to set up a custom runner on a beefy KVM-capable server somewhere where
>> we could run the "make vm-build-*bsd" commands. By the way, are Eldon's CI
>> runners still around? IIRC they were capable of running KVM ?
> 
> My datacenter had a power outage recently, so I disable my runner, and
> haven't prioritized bringing it back up until now. I am glad to get this
> going again, I'll look at it this weekend.

It's not for me to decide, but IMHO it would be a great possibility to run 
some additional KVM-based tests (like the vm-build-*bsd tests) in QEMU's CI!

> There should also be plenty of space to build *bsd VM's. Do pre-existing
> upstream BSD images have an nocloud support so that we can build from a
> stable updated base? Sorry I'm not super familiar with the BSD
> ecosystems, but happy to try to fill in the gaps.

QEMU's test suite comes with a handy way of doing tests on OpenBSD, NetBSD 
and FreeBSD: If you've got a KVM-capable Linux host, you just have to type 
"make vm-build-freebsd J=$(nproc)" to build and test the QEMU sources in a 
FreeBSD VM. It will automatically fetch and install a VM image for you.

  Thomas