[PATCH] .gitlab-ci.d/cirrus.yml: Shorten the runtime of the macOS and FreeBSD jobs

[PATCH] .gitlab-ci.d/cirrus.yml: Shorten the runtime of the macOS and FreeBSD jobs

[Thomas Huth]

Cirrus-CI introduced limitations to the free CI minutes. To avoid that
we are consuming them too fast, let's drop the usual targets that are
not that important since they are either a subset of another target
(like i386 or ppc being a subset of x86_64 or ppc64 respectively), or
since there is still a similar target with the opposite endianness
(like xtensa/xtensael, microblaze/microblazeel etc.).

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.d/cirrus.yml | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
index 74de2edbb4..75df1273bc 100644
--- a/.gitlab-ci.d/cirrus.yml
+++ b/.gitlab-ci.d/cirrus.yml
@@ -57,6 +57,7 @@ x64-freebsd-13-build:
     CIRRUS_VM_RAM: 8G
     UPDATE_COMMAND: pkg update; pkg upgrade -y
     INSTALL_COMMAND: pkg install -y
+    CONFIGURE_ARGS: --target-list-exclude=arm-softmmu,i386-softmmu,microblaze-softmmu,mips64el-softmmu,mipsel-softmmu,mips-softmmu,ppc-softmmu,sh4eb-softmmu,xtensa-softmmu
     TEST_TARGETS: check
 
 aarch64-macos-13-base-build:
@@ -72,6 +73,7 @@ aarch64-macos-13-base-build:
     INSTALL_COMMAND: brew install
     PATH_EXTRA: /opt/homebrew/ccache/libexec:/opt/homebrew/gettext/bin
     PKG_CONFIG_PATH: /opt/homebrew/curl/lib/pkgconfig:/opt/homebrew/ncurses/lib/pkgconfig:/opt/homebrew/readline/lib/pkgconfig
+    CONFIGURE_ARGS: --target-list-exclude=arm-softmmu,i386-softmmu,microblazeel-softmmu,mips64-softmmu,mipsel-softmmu,mips-softmmu,ppc-softmmu,sh4-softmmu,xtensaeb-softmmu
     TEST_TARGETS: check-unit check-block check-qapi-schema check-softfloat check-qtest-x86_64
 
 aarch64-macos-14-base-build:
-- 
2.44.0

Re: [PATCH] .gitlab-ci.d/cirrus.yml: Shorten the runtime of the macOS and FreeBSD jobs

[Thomas Huth]

Looks like the reduction of this patch was not enough, we've run out of 
Cirrus-CI compute time again ... does anybody have additional ideas how we 
could avoid that in the future?

  Thomas


On 29/04/2024 12.01, Thomas Huth wrote:
> Cirrus-CI introduced limitations to the free CI minutes. To avoid that
> we are consuming them too fast, let's drop the usual targets that are
> not that important since they are either a subset of another target
> (like i386 or ppc being a subset of x86_64 or ppc64 respectively), or
> since there is still a similar target with the opposite endianness
> (like xtensa/xtensael, microblaze/microblazeel etc.).
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>   .gitlab-ci.d/cirrus.yml | 2 ++
>   1 file changed, 2 insertions(+)
> 
> diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
> index 74de2edbb4..75df1273bc 100644
> --- a/.gitlab-ci.d/cirrus.yml
> +++ b/.gitlab-ci.d/cirrus.yml
> @@ -57,6 +57,7 @@ x64-freebsd-13-build:
>       CIRRUS_VM_RAM: 8G
>       UPDATE_COMMAND: pkg update; pkg upgrade -y
>       INSTALL_COMMAND: pkg install -y
> +    CONFIGURE_ARGS: --target-list-exclude=arm-softmmu,i386-softmmu,microblaze-softmmu,mips64el-softmmu,mipsel-softmmu,mips-softmmu,ppc-softmmu,sh4eb-softmmu,xtensa-softmmu
>       TEST_TARGETS: check
>   
>   aarch64-macos-13-base-build:
> @@ -72,6 +73,7 @@ aarch64-macos-13-base-build:
>       INSTALL_COMMAND: brew install
>       PATH_EXTRA: /opt/homebrew/ccache/libexec:/opt/homebrew/gettext/bin
>       PKG_CONFIG_PATH: /opt/homebrew/curl/lib/pkgconfig:/opt/homebrew/ncurses/lib/pkgconfig:/opt/homebrew/readline/lib/pkgconfig
> +    CONFIGURE_ARGS: --target-list-exclude=arm-softmmu,i386-softmmu,microblazeel-softmmu,mips64-softmmu,mipsel-softmmu,mips-softmmu,ppc-softmmu,sh4-softmmu,xtensaeb-softmmu
>       TEST_TARGETS: check-unit check-block check-qapi-schema check-softfloat check-qtest-x86_64
>   
>   aarch64-macos-14-base-build:

Re: [PATCH] .gitlab-ci.d/cirrus.yml: Shorten the runtime of the macOS and FreeBSD jobs

[Daniel P. Berrangé]

On Fri, Jul 26, 2024 at 11:18:43AM +0200, Thomas Huth wrote:
> 
> Looks like the reduction of this patch was not enough, we've run out of
> Cirrus-CI compute time again ... does anybody have additional ideas how we
> could avoid that in the future?

QEMU keeps getting bigger, so our attempts to cut down on what we
build mean we keep testing a narrower & narrower subset of
functionality. We long ago culled the obvious wastage.

One thing is to optimize our code such that it is less expensive
for the compiler. I feel like we've done that periodically already
with refactoring header files. It is a complex problem with no
quick wins to reduce compile time. If someone knows of more low
hanging fruit though, speak up.

Cirrus CI supports a "bring your hardware" model. We could do
that for FreeBSD testing, by leveraging our existnig Azure
account further to provide VMs. If doing that though, it'd
be saner to just do it in GitLab context directly. This huas
the burn of *us* maintaining FreeBSD images,and updating
them every time a new FreeBSD is released. ie you can't just
stick on 12.0, because when 12.1 is released, the ports
packages get rebuilt against new 12.1 and frequently become
uninstallable on the older minor release. IOW, we would have
a continual burden in updating our images to track FreeBSD
minor releases. Also this doesn't solve our macOS needs since
you can't just run macOS in any VM, it has to be Apple HW.

Cirrus CI lets you buy more CI credits. IIUC we get 50 credits
free each month. I can't see QEMU's usage, since it is private
to admins only:

  https://cirrus-ci.com/settings/github/qemu

if it is anything like libvirt though, I'm going to expect that
most of our credits are consumed by macOS, not FreeBSD, since
Cirrus CI charges for macOS at a higher level.

To stop running out of credits I guess we'll want to buy an
extra 20 credits in each freeze period - ie 60 a year. They
charge $1 per credit.

Does the QEMU project have funds available to spend on things
like this ?  It unfortunately requires a credit card to buy
more credits, so somehow QEMU would have to re-imburse an
individual maintainer for the purchase.

Final option is to make contact with Cirrus CI owners, and
ask if they might wish to directly sponsor QEMU in terms of
granting us extra credits, in exchange for including them on
our sponsor's page https://www.qemu.org/sponsors/.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|