From: Thomas Huth <thuth@redhat.com>
To: qemu-devel@nongnu.org, Richard Henderson <richard.henderson@linaro.org>
Cc: "Daniel P. Berrangé" <berrange@redhat.com>,
"Michael Tokarev" <mjt@tls.msk.ru>
Subject: [PULL 2/9] gitlab: centralize the container tag name
Date: Mon, 26 Jun 2023 10:14:08 +0200 [thread overview]
Message-ID: <20230626081415.64615-3-thuth@redhat.com> (raw)
In-Reply-To: <20230626081415.64615-1-thuth@redhat.com>
From: Daniel P. Berrangé <berrange@redhat.com>
We use a fixed container tag of 'latest' so that contributors' forks
don't end up with an ever growing number of containers as they work
on throwaway feature branches.
This fixed tag causes problems running CI upstream in stable staging
branches, however, because the stable staging branch will publish old
container content that clashes with that needed by primary staging
branch. This makes it impossible to reliably run CI pipelines in
parallel in upstream for different staging branches.
This introduces $QEMU_CI_CONTAINER_TAG global variable as a way to
change which tag container publishing uses. Initially it can be set
by contributors as a git push option if they want to override the
default use of 'latest' eg
git push gitlab <branch> -o ci.variable=QEMU_CONTAINER_TAG=fish
this is useful if contributors need to run pipelines for different
branches concurrently in their forks.
Reviewed-by: Michael Tokarev <mjt@tls.msk.ru>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230608164018.2520330-2-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
docs/devel/ci-jobs.rst.inc | 5 +++++
.gitlab-ci.d/base.yml | 6 ++++++
.gitlab-ci.d/buildtest-template.yml | 4 ++--
.gitlab-ci.d/buildtest.yml | 4 ++--
.gitlab-ci.d/container-template.yml | 3 ++-
.gitlab-ci.d/crossbuild-template.yml | 6 +++---
.gitlab-ci.d/static_checks.yml | 4 ++--
7 files changed, 22 insertions(+), 10 deletions(-)
diff --git a/docs/devel/ci-jobs.rst.inc b/docs/devel/ci-jobs.rst.inc
index 1f28fec0d0..f72537853b 100644
--- a/docs/devel/ci-jobs.rst.inc
+++ b/docs/devel/ci-jobs.rst.inc
@@ -70,6 +70,11 @@ in a handful of namespaces
repository CI settings, or as git push variables, to influence
which jobs get run in a pipeline
+ * QEMU_CI_CONTAINER_TAG - the tag used to publish containers
+ in stage 1, for use by build jobs in stage 2. Defaults to
+ 'latest', but if running pipelines for different branches
+ concurrently, it should be overridden per pipeline.
+
* nnn - other misc variables not falling into the above
categories, or using different names for historical reasons
and not yet converted.
diff --git a/.gitlab-ci.d/base.yml b/.gitlab-ci.d/base.yml
index 2fbb58d2a3..fba9d31cc6 100644
--- a/.gitlab-ci.d/base.yml
+++ b/.gitlab-ci.d/base.yml
@@ -1,4 +1,10 @@
+variables:
+ # On stable branches this needs changing. Should also be
+ # overridden per pipeline if running pipelines concurrently
+ # for different branches in contributor forks.
+ QEMU_CI_CONTAINER_TAG: latest
+
# The order of rules defined here is critically important.
# They are evaluated in order and first match wins.
#
diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index 76ff1dfcb6..5da61f4277 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -1,7 +1,7 @@
.native_build_job_template:
extends: .base_job_template
stage: build
- image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+ image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:$QEMU_CI_CONTAINER_TAG
before_script:
- JOBS=$(expr $(nproc) + 1)
script:
@@ -40,7 +40,7 @@
.common_test_job_template:
extends: .base_job_template
stage: test
- image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+ image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:$QEMU_CI_CONTAINER_TAG
script:
- scripts/git-submodule.sh update roms/SLOF
- meson subprojects download $(cd build/subprojects && echo *)
diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index 1922caf536..aa833b62ca 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -532,7 +532,7 @@ build-without-defaults:
build-libvhost-user:
extends: .base_job_template
stage: build
- image: $CI_REGISTRY_IMAGE/qemu/fedora:latest
+ image: $CI_REGISTRY_IMAGE/qemu/fedora:$QEMU_CI_CONTAINER_TAG
needs:
job: amd64-fedora-container
script:
@@ -572,7 +572,7 @@ build-tools-and-docs-debian:
# of what topic branch they're currently using
pages:
extends: .base_job_template
- image: $CI_REGISTRY_IMAGE/qemu/debian-amd64:latest
+ image: $CI_REGISTRY_IMAGE/qemu/debian-amd64:$QEMU_CI_CONTAINER_TAG
stage: test
needs:
- job: build-tools-and-docs-debian
diff --git a/.gitlab-ci.d/container-template.yml b/.gitlab-ci.d/container-template.yml
index 77aa839e9e..4eec72f383 100644
--- a/.gitlab-ci.d/container-template.yml
+++ b/.gitlab-ci.d/container-template.yml
@@ -5,7 +5,8 @@
services:
- docker:dind
before_script:
- - export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:latest"
+ - export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:$QEMU_CI_CONTAINER_TAG"
+ # Always ':latest' because we always use upstream as a common cache source
- export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/qemu/$NAME:latest"
- docker login $CI_REGISTRY -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD"
- until docker info; do sleep 1; done
diff --git a/.gitlab-ci.d/crossbuild-template.yml b/.gitlab-ci.d/crossbuild-template.yml
index 4f93b9e4e5..6efb0d2a54 100644
--- a/.gitlab-ci.d/crossbuild-template.yml
+++ b/.gitlab-ci.d/crossbuild-template.yml
@@ -1,7 +1,7 @@
.cross_system_build_job:
extends: .base_job_template
stage: build
- image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+ image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:$QEMU_CI_CONTAINER_TAG
timeout: 80m
script:
- mkdir build
@@ -27,7 +27,7 @@
.cross_accel_build_job:
extends: .base_job_template
stage: build
- image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+ image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:$QEMU_CI_CONTAINER_TAG
timeout: 30m
script:
- mkdir build
@@ -39,7 +39,7 @@
.cross_user_build_job:
extends: .base_job_template
stage: build
- image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+ image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:$QEMU_CI_CONTAINER_TAG
script:
- mkdir build
- cd build
diff --git a/.gitlab-ci.d/static_checks.yml b/.gitlab-ci.d/static_checks.yml
index b4cbdbce2a..ad9f426a52 100644
--- a/.gitlab-ci.d/static_checks.yml
+++ b/.gitlab-ci.d/static_checks.yml
@@ -26,7 +26,7 @@ check-dco:
check-python-minreqs:
extends: .base_job_template
stage: test
- image: $CI_REGISTRY_IMAGE/qemu/python:latest
+ image: $CI_REGISTRY_IMAGE/qemu/python:$QEMU_CI_CONTAINER_TAG
script:
- make -C python check-minreqs
variables:
@@ -37,7 +37,7 @@ check-python-minreqs:
check-python-tox:
extends: .base_job_template
stage: test
- image: $CI_REGISTRY_IMAGE/qemu/python:latest
+ image: $CI_REGISTRY_IMAGE/qemu/python:$QEMU_CI_CONTAINER_TAG
script:
- make -C python check-tox
variables:
--
2.39.3
next prev parent reply other threads:[~2023-06-26 8:15 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-06-26 8:14 [PULL 0/9] Testing / CI patches Thomas Huth
2023-06-26 8:14 ` [PULL 1/9] tests/qtest: Fix a comment typo in vhost-user-test.c Thomas Huth
2023-06-26 8:14 ` Thomas Huth [this message]
2023-06-26 8:14 ` [PULL 3/9] gitlab: allow overriding name of the upstream repository Thomas Huth
2023-06-26 8:14 ` [PULL 4/9] gitlab: stable staging branches publish containers in a separate tag Thomas Huth
2023-06-26 8:14 ` [PULL 5/9] gitlab: avoid extra pipelines for tags and stable branches Thomas Huth
2023-06-26 8:14 ` [PULL 6/9] gitlab: support disabling job auto-run in upstream Thomas Huth
2023-06-26 8:14 ` [PULL 7/9] gitlab-ci: grab msys2 meson-logs as artifacts Thomas Huth
2023-06-26 8:14 ` [PULL 8/9] gitlab-ci: add msys2 meson test to junit report Thomas Huth
2023-06-26 8:14 ` [PULL 9/9] tests/qtest/cxl-test: Clean up temporary directories after testing Thomas Huth
2023-06-26 10:45 ` [PULL 0/9] Testing / CI patches Richard Henderson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230626081415.64615-3-thuth@redhat.com \
--to=thuth@redhat.com \
--cc=berrange@redhat.com \
--cc=mjt@tls.msk.ru \
--cc=qemu-devel@nongnu.org \
--cc=richard.henderson@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).