Re: [PATCH 1/5] gitlab: centralize the container tag name

[Thomas Huth <thuth@redhat.com>]

On 17/05/2023 15.54, Daniel P. Berrangé wrote:
> We use a fixed container tag of 'latest' so that contributors' forks
> don't end up with an ever growing number of containers as they work
> on throwaway feature branches.
> 
> This fixed tag causes problems running CI upstream in stable staging
> branches, however, because the stable staging branch will publish old
> container content that clashes with that needed by primary staging
> branch. This makes it impossible to reliably run CI pipelines in
> parallel in upstream for different staging branches.
> 
> This introduces $QEMU_CI_CONTAINER_TAG global variable as a way to
> change which tag container publishing uses. Initially it can be set
> by contributors as a git push option if they want to override the
> default use of 'latest' eg
> 
>    git push gitlab <branch> -o ci.variable=QEMU_CONTAINER_TAG=fish
> 
> this is useful if contributors need to run pipelines for different
> branches concurrently in their forks.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>   .gitlab-ci.d/base.yml                | 6 ++++++
>   .gitlab-ci.d/buildtest-template.yml  | 4 ++--
>   .gitlab-ci.d/buildtest.yml           | 4 ++--
>   .gitlab-ci.d/container-template.yml  | 3 ++-
>   .gitlab-ci.d/crossbuild-template.yml | 6 +++---
>   .gitlab-ci.d/static_checks.yml       | 4 ++--
>   docs/devel/ci-jobs.rst.inc           | 5 +++++
>   7 files changed, 22 insertions(+), 10 deletions(-)
> 
> diff --git a/.gitlab-ci.d/base.yml b/.gitlab-ci.d/base.yml
> index 2fbb58d2a3..fba9d31cc6 100644
> --- a/.gitlab-ci.d/base.yml
> +++ b/.gitlab-ci.d/base.yml
> @@ -1,4 +1,10 @@
>   
> +variables:
> +  # On stable branches this needs changing. Should also be
> +  # overridden per pipeline if running pipelines concurrently
> +  # for different branches in contributor forks.
> +  QEMU_CI_CONTAINER_TAG: latest
> +
>   # The order of rules defined here is critically important.
>   # They are evaluated in order and first match wins.
>   #
> diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
> index a6cfe9be97..094ae485b8 100644
> --- a/.gitlab-ci.d/buildtest-template.yml
> +++ b/.gitlab-ci.d/buildtest-template.yml
> @@ -1,7 +1,7 @@
>   .native_build_job_template:
>     extends: .base_job_template
>     stage: build
> -  image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
> +  image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:$QEMU_CI_CONTAINER_TAG
>     before_script:
>       - JOBS=$(expr $(nproc) + 1)
>     script:
> @@ -44,7 +44,7 @@
>   .common_test_job_template:
>     extends: .base_job_template
>     stage: test
> -  image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
> +  image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:$QEMU_CI_CONTAINER_TAG
>     script:
>       - scripts/git-submodule.sh update
>           $(sed -n '/GIT_SUBMODULES=/ s/.*=// p' build/config-host.mak)
> diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
> index bb3650a51c..35867dfe77 100644
> --- a/.gitlab-ci.d/buildtest.yml
> +++ b/.gitlab-ci.d/buildtest.yml
> @@ -532,7 +532,7 @@ build-without-defaults:
>   build-libvhost-user:
>     extends: .base_job_template
>     stage: build
> -  image: $CI_REGISTRY_IMAGE/qemu/fedora:latest
> +  image: $CI_REGISTRY_IMAGE/qemu/fedora:$QEMU_CI_CONTAINER_TAG
>     needs:
>       job: amd64-fedora-container
>     script:
> @@ -572,7 +572,7 @@ build-tools-and-docs-debian:
>   # of what topic branch they're currently using
>   pages:
>     extends: .base_job_template
> -  image: $CI_REGISTRY_IMAGE/qemu/debian-amd64:latest
> +  image: $CI_REGISTRY_IMAGE/qemu/debian-amd64:$QEMU_CI_CONTAINER_TAG
>     stage: test
>     needs:
>       - job: build-tools-and-docs-debian
> diff --git a/.gitlab-ci.d/container-template.yml b/.gitlab-ci.d/container-template.yml
> index 519b8a9482..83c693c807 100644
> --- a/.gitlab-ci.d/container-template.yml
> +++ b/.gitlab-ci.d/container-template.yml
> @@ -5,7 +5,8 @@
>     services:
>       - docker:dind
>     before_script:
> -    - export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:latest"
> +    - export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:$QEMU_CI_CONTAINER_TAG"
> +    # Always ':latest' because we always use upstream as a common cache source
>       - export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/qemu/$NAME:latest"
>       - apk add python3
>       - docker info

This patch no longer applies ... could you please rebase and resend a v2? 
Thanks!

  Thomas