[RFC PATCH] gitlab: add special rule for the hexagon container

[RFC PATCH] gitlab: add special rule for the hexagon container

[Alex Bennée]

The hexagon container is always manually built but of course not
everyone will be building it themselves and pushing to their
registries. We still need to create a "local" registry copy for the
actual gitlab tests to run. We don't build it in this case, just pull
it across from the upstream registry. We disable this rule from
running on the qemu-project itself so it doesn't accidentally wipe out
our master copy.

Fixes: 910c40ee94 ("gitlab: add build-user-hexagon test")
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Cornelia Huck <cohuck@redhat.com>
---
 .gitlab-ci.d/containers.yml | 27 +++++++++++++++++++++++++++
 .gitlab-ci.yml              |  2 ++
 2 files changed, 29 insertions(+)

diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml
index 3fb3c14f06..088c7e68c3 100644
--- a/.gitlab-ci.d/containers.yml
+++ b/.gitlab-ci.d/containers.yml
@@ -101,6 +101,33 @@ armhf-debian-cross-container:
   variables:
     NAME: debian-armhf-cross
 
+# We never want to build hexagon in the CI system and by default we
+# always want to refer to the master registry where it lives.
+hexagon-cross-container:
+  image: docker:stable
+  stage: containers
+  except:
+    variables:
+      - $CI_PROJECT_NAMESPACE == 'qemu-project'
+  variables:
+    NAME: debian-hexagon-cross
+    GIT_DEPTH: 1
+  services:
+    - docker:dind
+  before_script:
+    - export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:latest"
+    - export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/qemu/$NAME:latest"
+    - docker info
+    - docker login $CI_REGISTRY -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD"
+  script:
+    - echo "TAG:$TAG"
+    - echo "COMMON_TAG:$COMMON_TAG"
+    - docker pull $COMMON_TAG
+    - docker tag $COMMON_TAG $TAG
+    - docker push "$TAG"
+  after_script:
+    - docker logout
+
 hppa-debian-cross-container:
   extends: .container_job_template
   stage: containers-layer2
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index f718b61fa7..a04cca9db0 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -421,6 +421,8 @@ build-user-static:
 # declared. The image is manually uploaded.
 build-user-hexagon:
   extends: .native_build_job_template
+  needs:
+    hob: hexagon-cross-container
   variables:
     IMAGE: debian-hexagon-cross
     TARGETS: hexagon-linux-user
-- 
2.20.1

Re: [RFC PATCH] gitlab: add special rule for the hexagon container

[Richard Henderson]

On 5/20/21 8:19 AM, Alex Bennée wrote:
> The hexagon container is always manually built but of course not
> everyone will be building it themselves and pushing to their
> registries. We still need to create a "local" registry copy for the
> actual gitlab tests to run. We don't build it in this case, just pull
> it across from the upstream registry. We disable this rule from
> running on the qemu-project itself so it doesn't accidentally wipe out
> our master copy.
> 
> Fixes: 910c40ee94 ("gitlab: add build-user-hexagon test")
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Cc: Cornelia Huck <cohuck@redhat.com>
> ---

I get

Found errors in your .gitlab-ci.yml:
jobs:build-user-hexagon:needs config uses invalid types: bridge


r~


>   .gitlab-ci.d/containers.yml | 27 +++++++++++++++++++++++++++
>   .gitlab-ci.yml              |  2 ++
>   2 files changed, 29 insertions(+)
> 
> diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml
> index 3fb3c14f06..088c7e68c3 100644
> --- a/.gitlab-ci.d/containers.yml
> +++ b/.gitlab-ci.d/containers.yml
> @@ -101,6 +101,33 @@ armhf-debian-cross-container:
>     variables:
>       NAME: debian-armhf-cross
>   
> +# We never want to build hexagon in the CI system and by default we
> +# always want to refer to the master registry where it lives.
> +hexagon-cross-container:
> +  image: docker:stable
> +  stage: containers
> +  except:
> +    variables:
> +      - $CI_PROJECT_NAMESPACE == 'qemu-project'
> +  variables:
> +    NAME: debian-hexagon-cross
> +    GIT_DEPTH: 1
> +  services:
> +    - docker:dind
> +  before_script:
> +    - export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:latest"
> +    - export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/qemu/$NAME:latest"
> +    - docker info
> +    - docker login $CI_REGISTRY -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD"
> +  script:
> +    - echo "TAG:$TAG"
> +    - echo "COMMON_TAG:$COMMON_TAG"
> +    - docker pull $COMMON_TAG
> +    - docker tag $COMMON_TAG $TAG
> +    - docker push "$TAG"
> +  after_script:
> +    - docker logout
> +
>   hppa-debian-cross-container:
>     extends: .container_job_template
>     stage: containers-layer2
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index f718b61fa7..a04cca9db0 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -421,6 +421,8 @@ build-user-static:
>   # declared. The image is manually uploaded.
>   build-user-hexagon:
>     extends: .native_build_job_template
> +  needs:
> +    hob: hexagon-cross-container
>     variables:
>       IMAGE: debian-hexagon-cross
>       TARGETS: hexagon-linux-user
> 

Re: [RFC PATCH] gitlab: add special rule for the hexagon container

[Daniel P. Berrangé]

On Mon, May 24, 2021 at 07:35:59AM -0700, Richard Henderson wrote:
> On 5/20/21 8:19 AM, Alex Bennée wrote:
> > The hexagon container is always manually built but of course not
> > everyone will be building it themselves and pushing to their
> > registries. We still need to create a "local" registry copy for the
> > actual gitlab tests to run. We don't build it in this case, just pull
> > it across from the upstream registry. We disable this rule from
> > running on the qemu-project itself so it doesn't accidentally wipe out
> > our master copy.
> > 
> > Fixes: 910c40ee94 ("gitlab: add build-user-hexagon test")
> > Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> > Cc: Cornelia Huck <cohuck@redhat.com>
> > ---
> 
> I get
> 
> Found errors in your .gitlab-ci.yml:
> jobs:build-user-hexagon:needs config uses invalid types: bridge

Bizarre message, but the problem looks like this:


> > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> > index f718b61fa7..a04cca9db0 100644
> > --- a/.gitlab-ci.yml
> > +++ b/.gitlab-ci.yml
> > @@ -421,6 +421,8 @@ build-user-static:
> >   # declared. The image is manually uploaded.
> >   build-user-hexagon:
> >     extends: .native_build_job_template
> > +  needs:
> > +    hob: hexagon-cross-container

Indent and syntax is wrong - should be

   needs:
     - job: hexagon-cross-container

with indent at same level as 'extends'

> >     variables:
> >       IMAGE: debian-hexagon-cross
> >       TARGETS: hexagon-linux-user
> > 
> 
> 

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [RFC PATCH] gitlab: add special rule for the hexagon container

[Alex Bennée]

Daniel P. Berrangé <berrange@redhat.com> writes:

> On Mon, May 24, 2021 at 07:35:59AM -0700, Richard Henderson wrote:
>> On 5/20/21 8:19 AM, Alex Bennée wrote:
>> > The hexagon container is always manually built but of course not
>> > everyone will be building it themselves and pushing to their
>> > registries. We still need to create a "local" registry copy for the
>> > actual gitlab tests to run. We don't build it in this case, just pull
>> > it across from the upstream registry. We disable this rule from
>> > running on the qemu-project itself so it doesn't accidentally wipe out
>> > our master copy.
>> > 
>> > Fixes: 910c40ee94 ("gitlab: add build-user-hexagon test")
>> > Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> > Cc: Cornelia Huck <cohuck@redhat.com>
>> > ---
>> 
>> I get
>> 
>> Found errors in your .gitlab-ci.yml:
>> jobs:build-user-hexagon:needs config uses invalid types: bridge
>
> Bizarre message, but the problem looks like this:
>
>
>> > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>> > index f718b61fa7..a04cca9db0 100644
>> > --- a/.gitlab-ci.yml
>> > +++ b/.gitlab-ci.yml
>> > @@ -421,6 +421,8 @@ build-user-static:
>> >   # declared. The image is manually uploaded.
>> >   build-user-hexagon:
>> >     extends: .native_build_job_template
>> > +  needs:
>> > +    hob: hexagon-cross-container
>
> Indent and syntax is wrong - should be
>
>    needs:
>      - job: hexagon-cross-container
>
> with indent at same level as 'extends'

This is why v2 was posted fairly quickly :-/

Also in the rolled up series I posted on Thursday:

  Subject: [PATCH  v1 0/8] various misc fixes (gitlab, gdbstub, plugins)
  Date: Thu, 20 May 2021 18:42:55 +0100
  Message-Id: <20210520174303.12310-1-alex.bennee@linaro.org>

>
>> >     variables:
>> >       IMAGE: debian-hexagon-cross
>> >       TARGETS: hexagon-linux-user
>> > 
>> 
>> 
>
> Regards,
> Daniel


-- 
Alex Bennée