Re: Serious doubts about Gitlab CI

["Daniel P. Berrangé" <berrange@redhat.com>]

On Tue, Mar 30, 2021 at 04:23:35PM +0200, Paolo Bonzini wrote:
> On 30/03/21 16:13, Stefan Hajnoczi wrote:
> > On Tue, Mar 30, 2021 at 01:55:48PM +0200, Thomas Huth wrote:
> > > On 30/03/2021 13.19, Daniel P. Berrangé wrote:
> > > > On Mon, Mar 29, 2021 at 03:10:36PM +0100, Stefan Hajnoczi wrote:
> > > > > Hi,
> > > > > I wanted to follow up with a summary of the CI jobs:
> > > > > 
> > > > > 1. Containers & Containers Layer2 - ~3 minutes/job x 39 jobs
> > > > > 2. Builds - ~50 minutes/job x 61 jobs
> > > > > 3. Tests - ~12 minutes/job x 20 jobs
> > > > > 4. Deploy - 52 minutes x 1 job
> > > 
> > > I hope that 52 was just a typo ... ?
> > 
> > No, but I think Dan already found this issue a little while ago. The
> > deploy job uses "make install":
> > 
> >    # Prepare for GitLab pages deployment. Anything copied into the
> >    # "public" directory will be deployed to $USER.gitlab.io/$PROJECT
> >    pages:
> >      image: $CI_REGISTRY_IMAGE/qemu/debian-amd64:latest
> >      stage: test
> >      needs:
> >        - job: build-tools-and-docs-debian
> >      script:
> >        - mkdir -p public
> >        # HTML-ised source tree
> >        - make gtags
> >        - htags -anT --tree-view=filetree -m qemu_init
> >            -t "Welcome to the QEMU sourcecode"
> >        - mv HTML public/src
> >        # Project documentation
> >        - make -C build install DESTDIR=$(pwd)/temp-install
> >        - mv temp-install/usr/local/share/doc/qemu/* public/
> >      artifacts:
> >        paths:
> >          - public
> > 
> > Do we have/need a docs-only install target?
> 
> The problem is that after "git clone" the artifacts from the previous stage
> are stale.  We can use the "NINJA=:" hack/workaround on the make command
> line that we already use for tests.

The problem we have is that the publishing job needs to be called "pages"
in the gitlab-ci.yml file.  We didn't want to rebuild everything, so we
have the dependancy on the build-tools-and-docs-debian job to do the
build phase, and the "pages" job just does "make install" + list the
artifacts.

We can avoid the accidental rebuild if we just change what we do in the
"pages" job.  Move the "make install" command into the
"build-tools-and-docs-debian" job.

The "pages" job then reduces to only

 - mv temp-install/usr/local/share/doc/qemu/* public/

and thus is guaranteed to not trigger a rebuild.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|