[PATCH 0/2] gitlab: limit jobs executed in most pipelines by default

["Daniel P. Berrangé" <berrange@redhat.com>]

The QEMU pipeline has as many as 120+ jobs, almost all of which are
always run. This is wasteful in many ways. When merging to the main
repo, Peter runs CI on 'staging' branch, which then gets immediately
re-run on the 'master' branch.  The first patch in this series
culls that redundancy by always skipping CI on pushes to 'master'.

I suggest we setup a nightly scheduled job that runs on current
contents of 'master' once a day, as a way to catch bitrot from OS
distro's changing under us.

I think we want the first patch as a high priority, since I think
the duplicated jobs on master are really impacting our integration
with Cirrus CI, delaying jobs since Cirrus has very strictly
applied concurrency limits.



The second patch is more speculative, and illustrates how to we
can reduce the number of CI jobs run for *all* contributors,
when they push to gitlab. There is a key decision to make here
between:

 a. run no jobs at all by default on push
 b. run a few jobs by default on push to catch 90% of problems
 c. run all jobs on by default on push

Currently we're doing (c), and patch 2 implements (b), with an
easy opt-in to trigger (c).

There is a credible variant of patch 2 though, which could easily
implement (a) with an opt-in to run (b), and a further opt-in to
run (c).

Or we can get more advanced and add ability for individual
contributors to decide between (a) and (b) for default behaviour,
based on a variable configured globally in their gitlab fork.

There is an illustration of results of (b) here:

   https://gitlab.com/berrange/qemu/-/pipelines/339281325

Note that for all the non-defualt jobs, I've left them as
manually triggerable. This makes a the pipeline UI a bit
cluttered by default but lets you trigger jobs from the
web UI individually. An alternative would be to never
create the non-defualt jobs at all. They would have to be
requested explicitly at time of push in some way.

Daniel P. Berrangé (2):
  gitlab: prevent CI jobs running pushes to default branch or tags
  gitlab: cut down on jobs run by default in user forks

 .gitlab-ci.d/buildtest-template.yml  |  17 +---
 .gitlab-ci.d/buildtest.yml           |  14 +++
 .gitlab-ci.d/cirrus.yml              |   5 +-
 .gitlab-ci.d/container-core.yml      |   3 +
 .gitlab-ci.d/container-cross.yml     |   9 +-
 .gitlab-ci.d/container-template.yml  |   1 +
 .gitlab-ci.d/containers.yml          |   2 +
 .gitlab-ci.d/crossbuild-template.yml |   3 +
 .gitlab-ci.d/crossbuilds.yml         |   3 +
 .gitlab-ci.d/qemu-project.yml        |   1 +
 .gitlab-ci.d/workflow.yml            | 146 +++++++++++++++++++++++++++
 .gitlab-ci.yml                       |  21 +---
 12 files changed, 188 insertions(+), 37 deletions(-)
 create mode 100644 .gitlab-ci.d/workflow.yml

-- 
2.31.1