qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 0/2] gitlab: limit jobs executed in most pipelines by default
@ 2021-07-19 12:12 Daniel P. Berrangé
  2021-07-19 12:12 ` [PATCH 1/2] gitlab: prevent CI jobs running pushes to default branch or tags Daniel P. Berrangé
  2021-07-19 12:12 ` [PATCH 2/2] gitlab: cut down on jobs run by default in user forks Daniel P. Berrangé
  0 siblings, 2 replies; 3+ messages in thread
From: Daniel P. Berrangé @ 2021-07-19 12:12 UTC (permalink / raw)
  To: qemu-devel
  Cc: Thomas Huth, Daniel P. Berrangé, Philippe Mathieu-Daudé,
	Wainer dos Santos Moschetta, Willian Rampazzo, Alex Bennée

The QEMU pipeline has as many as 120+ jobs, almost all of which are
always run. This is wasteful in many ways. When merging to the main
repo, Peter runs CI on 'staging' branch, which then gets immediately
re-run on the 'master' branch.  The first patch in this series
culls that redundancy by always skipping CI on pushes to 'master'.

I suggest we setup a nightly scheduled job that runs on current
contents of 'master' once a day, as a way to catch bitrot from OS
distro's changing under us.

I think we want the first patch as a high priority, since I think
the duplicated jobs on master are really impacting our integration
with Cirrus CI, delaying jobs since Cirrus has very strictly
applied concurrency limits.



The second patch is more speculative, and illustrates how to we
can reduce the number of CI jobs run for *all* contributors,
when they push to gitlab. There is a key decision to make here
between:

 a. run no jobs at all by default on push
 b. run a few jobs by default on push to catch 90% of problems
 c. run all jobs on by default on push

Currently we're doing (c), and patch 2 implements (b), with an
easy opt-in to trigger (c).

There is a credible variant of patch 2 though, which could easily
implement (a) with an opt-in to run (b), and a further opt-in to
run (c).

Or we can get more advanced and add ability for individual
contributors to decide between (a) and (b) for default behaviour,
based on a variable configured globally in their gitlab fork.

There is an illustration of results of (b) here:

   https://gitlab.com/berrange/qemu/-/pipelines/339281325

Note that for all the non-defualt jobs, I've left them as
manually triggerable. This makes a the pipeline UI a bit
cluttered by default but lets you trigger jobs from the
web UI individually. An alternative would be to never
create the non-defualt jobs at all. They would have to be
requested explicitly at time of push in some way.

Daniel P. Berrangé (2):
  gitlab: prevent CI jobs running pushes to default branch or tags
  gitlab: cut down on jobs run by default in user forks

 .gitlab-ci.d/buildtest-template.yml  |  17 +---
 .gitlab-ci.d/buildtest.yml           |  14 +++
 .gitlab-ci.d/cirrus.yml              |   5 +-
 .gitlab-ci.d/container-core.yml      |   3 +
 .gitlab-ci.d/container-cross.yml     |   9 +-
 .gitlab-ci.d/container-template.yml  |   1 +
 .gitlab-ci.d/containers.yml          |   2 +
 .gitlab-ci.d/crossbuild-template.yml |   3 +
 .gitlab-ci.d/crossbuilds.yml         |   3 +
 .gitlab-ci.d/qemu-project.yml        |   1 +
 .gitlab-ci.d/workflow.yml            | 146 +++++++++++++++++++++++++++
 .gitlab-ci.yml                       |  21 +---
 12 files changed, 188 insertions(+), 37 deletions(-)
 create mode 100644 .gitlab-ci.d/workflow.yml

-- 
2.31.1




^ permalink raw reply	[flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-07-19 12:13 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-07-19 12:12 [PATCH 0/2] gitlab: limit jobs executed in most pipelines by default Daniel P. Berrangé
2021-07-19 12:12 ` [PATCH 1/2] gitlab: prevent CI jobs running pushes to default branch or tags Daniel P. Berrangé
2021-07-19 12:12 ` [PATCH 2/2] gitlab: cut down on jobs run by default in user forks Daniel P. Berrangé

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).