Re: [RFC] QEMU Gating CI

[Cleber Rosa <crosa@redhat.com>]

On Mon, Dec 02, 2019 at 11:36:35AM -0700, Warner Losh wrote:
> 
> Just make sure that any pipeline and mandatory CI steps don't slow things
> down too much... While the examples have talked about 1 or 2 pull requests
> getting done in parallel, and that's great, the problem is when you try to
> land 10 or 20 all at once, one that causes the failure and you aren't sure
> which one it actually is... Make sure whatever you design has sane
> exception case handling to not cause too much collateral damage... I worked
> one place that would back everything out if a once-a-week CI test ran and
> had failures... That CI test-run took 2 days to run, so it wasn't practical
> to run it often, or for every commit. In the end, though, the powers that
> be implemented a automated bisection tool that made it marginally less
> sucky..
> 
> Warner

What I would personally like to see is the availability of enough
resources to give a ~2 hour max result turnaround, that is, the
complete pipeline finishes within that 2 hours.  Of course the exact
max time should be a constructed consensus.

If someone is contributing a new job supposed to run on existing
hardware, its acceptance should be carefully considered.  If more
hardware is being added and the job is capable of running parallel
with others, than it shouldn't be an issue (I don't think we'll hit
GitLab's scheduling limits anytime soon).

With regards to the "1 or 2 pull requests done in parallel", of course
there could be a queue of pending jobs, but given that the idea is for
these jobs to be run based on maintainers actions (say a Merge
Request), the volume should be much lower than if individual
contributors were triggering the same jobs on their patch series, and
not at all on every commit (as you describe with the ~2 days jobs).

Anyway, thanks for the feedback and please do not refrain from further
participation in this effort.  Your experience seems quite valuable.

Thanks,
- Cleber.