Re: [Buildroot] [PATCH v2 RFC] support/scripts: don't build board defconfigs with Gitlab's pipelines trigged on tag

From: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
To: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Romain Naour <romain.naour@gmail.com>, buildroot@buildroot.org
Subject: Re: [Buildroot] [PATCH v2 RFC] support/scripts: don't build board defconfigs with Gitlab's pipelines trigged on tag
Date: Thu, 11 Nov 2021 11:02:25 +0100	[thread overview]
Message-ID: <20211111110225.50c73fd9@windsurf> (raw)
In-Reply-To: <20211111082548.GA2504300@scaer>

Hello,

On Thu, 11 Nov 2021 09:25:48 +0100
"Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> On 2021-11-09 23:03 +0100, Romain Naour spake thusly:
> > Currently when a tag is added to the Buildroot git tree, the gitlab-ci
> > create a pipeline with several hundred of jobs (~750) to build all
> > defconfigs and execute the Buildroot testsuite.
> > 
> > However, there is only a limited number of gitlab-ci runner (9 runners)
> > and some jobs reach the timeout limit (24h) while waiting for a runner
> > [1]. Indeed, the Buildroot project doesn't use the Gitlab's shared
> > runners.
> > 
> > In addition to the pipeline created when a new tag is added to the
> > git repository, two pipelines are created each weeks to execute the
> > Buildroot testsuite (on monday [2]) and build all defconfigs (on
> > Thursday [3]).
> > 
> > At some point there are too many jobs waiting in gitlab due board
> > defconfigs builds. Indded a board defconfig requires a lot of time
> > (~30min) compared to other jobs in order to build a toolchain and a
> > kernel linux along with a basic rootfs. There is currently 262
> > defconfigs.
> > 
> > This is even worse when several pipelines are trigged at the same
> > time (new git tag and scheduled pipeline trigger).
> > 
> > In order to reduce the number of long jobs, don't build board
> > defconfigs with pipelines trigged on tag, keeping only the runtime
> > tests and the Qemu's defconfigs.
> > 
> > [1] https://gitlab.com/buildroot.org/buildroot/-/jobs/1758966541
> > [2] https://gitlab.com/buildroot.org/buildroot/-/pipelines/404035190
> > [3] https://gitlab.com/buildroot.org/buildroot/-/pipelines/401685550
> > 
> > Signed-off-by: Romain Naour <romain.naour@gmail.com>
> > Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> > Cc: Yann E. MORIN <yann.morin.1998@free.fr>
> > Cc: Arnout Vandecappelle <arnout@mind.be>  
> 
> Applied to master, thanks.

I hadn't had the chance to chime in on this one, but I don't entirely
agree with the change (I also don't entirely disagree!). Indeed, I
believe we do want to have the results of the defconfigs builds on
tags. If we missed something, and one of our release has defconfigs
broken, we definitely want to know it and fix it.

This disabling of defconfig builds is only done because there is an
infrastructure issue. But really, from a proper QA/CI point of view, we
really want to build the defconfigs on tags I think.

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot