Re: [PATCH] CI: Add automatic retry for test.py jobs

[Tom Rini <trini@konsulko.com>]

[-- Attachment #1: Type: text/plain, Size: 1276 bytes --]

On Wed, Jul 12, 2023 at 08:00:23AM -0600, Simon Glass wrote:
> Hi Tom,
> 
> On Tue, 11 Jul 2023 at 20:33, Tom Rini <trini@konsulko.com> wrote:
> >
> > It is not uncommon for some of the QEMU-based jobs to fail not because
> > of a code issue but rather because of a timing issue or similar problem
> > that is out of our control. Make use of the keywords that Azure and
> > GitLab provide so that we will automatically re-run these when they fail
> > 2 times. If they fail that often it is likely we have found a real issue
> > to investigate.
> >
> > Signed-off-by: Tom Rini <trini@konsulko.com>
> > ---
> >  .azure-pipelines.yml | 1 +
> >  .gitlab-ci.yml       | 1 +
> >  2 files changed, 2 insertions(+)
> 
> This seems like a slippery slope. Do we know why things fail? I wonder
> if we should disable the tests / builders instead, until it can be
> corrected?

It happens in Azure, so it's not just the broken runner problem we have
in GitLab. And the problem is timing, as I said in the commit.
Sometimes we still get the RTC test failing. Other times we don't get
QEMU + U-Boot spawned in time (most often m68k, but sometimes x86).

> I'll note that we don't have this problem with sandbox tests.

OK, but that's not relevant?

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]