make a-quick quick again (and light on the AB, particularly arm)

make a-quick quick again (and light on the AB, particularly arm)

[Alexander Kanavin]

Hello,

I feel that a-quick builds as they are have ceased to be quick or
lightweight, and I'd like to trim heavy items from them. The goal is
to:

- have a-quick complete in under an hour with fully populated sstate
- lessen the load on arm workers in particular; there is not enough of
them, and they should be used only in a-full
- make a-quick useful as a quick smoke check for patchsets that
reveals obvious issues rapidly, completes quickly, and doesn't compete
with a-full for resources too much. I would then be willing to use
them for that purpose.

The definitions of a-quick and a-full are here:
https://git.yoctoproject.org/yocto-autobuilder2/tree/config.py#n75

I'd like to make the following changes to reduce the load on arm workers:
- oe-selftest-armhost moves from shared to full. This is the biggest offender.

- qemuarm64-armhost does the same

There are also other heavy builds in a-quick that I think shouldn't be
a part of it:

- reproducible should move from shared to full. There's no way to make
it quicker.

- oe-selftest should be dropped from quick, or we could introduce
oe-selftest-quick that runs only a subset of sefltests that completes
under an hour (existing logs can be used to make that selection, e.g.
by tagging known slow test classes).

Thoughts?

Alex

Re: [yocto] make a-quick quick again (and light on the AB, particularly arm)

[Ross Burton]

On 20 Mar 2024, at 09:33, Alexander Kanavin via lists.yoctoproject.org <alex.kanavin=gmail.com@lists.yoctoproject.org> wrote:
> I'd like to make the following changes to reduce the load on arm workers:
> - oe-selftest-armhost moves from shared to full. This is the biggest offender.
> 
> - qemuarm64-armhost does the same
> 
> There are also other heavy builds in a-quick that I think shouldn't be
> a part of it:

I’d be against removing _all_ arm hosts from the quick build.  With the in-progress changes to the autobuilder hosting we should be able to retire the slower Arm workers and a build running on the Arm hosts shouldn’t be a problem.  It would be a waste of effort to attempt to optimise the build for the current cluster when there are plans to relocate shortly anyway.

Ross

Re: [yocto] make a-quick quick again (and light on the AB, particularly arm)

[Alexander Kanavin]

On Wed, 20 Mar 2024 at 11:55, Ross Burton <Ross.Burton@arm.com> wrote:
> I’d be against removing _all_ arm hosts from the quick build.  With the in-progress changes to the autobuilder hosting we should be able to retire the slower Arm workers and a build running on the Arm hosts shouldn’t be a problem.  It would be a waste of effort to attempt to optimise the build for the current cluster when there are plans to relocate shortly anyway.

Right, then let's not make this about arm builders, and simply drop
selftest of any kind and reproducible from a-quick. How about that?

Particularly selftest keeps getting bigger and slower and that isn't
going to change. It used to take under an hour, nowadays it's 2+ hours
on x86 and 3+ on arm, with full sstate ready from the start.

Alex

Re: [yocto] make a-quick quick again (and light on the AB, particularly arm)

[Richard Purdie]

On Wed, 2024-03-20 at 12:27 +0100, Alexander Kanavin wrote:
> On Wed, 20 Mar 2024 at 11:55, Ross Burton <Ross.Burton@arm.com>
> wrote:
> > I’d be against removing _all_ arm hosts from the quick build.  With
> > the in-progress changes to the autobuilder hosting we should be
> > able to retire the slower Arm workers and a build running on the
> > Arm hosts shouldn’t be a problem.  It would be a waste of effort to
> > attempt to optimise the build for the current cluster when there
> > are plans to relocate shortly anyway.
> 
> Right, then let's not make this about arm builders, and simply drop
> selftest of any kind and reproducible from a-quick. How about that?
> 
> Particularly selftest keeps getting bigger and slower and that isn't
> going to change. It used to take under an hour, nowadays it's 2+
> hours on x86 and 3+ on arm, with full sstate ready from the start.

It does depend what you're testing btw. You tend to test package
upgrades which don't really affect selftest as much as some of the
other changes I'm often dealing with.

I don't disagree with adding a separate target but I do want to make it
clear we should improve the speed of selftest too, I'm not ready to
give up on that yet!

Cheers,

Richard

Re: [yocto] make a-quick quick again (and light on the AB, particularly arm)

[Richard Purdie]

On Wed, 2024-03-20 at 10:33 +0100, Alexander Kanavin wrote:
> I feel that a-quick builds as they are have ceased to be quick or
> lightweight, and I'd like to trim heavy items from them. The goal is
> to:
> 
> - have a-quick complete in under an hour with fully populated sstate
> - lessen the load on arm workers in particular; there is not enough of
> them, and they should be used only in a-full
> - make a-quick useful as a quick smoke check for patchsets that
> reveals obvious issues rapidly, completes quickly, and doesn't compete
> with a-full for resources too much. I would then be willing to use
> them for that purpose.
> 
> The definitions of a-quick and a-full are here:
> https://git.yoctoproject.org/yocto-autobuilder2/tree/config.py#n75
> 
> I'd like to make the following changes to reduce the load on arm workers:
> - oe-selftest-armhost moves from shared to full. This is the biggest offender.
> 
> - qemuarm64-armhost does the same
> 
> There are also other heavy builds in a-quick that I think shouldn't be
> a part of it:
> 
> - reproducible should move from shared to full. There's no way to make
> it quicker.
> 
> - oe-selftest should be dropped from quick, or we could introduce
> oe-selftest-quick that runs only a subset of sefltests that completes
> under an hour (existing logs can be used to make that selection, e.g.
> by tagging known slow test classes).
> 
> Thoughts?

a-quick was added with the idea of covering the key things which catch
errors without the duplication that "full" has. In that sense, "quick"
was less of the intent than allowing relaxing of some of the tests
where it was possible to do so without breaking coverage too much.

For various reasons people haven't really used "quick". Also, in
general people do want to know about all the issues, including
reproducibility and selftest (on arm).

Host specific issues are comparatively rare on selftest now (gl
passthrough being an exception) and we already stopped per-distro
reproducibility testing.

So I'm torn. It would be fine and easy to maintain another target
alongside quick and full though so there isn't really a good reason not
to add something if it would be useful.

Cheers,

Richard