Re: [RFC PATCH 0/3] Build once, test everywhere

[Chuck Lever <cel@kernel.org>]

On 4/23/25 8:34 AM, Daniel Gomez wrote:
> On Tue, Apr 22, 2025 at 11:49:03AM +0100, cel@kernel.org wrote:
>> From: Chuck Lever <chuck.lever@oracle.com>
>>
>> With apologies to the Java folks.
>>
>> I've been looking for ways to make our workflow runs more efficient
>> because my lab resources are limited, and moving to the cloud costs
>> money per CPU second.
>>
>> One thing that seems like an obvious win would be to build the test
>> kernel one time (for example, after a merge/pull request), then
>> re-use that kernel binary for all the workflows we want to run on
>> it.
> 
> What about rebuilds between iterations?
> I think an extra step to improve that build time could be adding support for
> ccache in our kernel builds. Would it make sense to extend this in the future to
> enable a redis node?
> 
> https://github.com/ccache/ccache/wiki/Redis-storage
> 
> With the workflow suggestion below, I'd then do:
> 
> 1. Bring up a redis node guest
> 2. Bring up a kernel builder node that pulls from redis node ccache
> 3. Push redis artifacts to control host (for later sync up) and destroy redis guest
> 4. Push kernel package to control host

So I've asked our internal testing folks about using ccache to speed up
kernel builds. There were various reasons not to do this, one of which
is we want to have a clean trustworthy and repeatable kernel build to
test with.

I'm not sure I have a specific technical argument against using ccache
except that our builder node is ephemeral, making its ccache usable for
only a single build. A Redis storage node might solve that issue.
However I wonder if the time it takes to bring up the node, reload its
storage, then after the build save the redis artifacts and destroy it
might wipe out any benefit.

With libvirt guests we can save several minutes by doing the guest bring
up in parallel (via Ansible) instead of serially via a shell script.
I've got some ideas about that.


> That said, all this new workflow makes sense to me. I guess instead of
> duplicating the bootlinux role into build_linux I'd like to see a linux
> role that handles build location (localhost/specific guest), output package
> generation or not, deployment, etc. Does that make sense to you too?

The bootlinux role has grown a slew of special cases over the years, so
I'm in the mood to help reorganize it a little.

At least split it into a role that handles the kernel build, and one
that handles the minutiae of installing the builds on test runners.

I'm not quite following your suggestion, but I'm interested in hearing
more detail.


-- 
Chuck Lever