Re: [PATCH 4/4] bootlinux: add support for A/B kernel testing

[Luis Chamberlain <mcgrof@kernel.org>]

On Sat, Jul 26, 2025 at 07:35:54PM -0400, Chuck Lever wrote:
> On 7/26/25 4:21 PM, Luis Chamberlain wrote:
> > If you give me a branch, I can just fetch it and ask Claude Code to
> > refactor my branch with your and take preference for style for your
> > branch. Its as easy as that.
> 
> https://github.com/chucklever/kdevops/tree/builder
> 
> You are welcome to experiment and/or provide feedback to me about the
> ideas.

Will do, specially given your employer constraints.

> > You can try it as well too, if you like as my tree is public already.
> > And it may help you test / fix any outstanding bugs.
> 
> Unfortunately my employer has a policy that we cannot make use of public
> AI code generators at this time, so I can't try it quite yet.

I hope PROMPTS.md's growth on kdevops will show the value and without
it folks will just fall behind. Fast. I intentionally made PROMPTS.md
to help folks not only learn but also help with employers.

> >> What my set does is change the operation of the "make linux" target so
> >> that, depending on Kconfig settings, it either:
> >>
> >> - Works as it does today (builds the kernel on each test runner)
> >> - Builds the kernel and packages it on a separate node, saving the
> >>   package
> > 
> > I see, OK so an extra option is available to "build on foo system which
> > is wicked fast".
> 
> Yes, one new option for that, and one new option for "please install
> from the packages sitting in this directory on the Ansible controller.

Sure.

> I'm also thinking that, for cloud, kdevops could set up a build pipeline
> to build the kernel. So far, my tenancies are limited to 10 vCPUs at a
> time. But the pipelines seem to get as many as 16, making the builds
> quick.

Indeed. A "super build host" for cloud seems sensible.

> >> - Finds the packaged kernel and installs it on the test runners
> > 
> > Sweeeeet. It would be nice for us to have a sort of registry for kdevops
> > where if a kernel is already built and available we can use it but alas
> > we don't yet have any public infrastructure for that.
> 
> Yes. Scott and I have been going back and forth about how to do such a
> thing.
> 
> For cloud, it is typical for artifacts built by a pipeline to land in
> a public S3-style bucket. We could certainly put them there and have
> some scripting to create a website around it.

Neat. Makes sense.

> For guestfs, kdevops could set up a local minIO instance to do much the
> same. Or the guests could NFS-mount the controller.

OK yes I see. We already have "enterprise" uses of kdevops in the sense
that for example /mirror/ is on an NFS mount. So it would make sense
to then also use minio for these possible pre-built daily kernels.
We'd probably want a sort of master and allow the one off servers in
a corp set up to essentially go and fetch the latest 30 kernel builds
using S3 or whatever.

> > Maybe we can just
> > regularly later build using OBS daily for next and weekly for RCs and
> > see if the latest build is availble. That is, we just have a secondary
> > process which all it does is fetch and build and push to OBS. If using
> > a vanilla kernel or next kernel then it can use that. This limits the
> > functionality to standard trees but a possible nice enhacement later.
> 
> The primary goal is reducing the number of times we have to build any
> kernel. Building them on a schedule (if there is a change) is better
> than building the kernel every time a change is noticed, I think (and
> unfortunately that is what I'm doing right now).

Indeed.

> The reason to build kernels on the spot is for development work. For
> CI, having one build, and using that build everywhere you can, seems
> like that should be our goal.

Agreed.

> Interesting -- I didn't know you have so much bare metal work going on!
> 
> AFAI recall, kdevops/aws is the only provider where we offer a bare
> metal choice. I can have a look at the others to see what is easy to
> introduce.

Indeed. And even with the marketed "AWS bare metal" setups, you still
end up without some interesting performance counters... so I couldn't
evaluate TLB coalsecing for instnace on AMD which they support since
early days.

> > I see -- I thought CONFIG_LOCALVERSION_AUTO was already nice enough,
> > so you wan to track configs more closely.
> 
> I set CONFIG_LOCALVERSION_AUTO pretty much everywhere. I am already a
> client, as the TV ads used to say.
> 
> I'm just pointing out a possible use case where it wouldn't be enough.
> And maybe we worry about that use case some other time. There is a lot
> going on in this one patch!

Punting for later sure.

  Luis