Re: [PATCH] build-linux: add workflow for repeated kernel builds

[Daniel Gomez <da.gomez@kernel.org>]

On 19/09/2025 05.51, Luis Chamberlain wrote:
> Add a new workflow that allows building the Linux kernel multiple times
> to measure build time variations and performance. This is useful for
> benchmarking build systems and compiler performance testing.
> 
> This goes in with monitoring support so we can do AB testing against
> different filesystems.
> 
> Generated-by: Claude AI
> Suggested-by: David Bueso <dave@stgolabs.net>
> Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
> ---
> 
> Demo of visualization of results:
> 
> https://htmlpreview.github.io/?https://github.com/mcgrof/plot-build/blob/main/index.html

This is really interesting and cool work.

However, I'm generally a strong advocate for reusing projects as much as
possible. So this feels like a missed opportunity not to build on existing
projects designed for this purpose (e.g. hyperfine [1]).

Link: https://github.com/sharkdp/hyperfine [1]

It would be helpful to understand the reasoning behind choosing one
approach (reusing something like hyperfine) over another (adding custom ansible
tasks/playbooks...)?

For reference, I put together a quick and hacky example for benchmarking kernel
builds using hyperfine and a simple Makefile, available here:

https://github.com/dkruces/linux-benchmarks/

Report example:
https://github.com/dkruces/linux-benchmarks/blob/automation-make/results/mac1611/REPORT.md

To clarify, I'm not oppose in any form to the work here. It's really great.
Please, merge & push! :)

One think I'd like to see as part of the report is:
* The kernel version
* The kernel configuration. Hopefully, the fragment work allows to have
better control on this. And better when we have SAT support in the kernel
* Toolchain: GCC, LLVM/Clang
* Host info: baremetal/vm, number of cores, memory available, etc.

I've also noticed some inconsistencies between runs, which could be due to
memory behavior, folio migration, fragmentation, and similar factors. To ensure
full confidence when comparing build times, we should rely on reproducible
builds [2].

Link: https://docs.kernel.org/kbuild/reproducible-builds.html#reproducible-builds [2]

FYI, I have reproducible build support patches ready. I'll post them today.