From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C289A2E3391 for ; Fri, 19 Sep 2025 08:29:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758270588; cv=none; b=NwjNakmQpFXumCWzNAwycFdhTw8Eu8JXqksKTlYm4JCretI/NEl+W/HMYmwfbNhvNz8fJOQ/ukWqxp4aOG41BaVOU926/K451ZzHx/3NwDm3CHw4TmzrsUDYNuppvGrnzLJ5w0wdgMfEcNG8wFIr6pM0ykBQmO/dXL8QuZgWn7M= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758270588; c=relaxed/simple; bh=J3Im3K0fBjBqGXR2W4zYJASo5WCdrTqVUVVYP5tx7x0=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=g9qu6jnPwnZamB0GjS9kMdy7niNuu8bta3zvO1RUI08KK8r/ybO7AwL83fPcm0Uam2+f/2Fqb1PhDcJ15/lcDdiEaiYDit9/nncKwwuLugediJw2QbvzXBSQBqax6CBTOOZAblasofWY+DA8s0dzwDdzISZ9QX5kPRT9LSH8c7w= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=BBWGibDv; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="BBWGibDv" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5077DC4CEF0; Fri, 19 Sep 2025 08:29:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1758270588; bh=J3Im3K0fBjBqGXR2W4zYJASo5WCdrTqVUVVYP5tx7x0=; h=Date:Reply-To:Subject:To:Cc:References:From:In-Reply-To:From; b=BBWGibDvwPL3cj6c0wjfxPgppEzuqNZmlcCgGDqicS0Znk38O9NqO9BDDc6R7SbbA SMNlLquEFECgs2m7CSADFSlew7D5h647L+5S0ydBNP4YmGj02RmNgqfOCoZnQsISrH HUfCnBpROJ6LIhoJocY6MMTkpumlamzbarBstF9dCIrrwQhoenzqF8APCUwsNNwgBa LYvXmr8KZTywFTTRqO7gffbutC3VMVjVZnFwA8lbO5xWN3PMUsjscx+UYyc0Hh1kxb Kd0APmKJaZ7nKNxSbxDFlu1tWbuRvdwYXVm/V0m4F310xidI3ALIBNd07i9D1dsw4K tdyRXXvLYjezw== Message-ID: Date: Fri, 19 Sep 2025 10:29:45 +0200 Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Reply-To: Daniel Gomez Subject: Re: [PATCH] build-linux: add workflow for repeated kernel builds To: Luis Chamberlain , Chuck Lever , Daniel Gomez , kdevops@lists.linux.dev Cc: David Bueso References: <20250919035158.422509-1-mcgrof@kernel.org> Content-Language: en-US From: Daniel Gomez Organization: kernel.org In-Reply-To: <20250919035158.422509-1-mcgrof@kernel.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On 19/09/2025 05.51, Luis Chamberlain wrote: > Add a new workflow that allows building the Linux kernel multiple times > to measure build time variations and performance. This is useful for > benchmarking build systems and compiler performance testing. > > This goes in with monitoring support so we can do AB testing against > different filesystems. > > Generated-by: Claude AI > Suggested-by: David Bueso > Signed-off-by: Luis Chamberlain > --- > > Demo of visualization of results: > > https://htmlpreview.github.io/?https://github.com/mcgrof/plot-build/blob/main/index.html This is really interesting and cool work. However, I'm generally a strong advocate for reusing projects as much as possible. So this feels like a missed opportunity not to build on existing projects designed for this purpose (e.g. hyperfine [1]). Link: https://github.com/sharkdp/hyperfine [1] It would be helpful to understand the reasoning behind choosing one approach (reusing something like hyperfine) over another (adding custom ansible tasks/playbooks...)? For reference, I put together a quick and hacky example for benchmarking kernel builds using hyperfine and a simple Makefile, available here: https://github.com/dkruces/linux-benchmarks/ Report example: https://github.com/dkruces/linux-benchmarks/blob/automation-make/results/mac1611/REPORT.md To clarify, I'm not oppose in any form to the work here. It's really great. Please, merge & push! :) One think I'd like to see as part of the report is: * The kernel version * The kernel configuration. Hopefully, the fragment work allows to have better control on this. And better when we have SAT support in the kernel * Toolchain: GCC, LLVM/Clang * Host info: baremetal/vm, number of cores, memory available, etc. I've also noticed some inconsistencies between runs, which could be due to memory behavior, folio migration, fragmentation, and similar factors. To ensure full confidence when comparing build times, we should rely on reproducible builds [2]. Link: https://docs.kernel.org/kbuild/reproducible-builds.html#reproducible-builds [2] FYI, I have reproducible build support patches ready. I'll post them today.