All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH dwarves 0/6] Add support for LTO kernel builds, other CI fixes
@ 2026-11-16 11:20 Alan Maguire
  2026-11-16 11:20 ` [PATCH dwarves 1/6] CI: Make kernel source fetch tolerant of git host failures Alan Maguire
                   ` (5 more replies)
  0 siblings, 6 replies; 7+ messages in thread
From: Alan Maguire @ 2026-11-16 11:20 UTC (permalink / raw)
  To: dwarves; +Cc: Alan Maguire

CI has been flaky lately, git.kernel.org clones in particular.
Provide fallback locations for cloning and retry (patch 1).

When testing with LTO kernels, function comparisons between
DWARF/BTF take a long time; use more efficient methods (patch 2).

Fix a few other issues in CI (patches 3, 4).

Finally add ThinLTO kernel build support (patch 5).

Results of this series on top of pahole's next branch at [1];
all pass.

[1] https://github.com/alan-maguire/dwarves/actions/runs/43849750600

Alan Maguire (6):
  CI: Make kernel source fetch tolerant of git host failures
  tests: Match BTF and DWARF functions with set operations in
    btf_functions.sh
  CI: Scope workflow concurrency groups to the branch for push runs
  tests: Expect no DWARF loader threads in reproducible_build.sh when
    CUs are merged
  CI: Fix ondemand workflow inputs and LLVM version in function
    comparison
  CI: Add clang ThinLTO kernel build to test matrix

 .github/scripts/build-kernel.sh      | 34 +++++++++++--
 .github/scripts/get-kernel-source.sh | 46 ++++++++++++++++++
 .github/scripts/run-selftests.sh     |  2 +-
 .github/workflows/build.yml          |  2 +-
 .github/workflows/ondemand.yml       | 11 ++++-
 .github/workflows/test.yml           | 10 +++-
 .github/workflows/vmtest.yml         | 20 +++++---
 tests/btf_functions.sh               | 71 +++++++++++++++++-----------
 tests/reproducible_build.sh          | 17 ++++++-
 tests/test_lib.sh                    | 36 ++++++++++++++
 10 files changed, 207 insertions(+), 42 deletions(-)
 create mode 100755 .github/scripts/get-kernel-source.sh

-- 
2.56.0


^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH dwarves 1/6] CI: Make kernel source fetch tolerant of git host failures
  2026-11-16 11:20 [PATCH dwarves 0/6] Add support for LTO kernel builds, other CI fixes Alan Maguire
@ 2026-11-16 11:20 ` Alan Maguire
  2026-11-16 11:20 ` [PATCH dwarves 2/6] tests: Match BTF and DWARF functions with set operations in btf_functions.sh Alan Maguire
                   ` (4 subsequent siblings)
  5 siblings, 0 replies; 7+ messages in thread
From: Alan Maguire @ 2026-11-16 11:20 UTC (permalink / raw)
  To: dwarves; +Cc: Alan Maguire

CI jobs have been failing with timeouts fetching bpf-next from
git.kernel.org.  Replace the libbpf/ci/get-linux-source action with
a script that does a shallow fetch, retrying across mirrors:

1. git.kernel.org bpf-next master
2. github.com/kernel-patches/bpf bpf-next_base (github-to-github;
   bpf-next plus BPF CI commits)
3. kernel.googlesource.com bpf-next master

with up to 3 rounds and backoff between them.  Each fetch attempt is
bounded by a timeout, and git's low-speed limits abort stalled
transfers so we move on to the next mirror rather than hanging until
the job times out.  The commit fetched is logged.

Since the source is now a detached shallow checkout, build-kernel.sh
no longer checks out REPO_BRANCH; its fallback clone (for local use)
is now a shallow clone of that branch.

Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Alan Maguire <almagui@kernel.org>
---
 .github/scripts/build-kernel.sh      |  4 +--
 .github/scripts/get-kernel-source.sh | 46 ++++++++++++++++++++++++++++
 .github/workflows/vmtest.yml         |  7 ++---
 3 files changed, 51 insertions(+), 6 deletions(-)
 create mode 100755 .github/scripts/get-kernel-source.sh

diff --git a/.github/scripts/build-kernel.sh b/.github/scripts/build-kernel.sh
index 63806f2..8326e1d 100755
--- a/.github/scripts/build-kernel.sh
+++ b/.github/scripts/build-kernel.sh
@@ -18,11 +18,11 @@ export PAHOLE=${GITHUB_WORKSPACE}/install/usr/local/bin/pahole
 which pahole
 $PAHOLE --version
 
+# In CI, get-kernel-source.sh has already fetched the source.
 if [[ ! -d $REPO_TARGET ]]; then
-	git clone $REPO $REPO_TARGET
+	git clone --depth=1 --branch $REPO_BRANCH $REPO $REPO_TARGET
 fi
 cd $REPO_TARGET
-git checkout $REPO_BRANCH
 
 buildopts=
 if [[ "$CC" = "clang" ]]; then
diff --git a/.github/scripts/get-kernel-source.sh b/.github/scripts/get-kernel-source.sh
new file mode 100755
index 0000000..2d28215
--- /dev/null
+++ b/.github/scripts/get-kernel-source.sh
@@ -0,0 +1,46 @@
+#!/usr/bin/bash
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Fetch bpf-next kernel source, tolerating failures/stalls of any
+# single git host by retrying across mirrors.
+#
+
+GITHUB_WORKSPACE=${GITHUB_WORKSPACE:-$(dirname $0)/../..}
+DEST=${GITHUB_WORKSPACE}/.kernel
+FETCH_TIMEOUT=${FETCH_TIMEOUT:-15m}
+ATTEMPTS=${ATTEMPTS:-3}
+
+# "URL REF" pairs, tried in order.  kernel-patches/bpf bpf-next_base tracks
+# bpf-next master plus BPF CI commits, and is github-to-github so is the
+# first fallback.
+MIRRORS=(
+	"https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master"
+	"https://github.com/kernel-patches/bpf.git bpf-next_base"
+	"https://kernel.googlesource.com/pub/scm/linux/kernel/git/bpf/bpf-next.git master"
+)
+
+git init -q "$DEST" && cd "$DEST" || exit 1
+# Abort stalled transfers (< 1KB/s for 60s) rather than hanging until
+# the job times out, so we move on to the next mirror.
+git config http.lowSpeedLimit 1000
+git config http.lowSpeedTime 60
+
+for attempt in $(seq 1 $ATTEMPTS); do
+	for mirror in "${MIRRORS[@]}"; do
+		read -r url ref <<< "$mirror"
+		echo "Fetching $ref from $url (attempt $attempt/$ATTEMPTS)"
+		if timeout $FETCH_TIMEOUT git fetch --depth=1 "$url" "$ref"; then
+			git checkout -q FETCH_HEAD
+			echo "Fetched kernel source from $url $ref:"
+			git log -1 --oneline
+			exit 0
+		fi
+		echo "Fetch from $url failed"
+	done
+	if [[ $attempt -lt $ATTEMPTS ]]; then
+		sleep $((attempt * 30))
+	fi
+done
+
+echo "Failed to fetch kernel source from all mirrors" >&2
+exit 1
diff --git a/.github/workflows/vmtest.yml b/.github/workflows/vmtest.yml
index d6f9f79..1e876a0 100644
--- a/.github/workflows/vmtest.yml
+++ b/.github/workflows/vmtest.yml
@@ -55,10 +55,9 @@ jobs:
         run: .github/scripts/build-pahole.sh
 
       - name: Get kernel source
-        uses: libbpf/ci/get-linux-source@v3
-        with:
-          repo: 'https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git'
-          dest: '${{ github.workspace }}/.kernel'
+        shell: bash
+        timeout-minutes: 60
+        run: .github/scripts/get-kernel-source.sh
 
       - name: Configure, build kernel with current pahole
         shell: bash
-- 
2.56.0


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [PATCH dwarves 2/6] tests: Match BTF and DWARF functions with set operations in btf_functions.sh
  2026-11-16 11:20 [PATCH dwarves 0/6] Add support for LTO kernel builds, other CI fixes Alan Maguire
  2026-11-16 11:20 ` [PATCH dwarves 1/6] CI: Make kernel source fetch tolerant of git host failures Alan Maguire
@ 2026-11-16 11:20 ` Alan Maguire
  2026-11-16 11:20 ` [PATCH dwarves 3/6] CI: Scope workflow concurrency groups to the branch for push runs Alan Maguire
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 7+ messages in thread
From: Alan Maguire @ 2026-11-16 11:20 UTC (permalink / raw)
  To: dwarves; +Cc: Alan Maguire

btf_functions.sh validates each BTF function prototype by running
grep -F over the full list of DWARF prototypes, once per BTF function.
That is O(BTF * DWARF) with a fork per function (~50k on a typical
kernel), and on a clang ThinLTO kernel the test no longer completes in
a reasonable time in CI.

Replace the loop with sorted set operations: comm for exact and
"inline" matches, and comm/join on const-stripped prototypes for the
const-insensitive case, keeping the original BTF prototype so
mismatches are reported as before.  Run them under LC_ALL=C so sort,
comm and join agree on collation.

Matching is now on whole lines rather than substrings.  Previously
grep -F could return several DWARF lines (e.g. "int foo(void)" also
matches "unsigned int foo(void)", or const and non-const instances of
the same function), so such functions failed the exact comparison and
were only accepted via the const-insensitive fallback.  They are now
counted as exact matches; the total number of matched functions is
unchanged.

On a local x86_64 vmlinux the test drops from 2m44s to 34s with
identical results otherwise.

Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Alan Maguire <almagui@kernel.org>
---
 tests/btf_functions.sh | 71 ++++++++++++++++++++++++++----------------
 1 file changed, 44 insertions(+), 27 deletions(-)

diff --git a/tests/btf_functions.sh b/tests/btf_functions.sh
index 436ebeb..778f9fa 100755
--- a/tests/btf_functions.sh
+++ b/tests/btf_functions.sh
@@ -55,35 +55,52 @@ pfunct --all --no_parm_names --format_path=dwarf $vmlinux | \
 pfunct --all --no_parm_names --format_path=btf $outdir/vmlinux.btf 2>/dev/null|\
 	awk '{ gsub("^(bpf_kfunc |bpf_fastcall )+",""); print $0}'|sort|uniq > $outdir/btf.funcs
 
-exact=0
-inline=0
-const_insensitive=0
+# Match BTF prototypes against DWARF prototypes with sorted set operations
+# (comm/join) rather than one grep of dwarf.funcs per BTF function; the
+# latter is O(BTF * DWARF) and becomes very slow on large (e.g. LTO) kernels.
+# Matching is on whole lines.  Use the C locale so sort, comm and join
+# agree on collation order.
+(
+export LC_ALL=C
+cd $outdir
+
+sort -u btf.funcs > btf.sorted
+sort -u dwarf.funcs > dwarf.sorted
+
+# 1. exact matches
+comm -12 btf.sorted dwarf.sorted > match.exact
+comm -23 btf.sorted dwarf.sorted > nomatch.exact
+
+# 2. function might be declared inline in DWARF.
+sed 's/^/inline /' nomatch.exact | sort > nomatch.exact.inline
+comm -12 nomatch.exact.inline dwarf.sorted > match.inline
+comm -23 nomatch.exact.inline dwarf.sorted | sed 's/^inline //' > nomatch.inline
+
+# 3. some functions have multiple instances in DWARF where one has
+# const param(s) and another does not (see errpos()).  We do not
+# mark these functions inconsistent as though they technically
+# have different prototypes, the data itself is not different.
+# Compare with "const " (and any DWARF "inline " prefix) removed;
+# keep "<noconst>\t<original>" pairs so mismatches are reported
+# with the original BTF prototype.
+sed -e 's/const //g' -e 's/^inline //' dwarf.sorted | sort -u > dwarf.noconst
+sed 's/const //g' nomatch.inline | paste - nomatch.inline | \
+	sort -t $'\t' -k1,1 > nomatch.inline.keyed
+cut -f1 nomatch.inline.keyed | sort -u | comm -23 - dwarf.noconst > nomatch.noconst
+join -t $'\t' nomatch.noconst nomatch.inline.keyed | cut -f2 > mismatches
+)
+
+exact=$(wc -l < $outdir/match.exact)
+inline=$(wc -l < $outdir/match.inline)
+const_insensitive=$(( $(wc -l < $outdir/nomatch.inline) - $(wc -l < $outdir/mismatches) ))
 
 while IFS= read -r btf ; do
-	# look for non-inline match first
-	dwarf=$(grep -F "$btf" $outdir/dwarf.funcs)
-	if [[ "$btf" != "$dwarf" ]]; then
-		# function might be declared inline in DWARF.
-		if [[ "inline $btf" != "$dwarf" ]]; then
-			# some functions have multiple instances in DWARF where one has
-			# const param(s) and another does not (see errpos()).  We do not
-			# mark these functions inconsistent as though they technically
-			# have different prototypes, the data itself is not different.
-			btf_noconst=$(echo $btf | awk '{gsub("const ",""); print $0 }')
-			dwarf_noconst=$(echo $dwarf | awk '{gsub("const ",""); print $0 }')
-			if [[ "$dwarf_noconst" =~ "$btf_noconst" ]]; then
-				const_insensitive=$((const_insensitive+1))
-			else
-				error_log "ERROR: mismatch : BTF '$btf' not found; DWARF '$dwarf'"
-				test_fail
-			fi
-		else
-			inline=$((inline+1))
-		fi
-	else
-		exact=$((exact+1))
-	fi
-done < $outdir/btf.funcs
+	# show DWARF prototypes for the same function name to aid debugging.
+	fn=$(echo "$btf" | sed -e 's/(.*//' -e 's/.*[ *]//')
+	dwarf=$(grep -F " $fn(" $outdir/dwarf.funcs; grep -F "*$fn(" $outdir/dwarf.funcs)
+	error_log "ERROR: mismatch : BTF '$btf' not found; DWARF '$dwarf'"
+	test_fail
+done < $outdir/mismatches
 
 verbose_log "Matched $exact functions exactly."
 verbose_log "Matched $inline functions with inlines."
-- 
2.56.0


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [PATCH dwarves 3/6] CI: Scope workflow concurrency groups to the branch for push runs
  2026-11-16 11:20 [PATCH dwarves 0/6] Add support for LTO kernel builds, other CI fixes Alan Maguire
  2026-11-16 11:20 ` [PATCH dwarves 1/6] CI: Make kernel source fetch tolerant of git host failures Alan Maguire
  2026-11-16 11:20 ` [PATCH dwarves 2/6] tests: Match BTF and DWARF functions with set operations in btf_functions.sh Alan Maguire
@ 2026-11-16 11:20 ` Alan Maguire
  2026-11-16 11:20 ` [PATCH dwarves 4/6] tests: Expect no DWARF loader threads in reproducible_build.sh when CUs are merged Alan Maguire
                   ` (2 subsequent siblings)
  5 siblings, 0 replies; 7+ messages in thread
From: Alan Maguire @ 2026-11-16 11:20 UTC (permalink / raw)
  To: dwarves; +Cc: Alan Maguire

test.yml and build.yml use ci-{test,build}-${{ github.head_ref }} as
their concurrency group with cancel-in-progress.  github.head_ref is
only set for pull_request events, so for push and scheduled runs the
group is just "ci-test-"/"ci-build-": a push to any branch, or the
daily scheduled run, cancels in-progress push runs on every other
branch.

Fall back to github.ref when head_ref is empty so that only runs for
the same PR or branch cancel each other.

Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Alan Maguire <almagui@kernel.org>
---
 .github/workflows/build.yml | 2 +-
 .github/workflows/test.yml  | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
index 25a395f..d45b05a 100644
--- a/.github/workflows/build.yml
+++ b/.github/workflows/build.yml
@@ -7,7 +7,7 @@ on:
     - cron:  '0 18 * * *'
 
 concurrency:
-  group: ci-build-${{ github.head_ref }}
+  group: ci-build-${{ github.head_ref || github.ref }}
   cancel-in-progress: true
 
 jobs:
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index 940dab1..8bff4e4 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -7,7 +7,7 @@ on:
     - cron:  '0 18 * * *'
 
 concurrency:
-  group: ci-test-${{ github.head_ref }}
+  group: ci-test-${{ github.head_ref || github.ref }}
   cancel-in-progress: true
 
 jobs:
-- 
2.56.0


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [PATCH dwarves 4/6] tests: Expect no DWARF loader threads in reproducible_build.sh when CUs are merged
  2026-11-16 11:20 [PATCH dwarves 0/6] Add support for LTO kernel builds, other CI fixes Alan Maguire
                   ` (2 preceding siblings ...)
  2026-11-16 11:20 ` [PATCH dwarves 3/6] CI: Scope workflow concurrency groups to the branch for push runs Alan Maguire
@ 2026-11-16 11:20 ` Alan Maguire
  2026-11-16 11:20 ` [PATCH dwarves 5/6] CI: Fix ondemand workflow inputs and LLVM version in function comparison Alan Maguire
  2026-11-16 11:20 ` [PATCH dwarves 6/6] CI: Add clang ThinLTO kernel build to test matrix Alan Maguire
  5 siblings, 0 replies; 7+ messages in thread
From: Alan Maguire @ 2026-11-16 11:20 UTC (permalink / raw)
  To: dwarves; +Cc: Alan Maguire

reproducible_build.sh checks that "pahole -jN" starts N threads.  When
pahole merges all CUs into one, as it does for LTO kernels, DWARF is
loaded serially and no worker threads are started, so the test fails
on a clang ThinLTO vmlinux, as seen in the CI clang-lto-thin x86_64
job:

  ERROR: pahole asked to start 1 encoding threads, started 0

Add a dwarf_cus_merged() helper to test_lib.sh mirroring
cus__merging_cu(): the kernel's LINUX_ELFNOTE_LTO_INFO note is set, or
the DWARF uses DW_FORM_ref_addr.  When CUs are merged, expect zero
worker threads; the serial vs parallel BTF comparison is still done.

The helper walks the .notes section itself using the 4-byte layout of
the kernel's ELFNOTE() notes rather than relying on readelf -n.  On
kernels whose linker script does not discard .note.gnu.property (it is
discarded since kernel commit f7ba52f302fd ("vmlinux.lds.h: Discard
.note.gnu.property section")), .notes picks up the 8-byte alignment of
that section and readelf -n misparses the 4-byte aligned notes.

Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Alan Maguire <almagui@kernel.org>
---
 tests/reproducible_build.sh | 17 +++++++++++++++--
 tests/test_lib.sh           | 36 ++++++++++++++++++++++++++++++++++++
 2 files changed, 51 insertions(+), 2 deletions(-)

diff --git a/tests/reproducible_build.sh b/tests/reproducible_build.sh
index 532b320..8b7b8eb 100755
--- a/tests/reproducible_build.sh
+++ b/tests/reproducible_build.sh
@@ -27,6 +27,16 @@ export PAHOLE_LANG_EXCLUDE=rust
 pahole --btf_features=default --btf_encode_detached=$outdir/vmlinux.btf.serial $vmlinux
 bpftool btf dump file $outdir/vmlinux.btf.serial > $outdir/bpftool.output.vmlinux.btf.serial
 
+# When CUs are merged (e.g. LTO kernels) pahole loads DWARF serially and
+# starts no loader threads whatever -j says; the encoding must still be
+# reproducible.
+if dwarf_cus_merged $vmlinux ; then
+	verbose_log "CUs are merged (LTO?); DWARF loading is serial, expecting no worker threads"
+	merged_cus=1
+else
+	merged_cus=0
+fi
+
 nr_proc=$(getconf _NPROCESSORS_ONLN)
 # Clamp to 1 so single-core machines don't produce -j0
 half_proc=$(( nr_proc / 2 > 0 ? nr_proc / 2 : 1 ))
@@ -43,8 +53,11 @@ for threads in $thread_list ; do
 	nr_threads_started=$(ls /proc/$pahole/task 2>/dev/null | wc -l)
 	nr_threads_started=$((nr_threads_started - 1)) # subtract main thread
 
-	if [ $threads != $nr_threads_started ] ; then
-		error_log "ERROR: pahole asked to start $threads encoding threads, started $nr_threads_started"
+	expected_threads=$threads
+	[ $merged_cus = 1 ] && expected_threads=0
+
+	if [ $expected_threads != $nr_threads_started ] ; then
+		error_log "ERROR: pahole asked to start $threads encoding threads, expected $expected_threads, started $nr_threads_started"
 		test_fail
 	fi
 
diff --git a/tests/test_lib.sh b/tests/test_lib.sh
index e2d0b0c..bd3784e 100755
--- a/tests/test_lib.sh
+++ b/tests/test_lib.sh
@@ -464,6 +464,42 @@ get_perf_with_debug()
 	fi
 }
 
+# Return success if pahole's DWARF loader will merge all CUs of $1 into
+# one and load them serially rather than with -j worker threads.  This
+# mirrors cus__merging_cu() in dwarf_loader.c: the kernel's
+# LINUX_ELFNOTE_LTO_INFO note says it was built with LTO, or the DWARF
+# uses DW_FORM_ref_addr (references across CUs, as with clang LTO).
+#
+# readelf -n gives up on the kernel's .notes section (4-byte aligned
+# notes in an 8-byte aligned section), so walk the notes directly.
+dwarf_cus_merged()
+{
+	local obj=$1 notes
+
+	notes=$(mktemp)
+	if objcopy -O binary --only-section=.notes "$obj" "$notes" 2>/dev/null &&
+	   od -An -v -tu4 -w4 "$notes" | awk '
+		# namesz, descsz, type, name (padded to 4), desc (padded to 4)
+		{ w[n++] = $1 }
+		END {
+			for (i = 0; i + 3 <= n; ) {
+				namesz = w[i]; descsz = w[i+1]; type = w[i+2]
+				i += 3 + int((namesz + 3) / 4)
+				# LINUX_ELFNOTE_LTO_INFO, owner "Linux"
+				if (type == 257 && namesz == 6 && descsz == 4 && w[i] != 0)
+					exit 0
+				i += int((descsz + 3) / 4)
+			}
+			exit 1
+		}'; then
+		rm -f "$notes"
+		return 0
+	fi
+	rm -f "$notes"
+
+	readelf --debug-dump=abbrev "$obj" 2>/dev/null | grep -q DW_FORM_ref_addr
+}
+
 make_tmpdir()
 {
 	# Ensure master test directory exists
-- 
2.56.0


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [PATCH dwarves 5/6] CI: Fix ondemand workflow inputs and LLVM version in function comparison
  2026-11-16 11:20 [PATCH dwarves 0/6] Add support for LTO kernel builds, other CI fixes Alan Maguire
                   ` (3 preceding siblings ...)
  2026-11-16 11:20 ` [PATCH dwarves 4/6] tests: Expect no DWARF loader threads in reproducible_build.sh when CUs are merged Alan Maguire
@ 2026-11-16 11:20 ` Alan Maguire
  2026-11-16 11:20 ` [PATCH dwarves 6/6] CI: Add clang ThinLTO kernel build to test matrix Alan Maguire
  5 siblings, 0 replies; 7+ messages in thread
From: Alan Maguire @ 2026-11-16 11:20 UTC (permalink / raw)
  To: dwarves; +Cc: Alan Maguire

The ondemand workflow calls vmtest.yml without the required "cc"
input, so manually dispatched runs fail.  Add a "cc" input (default
gcc) to ondemand.yml, pass it through, and include it in the job name.

The function comparison step in vmtest.yml referenced
inputs.llvm_version rather than inputs.llvm-version, so LLVM_VERSION
was always empty there and compare-functions.sh silently fell back to
its default of 22 whatever llvm-version the caller requested.

Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Alan Maguire <almagui@kernel.org>
---
 .github/workflows/ondemand.yml | 6 +++++-
 .github/workflows/vmtest.yml   | 2 +-
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/.github/workflows/ondemand.yml b/.github/workflows/ondemand.yml
index 6e48e12..09e7ccb 100644
--- a/.github/workflows/ondemand.yml
+++ b/.github/workflows/ondemand.yml
@@ -18,10 +18,13 @@ on:
       runs-on:
         default: 'ubuntu-24.04'
         required: true
+      cc:
+        default: 'gcc'
+        required: true
 
 jobs:
   vmtest:
-    name: ${{ inputs.kernel }} kernel llvm-${{ inputs.llvm-version }} pahole@${{ inputs.pahole }}
+    name: ${{ inputs.kernel }} kernel ${{ inputs.cc }} llvm-${{ inputs.llvm-version }} pahole@${{ inputs.pahole }}
     uses: ./.github/workflows/vmtest.yml
     with:
       runs_on: ${{ inputs.runs-on }}
@@ -29,3 +32,4 @@ jobs:
       arch: ${{ inputs.arch }}
       llvm-version: ${{ inputs.llvm-version }}
       pahole: ${{ inputs.pahole }}
+      cc: ${{ inputs.cc }}
diff --git a/.github/workflows/vmtest.yml b/.github/workflows/vmtest.yml
index 1e876a0..0bc19d7 100644
--- a/.github/workflows/vmtest.yml
+++ b/.github/workflows/vmtest.yml
@@ -74,6 +74,6 @@ jobs:
         shell: bash
         env:
           CC: ${{ inputs.cc }}
-          LLVM_VERSION: ${{ inputs.llvm_version }}
+          LLVM_VERSION: ${{ inputs.llvm-version }}
         run: .github/scripts/compare-functions.sh
 
-- 
2.56.0


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [PATCH dwarves 6/6] CI: Add clang ThinLTO kernel build to test matrix
  2026-11-16 11:20 [PATCH dwarves 0/6] Add support for LTO kernel builds, other CI fixes Alan Maguire
                   ` (4 preceding siblings ...)
  2026-11-16 11:20 ` [PATCH dwarves 5/6] CI: Fix ondemand workflow inputs and LLVM version in function comparison Alan Maguire
@ 2026-11-16 11:20 ` Alan Maguire
  5 siblings, 0 replies; 7+ messages in thread
From: Alan Maguire @ 2026-11-16 11:20 UTC (permalink / raw)
  To: dwarves; +Cc: Alan Maguire

LTO kernels are an important case for BTF generation: cross-CU
inlining and function merging change what pahole sees in DWARF and
the ELF symbol table.  Add an x86_64 clang job that builds the kernel
with CONFIG_LTO_CLANG_THIN=y.

vmtest.yml gets an optional "lto" input (default "none") that is
passed to build-kernel.sh, which enables LTO_CLANG_THIN via
scripts/config.  Since olddefconfig silently falls back to LTO_NONE
when a dependency is unmet (e.g. KASAN, GCOV), fail the build if
ThinLTO did not end up enabled rather than quietly testing a non-LTO
kernel.  The function comparison step reuses the existing .config, so
the baseline build is LTO too.  The ondemand workflow gets a matching
"lto" input.

The tests runner starts all tests in parallel by default.  Several of
them load the full vmlinux with pahole/pfunct, and on a ThinLTO kernel
running them concurrently can exhaust the runner's memory.  Have
run-selftests.sh pass -j $SELFTEST_JOBS to the tests runner when set,
and set it to 2 for LTO jobs in vmtest.yml.  Non-LTO jobs keep running
all tests in parallel.

Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Alan Maguire <almagui@kernel.org>
---
 .github/scripts/build-kernel.sh  | 30 +++++++++++++++++++++++++++++-
 .github/scripts/run-selftests.sh |  2 +-
 .github/workflows/ondemand.yml   |  7 ++++++-
 .github/workflows/test.yml       |  8 ++++++++
 .github/workflows/vmtest.yml     | 11 ++++++++++-
 5 files changed, 54 insertions(+), 4 deletions(-)

diff --git a/.github/scripts/build-kernel.sh b/.github/scripts/build-kernel.sh
index 8326e1d..18a6f47 100755
--- a/.github/scripts/build-kernel.sh
+++ b/.github/scripts/build-kernel.sh
@@ -11,6 +11,7 @@ REPO_BRANCH=${REPO_BRANCH:-master}
 REPO_TARGET=${GITHUB_WORKSPACE}/.kernel
 CC=${CC:-gcc}
 LLVM_VERSION=${LLVM_VERSION:-22}
+LTO=${LTO:-none}
 
 export PATH=${GITHUB_WORKSPACE}/install/usr/local/bin:${PATH}
 export PAHOLE=${GITHUB_WORKSPACE}/install/usr/local/bin/pahole
@@ -36,7 +37,34 @@ cat tools/testing/selftests/bpf/config \
 if [[ -f tools/testing/selftests/bpf/config.vm ]]; then
 	cat tools/testing/selftests/bpf/config.vm >> .config
 fi
-make $buildopts olddefconfig && make prepare
+make $buildopts olddefconfig
+
+case "$LTO" in
+none)
+	;;
+thin)
+	if [[ "$CC" != "clang" ]]; then
+		echo "LTO=thin requires CC=clang" >&2
+		exit 1
+	fi
+	echo "Enabling Clang ThinLTO"
+	scripts/config -d LTO_NONE -e LTO_CLANG_THIN
+	make $buildopts olddefconfig
+	# olddefconfig silently falls back to LTO_NONE if a dependency
+	# is not met (e.g. KASAN, GCOV); fail rather than test non-LTO.
+	if ! grep -q '^CONFIG_LTO_CLANG_THIN=y' .config; then
+		echo "CONFIG_LTO_CLANG_THIN could not be enabled:" >&2
+		grep -E 'LTO|HAS_LTO' .config >&2
+		exit 1
+	fi
+	;;
+*)
+	echo "Unsupported LTO mode '$LTO'" >&2
+	exit 1
+	;;
+esac
+
+make $buildopts prepare
 cat .config
 make -j $((4*$(nproc))) $buildopts all
 
diff --git a/.github/scripts/run-selftests.sh b/.github/scripts/run-selftests.sh
index bb826bf..748c4b9 100755
--- a/.github/scripts/run-selftests.sh
+++ b/.github/scripts/run-selftests.sh
@@ -13,5 +13,5 @@ export LLVM_OBJCOPY=objcopy
 which pahole
 pahole --version
 bpftool --version
-vmlinux=$VMLINUX ./tests
+vmlinux=$VMLINUX ./tests ${SELFTEST_JOBS:+-j $SELFTEST_JOBS}
 
diff --git a/.github/workflows/ondemand.yml b/.github/workflows/ondemand.yml
index 09e7ccb..d9f8617 100644
--- a/.github/workflows/ondemand.yml
+++ b/.github/workflows/ondemand.yml
@@ -21,10 +21,14 @@ on:
       cc:
         default: 'gcc'
         required: true
+      lto:
+        description: 'kernel LTO mode (none or thin; thin requires cc=clang)'
+        default: 'none'
+        required: true
 
 jobs:
   vmtest:
-    name: ${{ inputs.kernel }} kernel ${{ inputs.cc }} llvm-${{ inputs.llvm-version }} pahole@${{ inputs.pahole }}
+    name: ${{ inputs.kernel }} kernel ${{ inputs.cc }} lto-${{ inputs.lto }} llvm-${{ inputs.llvm-version }} pahole@${{ inputs.pahole }}
     uses: ./.github/workflows/vmtest.yml
     with:
       runs_on: ${{ inputs.runs-on }}
@@ -33,3 +37,4 @@ jobs:
       llvm-version: ${{ inputs.llvm-version }}
       pahole: ${{ inputs.pahole }}
       cc: ${{ inputs.cc }}
+      lto: ${{ inputs.lto }}
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index 8bff4e4..8bc3897 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -28,6 +28,13 @@ jobs:
             llvm-version: '22'
             pahole: 'none'
             cc: 'clang'
+          - kernel: 'LATEST'
+            runs_on: 'ubuntu-24.04'
+            arch: 'x86_64'
+            llvm-version: '22'
+            pahole: 'none'
+            cc: 'clang'
+            lto: 'thin'
           - kernel: 'LATEST'
             runs_on: 'ubuntu-24.04-arm'
             arch: 'aarch64'
@@ -49,3 +56,4 @@ jobs:
       llvm-version: ${{ matrix.llvm-version }}
       pahole: ${{ matrix.pahole }}
       cc: ${{ matrix.cc }}
+      lto: ${{ matrix.lto || 'none' }}
diff --git a/.github/workflows/vmtest.yml b/.github/workflows/vmtest.yml
index 0bc19d7..7010205 100644
--- a/.github/workflows/vmtest.yml
+++ b/.github/workflows/vmtest.yml
@@ -31,10 +31,15 @@ on:
         description: 'compiler'
         required: true
         type: string
+      lto:
+        description: 'kernel LTO mode (none or thin; thin requires clang)'
+        required: false
+        default: 'none'
+        type: string
 
 jobs:
   vmtest:
-    name: pahole-${{ inputs.cc }}@${{ inputs.arch }}
+    name: pahole-${{ inputs.cc }}${{ inputs.lto != 'none' && format('-lto-{0}', inputs.lto) || '' }}@${{ inputs.arch }}
     runs-on: ${{ inputs.runs_on }}
     steps:
 
@@ -64,10 +69,14 @@ jobs:
         env:
           CC: ${{ inputs.cc }}
           LLVM_VERSION: ${{ inputs.llvm-version }}
+          LTO: ${{ inputs.lto }}
         run: .github/scripts/build-kernel.sh
 
       - name: Run selftests
         shell: bash
+        env:
+          # ThinLTO vmlinux is large; limit how many tests load it at once.
+          SELFTEST_JOBS: ${{ inputs.lto != 'none' && '2' || '' }}
         run: .github/scripts/run-selftests.sh
 
       - name: Compare functions generated
-- 
2.56.0


^ permalink raw reply related	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2026-11-16 11:20 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-16 11:20 [PATCH dwarves 0/6] Add support for LTO kernel builds, other CI fixes Alan Maguire
2026-11-16 11:20 ` [PATCH dwarves 1/6] CI: Make kernel source fetch tolerant of git host failures Alan Maguire
2026-11-16 11:20 ` [PATCH dwarves 2/6] tests: Match BTF and DWARF functions with set operations in btf_functions.sh Alan Maguire
2026-11-16 11:20 ` [PATCH dwarves 3/6] CI: Scope workflow concurrency groups to the branch for push runs Alan Maguire
2026-11-16 11:20 ` [PATCH dwarves 4/6] tests: Expect no DWARF loader threads in reproducible_build.sh when CUs are merged Alan Maguire
2026-11-16 11:20 ` [PATCH dwarves 5/6] CI: Fix ondemand workflow inputs and LLVM version in function comparison Alan Maguire
2026-11-16 11:20 ` [PATCH dwarves 6/6] CI: Add clang ThinLTO kernel build to test matrix Alan Maguire

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.