From: Alan Maguire <almagui@kernel.org>
To: dwarves@vger.kernel.org
Cc: Alan Maguire <almagui@kernel.org>
Subject: [PATCH dwarves 2/6] tests: Match BTF and DWARF functions with set operations in btf_functions.sh
Date: Mon, 16 Nov 2026 11:20:39 +0000 [thread overview]
Message-ID: <20261116112043.3831978-3-almagui@kernel.org> (raw)
In-Reply-To: <20261116112043.3831978-1-almagui@kernel.org>
btf_functions.sh validates each BTF function prototype by running
grep -F over the full list of DWARF prototypes, once per BTF function.
That is O(BTF * DWARF) with a fork per function (~50k on a typical
kernel), and on a clang ThinLTO kernel the test no longer completes in
a reasonable time in CI.
Replace the loop with sorted set operations: comm for exact and
"inline" matches, and comm/join on const-stripped prototypes for the
const-insensitive case, keeping the original BTF prototype so
mismatches are reported as before. Run them under LC_ALL=C so sort,
comm and join agree on collation.
Matching is now on whole lines rather than substrings. Previously
grep -F could return several DWARF lines (e.g. "int foo(void)" also
matches "unsigned int foo(void)", or const and non-const instances of
the same function), so such functions failed the exact comparison and
were only accepted via the const-insensitive fallback. They are now
counted as exact matches; the total number of matched functions is
unchanged.
On a local x86_64 vmlinux the test drops from 2m44s to 34s with
identical results otherwise.
Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Alan Maguire <almagui@kernel.org>
---
tests/btf_functions.sh | 71 ++++++++++++++++++++++++++----------------
1 file changed, 44 insertions(+), 27 deletions(-)
diff --git a/tests/btf_functions.sh b/tests/btf_functions.sh
index 436ebeb..778f9fa 100755
--- a/tests/btf_functions.sh
+++ b/tests/btf_functions.sh
@@ -55,35 +55,52 @@ pfunct --all --no_parm_names --format_path=dwarf $vmlinux | \
pfunct --all --no_parm_names --format_path=btf $outdir/vmlinux.btf 2>/dev/null|\
awk '{ gsub("^(bpf_kfunc |bpf_fastcall )+",""); print $0}'|sort|uniq > $outdir/btf.funcs
-exact=0
-inline=0
-const_insensitive=0
+# Match BTF prototypes against DWARF prototypes with sorted set operations
+# (comm/join) rather than one grep of dwarf.funcs per BTF function; the
+# latter is O(BTF * DWARF) and becomes very slow on large (e.g. LTO) kernels.
+# Matching is on whole lines. Use the C locale so sort, comm and join
+# agree on collation order.
+(
+export LC_ALL=C
+cd $outdir
+
+sort -u btf.funcs > btf.sorted
+sort -u dwarf.funcs > dwarf.sorted
+
+# 1. exact matches
+comm -12 btf.sorted dwarf.sorted > match.exact
+comm -23 btf.sorted dwarf.sorted > nomatch.exact
+
+# 2. function might be declared inline in DWARF.
+sed 's/^/inline /' nomatch.exact | sort > nomatch.exact.inline
+comm -12 nomatch.exact.inline dwarf.sorted > match.inline
+comm -23 nomatch.exact.inline dwarf.sorted | sed 's/^inline //' > nomatch.inline
+
+# 3. some functions have multiple instances in DWARF where one has
+# const param(s) and another does not (see errpos()). We do not
+# mark these functions inconsistent as though they technically
+# have different prototypes, the data itself is not different.
+# Compare with "const " (and any DWARF "inline " prefix) removed;
+# keep "<noconst>\t<original>" pairs so mismatches are reported
+# with the original BTF prototype.
+sed -e 's/const //g' -e 's/^inline //' dwarf.sorted | sort -u > dwarf.noconst
+sed 's/const //g' nomatch.inline | paste - nomatch.inline | \
+ sort -t $'\t' -k1,1 > nomatch.inline.keyed
+cut -f1 nomatch.inline.keyed | sort -u | comm -23 - dwarf.noconst > nomatch.noconst
+join -t $'\t' nomatch.noconst nomatch.inline.keyed | cut -f2 > mismatches
+)
+
+exact=$(wc -l < $outdir/match.exact)
+inline=$(wc -l < $outdir/match.inline)
+const_insensitive=$(( $(wc -l < $outdir/nomatch.inline) - $(wc -l < $outdir/mismatches) ))
while IFS= read -r btf ; do
- # look for non-inline match first
- dwarf=$(grep -F "$btf" $outdir/dwarf.funcs)
- if [[ "$btf" != "$dwarf" ]]; then
- # function might be declared inline in DWARF.
- if [[ "inline $btf" != "$dwarf" ]]; then
- # some functions have multiple instances in DWARF where one has
- # const param(s) and another does not (see errpos()). We do not
- # mark these functions inconsistent as though they technically
- # have different prototypes, the data itself is not different.
- btf_noconst=$(echo $btf | awk '{gsub("const ",""); print $0 }')
- dwarf_noconst=$(echo $dwarf | awk '{gsub("const ",""); print $0 }')
- if [[ "$dwarf_noconst" =~ "$btf_noconst" ]]; then
- const_insensitive=$((const_insensitive+1))
- else
- error_log "ERROR: mismatch : BTF '$btf' not found; DWARF '$dwarf'"
- test_fail
- fi
- else
- inline=$((inline+1))
- fi
- else
- exact=$((exact+1))
- fi
-done < $outdir/btf.funcs
+ # show DWARF prototypes for the same function name to aid debugging.
+ fn=$(echo "$btf" | sed -e 's/(.*//' -e 's/.*[ *]//')
+ dwarf=$(grep -F " $fn(" $outdir/dwarf.funcs; grep -F "*$fn(" $outdir/dwarf.funcs)
+ error_log "ERROR: mismatch : BTF '$btf' not found; DWARF '$dwarf'"
+ test_fail
+done < $outdir/mismatches
verbose_log "Matched $exact functions exactly."
verbose_log "Matched $inline functions with inlines."
--
2.56.0
next prev parent reply other threads:[~2026-11-16 11:20 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-16 11:20 [PATCH dwarves 0/6] Add support for LTO kernel builds, other CI fixes Alan Maguire
2026-11-16 11:20 ` [PATCH dwarves 1/6] CI: Make kernel source fetch tolerant of git host failures Alan Maguire
2026-11-16 11:20 ` Alan Maguire [this message]
2026-11-16 11:20 ` [PATCH dwarves 3/6] CI: Scope workflow concurrency groups to the branch for push runs Alan Maguire
2026-11-16 11:20 ` [PATCH dwarves 4/6] tests: Expect no DWARF loader threads in reproducible_build.sh when CUs are merged Alan Maguire
2026-11-16 11:20 ` [PATCH dwarves 5/6] CI: Fix ondemand workflow inputs and LLVM version in function comparison Alan Maguire
2026-11-16 11:20 ` [PATCH dwarves 6/6] CI: Add clang ThinLTO kernel build to test matrix Alan Maguire
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261116112043.3831978-3-almagui@kernel.org \
--to=almagui@kernel.org \
--cc=dwarves@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.