Linux Perf Users
 help / color / mirror / Atom feed
From: sashiko-bot@kernel.org
To: "Arnaldo Carvalho de Melo" <acme@kernel.org>
Cc: linux-perf-users@vger.kernel.org
Subject: Re: [PATCH v5 01/12] perf scripts: Add a coverage harness for perf
Date: Tue, 17 Nov 2026 03:21:49 +0000	[thread overview]
Message-ID: <sashiko-outbox-218136@kernel.org> (raw)
In-Reply-To: <20261117025305.3958882-2-acme@kernel.org>

Thank you for your contribution! Sashiko AI review found 3 potential
issues to consider:
- [Medium] perf scripts: coverage.sh: libperf-jvmti.so linked without
  profile runtime
- [Low] perf scripts: coverage.sh: missing timed out test check in
  run_tests()
- [Low] perf scripts: coverage.sh report path regexes require
  tools/perf/ prefix
--

commit 8d4f44c46af7fe536a6ca79e56ff18db52b58791
Author: Arnaldo Carvalho de Melo <acme@redhat.com>

perf scripts: Add a coverage harness for perf

This commit adds tools/perf/scripts/coverage.sh and corresponding make
targets to build perf with clang source-based coverage, run the perf test
suite, and generate coverage reports and tables.

> diff --git a/tools/perf/scripts/coverage.sh b/tools/perf/scripts/coverage.sh
> new file mode 100755
> index 0000000000000..aa2649cf08c4a
> --- /dev/null
> +++ b/tools/perf/scripts/coverage.sh
> @@ -0,0 +1,738 @@

[ ... ]

> +build_perf()
> +{
> +	if [ "$SKIP_BUILD" = 1 ]; then
> +		check_coverage_build
> +		return
> +	fi
> +
> +	say "Building perf in $BUILD_DIR"
> +	mkdir -p "$BUILD_DIR"
> +	local debug=()
> +	[ "$DEBUG_BUILD" = 1 ] && debug=(DEBUG=1)
> +
> +	if ! make -C "$PERF_DIR" O="$BUILD_DIR" CC=clang \
> +		"${debug[@]+"${debug[@]}"}" \
> +		EXTRA_CFLAGS="-fprofile-instr-generate -fcoverage-mapping -gdwarf-$DWARF_VERSION" \
> +		-j"$JOBS" > "$BUILD_LOG" 2>&1; then

[Severity: Medium]
Does passing -fprofile-instr-generate only via EXTRA_CFLAGS leave
__llvm_profile_runtime unresolved in libperf-jvmti.so?

jvmti/jvmti-in.o is compiled with EXTRA_CFLAGS, but Makefile.perf links
libperf-jvmti.so without CFLAGS or EXTRA_CFLAGS:

tools/perf/Makefile.perf:
    $(OUTPUT)$(LIBJVMTI): $(LIBJVMTI_IN)
    	$(QUIET_LINK)$(CC) $(LDFLAGS) -shared -Wl,-soname -Wl,$(LIBJVMTI) -o $@ $<

Without -fprofile-instr-generate in LDFLAGS, clang does not link the
profiling runtime into libperf-jvmti.so.  When test_java_symbol.sh loads
$BUILD_DIR/libperf-jvmti.so into jshell via -J-agentpath, the JVM fails to
load the agent due to the unresolved symbol, failing the test and
producing no coverage for libperf-jvmti.so despite llvm_cov_objects()
passing --object=$BUILD_DIR/libperf-jvmti.so to llvm-cov.

[ ... ]

> +	# 'perf test' exits 0 even when tests fail, only the summary tells.
> +	if [ "$test_status" = 0 ] && grep -qE '^Failed tests +: +[1-9]' "$TEST_LOG"; then
> +		test_status=1
> +	fi
> +
> +	grep -E '^(Passed|Skipped|Failed) .*:' "$TEST_LOG" || true

[Severity: Low]
Does run_tests() miss tests that time out?

In tools/perf/tests/builtin-test.c, print_tests_summary() tracks and
prints "Timed out tests   : <N>" separately from "Failed tests      : <N>"
while returning 0.  If a test times out (which is more likely under the
default -O0 -g build) and no other test fails, run_tests() omits the timed
out count from the summary and coverage.sh exits with status 0.

[ ... ]

> +file_cov()
> +{
> +	awk -v f="$2" '
> +		function perf_relpath(p) {
> +			if (p ~ /(^|\/)tools\/perf\//)
> +				sub(/^.*tools\/perf\//, "", p)
> +			return p
> +		}
> +		{
> +			g = f
> +			sub(/^.*tools\/perf\//, "", g)
> +			rel = perf_relpath($1)
> +			if (rel == f || rel == g || $1 == f) {
> +				print $8, $9, $10, $7
> +				exit
> +			}
> +		}' "$1"
> +}

[ ... ]

> +print_worst_files()
> +{
> +	[ "$TOP" -gt 0 ] || return 0
> +	local rows missed lines cover fn path
> +	rows=$(awk -v min="$MIN_LINES" '
> +		$1 ~ /(^|\/)tools\/perf\// && $8 >= min && $9 > 0 {
> +			path = $1
> +			sub(/^.*tools\/perf\//, "", path)
> +			printf "%s %s %s %s %s\n", $9, $8, $10, $7, path
> +		}' "$REPORT" | sort -k1,1nr -k2,2nr | sed -n "1,${TOP}p")

[ ... ]

> +table_row()
> +{
> +	awk -v f="$2" '
> +		function perf_relpath(p) {
> +			if (p ~ /(^|\/)tools\/perf\//)
> +				sub(/^.*tools\/perf\//, "", p)
> +			return p
> +		}
> +		{
> +			g = f
> +			sub(/^.*tools\/perf\//, "", g)
> +			path = perf_relpath($1)
> +			if (path == f || path == g || $1 == f) {
> +				printf "%s\t%s\t%d\t%d\t%.2f\t%s\t%d\t%s\n",
> +				       path, $8, $8 - $9, $9, $10, $5, $5 - $6, $7
> +				exit
> +			}
> +		}' "$1"
> +}
> +
> +table_rows()
> +{
> +	if [ ${#FOCUS[@]} -gt 0 ]; then
> +		local file
> +		for file in "${FOCUS[@]}"; do
> +			table_row "$REPORT" "$file"
> +		done | sort -t$'\t' -k5,5gr
> +		return
> +	fi
> +
> +	awk -v min="$MIN_LINES" '
> +		$1 ~ /(^|\/)tools\/perf\// && $8 >= min && $9 > 0 {
> +			path = $1
> +			sub(/^.*tools\/perf\//, "", path)
> +			printf "%s\t%s\t%d\t%d\t%.2f\t%s\t%d\t%s\n",
> +			       path, $8, $8 - $9, $9, $10, $5, $5 - $6, $7
> +		}' "$REPORT" |
> +		sort -t$'\t' -k4,4nr |
> +		if [ "$TOP" -gt 0 ]; then sed -n "1,${TOP}p"; else cat; fi |
> +		sort -t$'\t' -k5,5gr
> +}

[Severity: Low]
Will these tools/perf/ path checks fail to match report entries when the
build directory is inside tools/ (for example, make -C tools/perf O=build
coverage)?

llvm-cov report strips the longest common path prefix shared by all
covered files.  Because all covered files in perf and its internal
libraries live under tools/, a build directory inside tools/ causes
llvm-cov report to strip the <srctree>/tools/ prefix and emit paths
starting with perf/ (such as perf/util/evsel.c) instead of tools/perf/.
As a result, print_worst_files() prints nothing, file_cov() reports
focused files as not covered, and print_table() exits with an error that
there is no coverage data in $REPORT.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261117025305.3958882-1-acme@kernel.org?part=1

  reply	other threads:[~2026-11-17  3:21 UTC|newest]

Thread overview: 25+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-17  2:52 [PATCH v5 0/12] perf coverage: A harness for perf, and the fixes it found Arnaldo Carvalho de Melo
2026-11-17  2:52 ` [PATCH v5 01/12] perf scripts: Add a coverage harness for perf Arnaldo Carvalho de Melo
2026-11-17  3:21   ` sashiko-bot [this message]
2026-11-17  2:52 ` [PATCH v5 02/12] perf annotate: Handle absolute addressing in data type profiling Arnaldo Carvalho de Melo
2026-11-17  3:17   ` sashiko-bot
2026-11-17  2:52 ` [PATCH v5 03/12] perf tests: Exercise DW_OP_addrx global variables Arnaldo Carvalho de Melo
2026-11-17  3:06   ` sashiko-bot
2026-11-17  2:52 ` [PATCH v5 04/12] perf tests: Exercise the flexible array member path Arnaldo Carvalho de Melo
2026-11-17  3:03   ` sashiko-bot
2026-11-17  2:52 ` [PATCH v5 05/12] perf annotate-data: Count flexible array accesses for the whole struct Arnaldo Carvalho de Melo
2026-11-17  3:01   ` sashiko-bot
2026-11-17  2:52 ` [PATCH v5 06/12] perf tests: Exercise data type resolution from a register Arnaldo Carvalho de Melo
2026-11-17  2:59   ` sashiko-bot
2026-11-17  2:52 ` [PATCH v5 07/12] perf annotate: Print the stats after the annotations Arnaldo Carvalho de Melo
2026-11-17  3:01   ` sashiko-bot
2026-11-17  2:53 ` [PATCH v5 08/12] perf tests: Silence the libbfd probe in build-test Arnaldo Carvalho de Melo
2026-11-17  2:56   ` sashiko-bot
2026-11-17  2:53 ` [PATCH v5 09/12] perf build: Record the display-only features in FEATURE-DUMP Arnaldo Carvalho de Melo
2026-11-17  3:04   ` sashiko-bot
2026-11-17  2:53 ` [PATCH v5 10/12] perf tests: Fix the Intel PT virtual LBR check Arnaldo Carvalho de Melo
2026-11-17  3:04   ` sashiko-bot
2026-11-17  2:53 ` [PATCH v5 11/12] perf test: Drop samples without an IP in the dlfilter test Arnaldo Carvalho de Melo
2026-11-17  3:02   ` sashiko-bot
2026-11-17  2:53 ` [PATCH v5 12/12] perf test: Skip the LBR inline unwinding check when unsupported Arnaldo Carvalho de Melo
2026-11-17  3:04   ` sashiko-bot

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=sashiko-outbox-218136@kernel.org \
    --to=sashiko-bot@kernel.org \
    --cc=acme@kernel.org \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=sashiko-reviews@lists.linux.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox