From: Arnaldo Carvalho de Melo <acme@kernel.org>
To: Namhyung Kim <namhyung@kernel.org>
Cc: Ingo Molnar <mingo@kernel.org>,
Thomas Gleixner <tglx@linutronix.de>,
James Clark <james.clark@linaro.org>,
Jiri Olsa <jolsa@kernel.org>, Ian Rogers <irogers@google.com>,
Adrian Hunter <adrian.hunter@intel.com>,
Clark Williams <williams@redhat.com>,
linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org,
Arnaldo Carvalho de Melo <acme@kernel.org>
Subject: [PATCH v11 00/19] perf coverage: A harness for perf, and the fixes it found
Date: Sun, 22 Nov 2026 18:40:06 -0300 [thread overview]
Message-ID: <20261122214025.16769-1-acme@kernel.org> (raw)
Hi,
Nothing measured how much of perf the test suite reaches. Patches 18-19
add coverage.sh, which builds perf with clang source based coverage, runs
the full test suite and reports line and function coverage, and
coverage-table.sh, which renders the biggest uncovered gaps over the
whole tree, not only the files the tests already touch.
Patches 1-7 are the fixes the first runs found, 8-17 their tests and the
test fixes. 1 is a build fix for the GTK UI, which the coverage build
needs; 5 and 6 are parser changes, not regression fixes.
- The GTK UI was not linked with EXTRA_CFLAGS, so the coverage build
never saw its counters (1).
- Absolute addressing was missing from the data type resolver (2);
movabs was not parsed (6).
- --insn-stat was printed before the samples were accounted (3).
- FEATURE-DUMP did not record features checked outside FEATURE_TESTS
(4).
- The flexible array member row showed 0.00% while the member said
100.00% (7).
- Tests cover the x86 operand parser (8), DW_OP_addrx globals (9, test
only: fixed upstream by c3991e42f58a3650), the flexible array path
(10), a struct pointer kept in a register at -O2 (11) and annotate
stats (12).
- Intel PT virtual LBR, dlfilter zeroed IP and LBR inline unwinding
test fixes (13-15).
The full root run used for this series, with clang DWARF 4 on this AMD
host, measured 112185 of 236044 lines (47.53%), 86374 of 174097 in
tools/perf/ (49.61%), 170 passed, 160 subtests, 27 skipped and one
failure (21.2, the known Tracepoint event kernel regression). The total
includes libbpf and the dlfilters; the Rust workload is built and run but
is not instrumented.
That one failure makes the coverage target exit 1 after writing the
report and the baseline; the data above is from that completed run.
On aarch64 (Raspberry Pi 4, Cortex-A72, clang 19), the full suite as
root, run sequentially, measured 101119 of 232961 lines (43.41%), 75800
of 171070 in tools/perf/ (44.31%), 142 passed, 133 subtests, 47 skipped,
two timed out and one failed (130, the stat test; 17 now selects an
absent CPU instead of an online one). The parallel test mode starts
every test at once and times out on the four cores, hence --sequential
and --timeout (18).
The harness, the tests and the review fixes were developed with an AI
coding assistant (OpenCode): it drafted the scripts and tests, ran the
coverage probes, and proposed the fixes for the review findings. The
latest edits were built and checked with the operand test and test-list
matching on toolbx.
What changed from v10:
- 2: recognize absolute addressing in both 32- and 64-bit objects,
while keeping the %gs:40 stack canary handling.
- 4: record late feature probes independently of FEATURE_DISPLAY.
- 5, 6: parse movabs and record x86 immediates in a 64-bit field, so
GNU and LLVM spellings of the KASAN shadow offset track the same
value.
- 7: count flexible-array accesses at the struct boundary, including
tail padding.
- 8: exercise both absolute-address conversions and reject invalid
operands; drop the call case the data type resolver does not follow.
- 10: test a flexible array after struct tail padding in all output modes.
- 11: restrict the annotation to touch and verify the register path is
not resolved by instruction tracking.
- 12: run instruction statistics without requiring memory events and
check the totals against the rows.
- 15: skip the LBR test only on the unsupported branch-stack error.
- 16: skip trace-summary BPF tests unless the running kernel exposes BTF.
- 17: avoid online CPUs in the offline stat test, while retaining
the check for an absent CPU.
- 18: preserve inherited CXXFLAGS, build with clang, rebuild dlfilters,
trim -t fragments, and support sequential and timed runs.
- 19: keep the coverage-table filtering and end-of-run table output
checks, including decimal --min-lines values.
Best regards,
- Arnaldo
Arnaldo Carvalho de Melo (19):
perf build: Link the GTK UI with EXTRA_CFLAGS
perf annotate: Handle absolute addressing in data type profiling
perf annotate: Print the stats after the annotations
perf build: Record the features checked outside FEATURE_TESTS
perf annotate-x86: Record immediates in a 64-bit field
perf annotate-x86: Parse movabs like a mov
perf annotate-data: Count flexible array accesses for the whole struct
perf test: Exercise the x86 operand parser
perf test: Exercise DW_OP_addrx global variables
perf test: Exercise the flexible array member path
perf test: Exercise data type resolution from a register
perf test: Exercise the annotate stats output
perf test: Fix the Intel PT virtual LBR check
perf test: Drop samples without an IP in the dlfilter test
perf test: Skip the LBR inline unwinding check when unsupported
perf test: Skip the trace summary BPF tests without kernel BTF
perf test: Avoid online CPUs in the offline stat test
perf scripts: Add a coverage harness for perf
perf scripts: Add a coverage table
tools/perf/.gitignore | 1 +
tools/perf/Documentation/perf-annotate.txt | 2 +-
tools/perf/Makefile | 15 +-
tools/perf/Makefile.config | 8 +
tools/perf/Makefile.perf | 12 +-
tools/perf/builtin-annotate.c | 20 +-
tools/perf/scripts/coverage-table.sh | 337 ++++++++++++
tools/perf/scripts/coverage.sh | 493 ++++++++++++++++++
tools/perf/tests/Build | 1 +
tools/perf/tests/annotate-ops.c | 172 ++++++
tools/perf/tests/builtin-test.c | 4 +
tools/perf/tests/shell/addr2line_inlines.sh | 19 +-
tools/perf/tests/shell/annotate_stats.sh | 172 ++++++
.../perf/tests/shell/data_type_flex_array.sh | 135 +++++
tools/perf/tests/shell/data_type_global.sh | 73 +++
tools/perf/tests/shell/data_type_register.sh | 86 +++
tools/perf/tests/shell/script_dlfilter.sh | 2 +-
tools/perf/tests/shell/stat.sh | 21 +-
tools/perf/tests/shell/test_intel_pt.sh | 10 +-
tools/perf/tests/shell/trace_summary.sh | 9 +
tools/perf/tests/tests.h | 4 +
tools/perf/tests/workloads/Build | 6 +
tools/perf/tests/workloads/flex_array.c | 71 +++
tools/perf/tests/workloads/global_data.c | 57 ++
tools/perf/tests/workloads/regptr.c | 60 +++
tools/perf/ui/browsers/annotate-data.c | 7 +-
tools/perf/util/annotate-arch/annotate-x86.c | 51 +-
tools/perf/util/annotate-data.c | 27 +-
tools/perf/util/annotate-data.h | 8 +
tools/perf/util/annotate.c | 31 +-
tools/perf/util/annotate.h | 13 +-
31 files changed, 1881 insertions(+), 46 deletions(-)
create mode 100755 tools/perf/scripts/coverage-table.sh
create mode 100755 tools/perf/scripts/coverage.sh
create mode 100644 tools/perf/tests/annotate-ops.c
create mode 100755 tools/perf/tests/shell/annotate_stats.sh
create mode 100755 tools/perf/tests/shell/data_type_flex_array.sh
create mode 100755 tools/perf/tests/shell/data_type_global.sh
create mode 100755 tools/perf/tests/shell/data_type_register.sh
create mode 100644 tools/perf/tests/workloads/flex_array.c
create mode 100644 tools/perf/tests/workloads/global_data.c
create mode 100644 tools/perf/tests/workloads/regptr.c
--
2.56.1
next reply other threads:[~2026-11-22 21:40 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-22 21:40 Arnaldo Carvalho de Melo [this message]
2026-11-22 21:40 ` [PATCH v11 01/19] perf build: Link the GTK UI with EXTRA_CFLAGS Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 02/19] perf annotate: Handle absolute addressing in data type profiling Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 03/19] perf annotate: Print the stats after the annotations Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 04/19] perf build: Record the features checked outside FEATURE_TESTS Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 05/19] perf annotate-x86: Record immediates in a 64-bit field Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 06/19] perf annotate-x86: Parse movabs like a mov Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 07/19] perf annotate-data: Count flexible array accesses for the whole struct Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 08/19] perf test: Exercise the x86 operand parser Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 09/19] perf test: Exercise DW_OP_addrx global variables Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 10/19] perf test: Exercise the flexible array member path Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 11/19] perf test: Exercise data type resolution from a register Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 12/19] perf test: Exercise the annotate stats output Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 13/19] perf test: Fix the Intel PT virtual LBR check Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 14/19] perf test: Drop samples without an IP in the dlfilter test Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 15/19] perf test: Skip the LBR inline unwinding check when unsupported Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 16/19] perf test: Skip the trace summary BPF tests without kernel BTF Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 17/19] perf test: Avoid online CPUs in the offline stat test Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 18/19] perf scripts: Add a coverage harness for perf Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 19/19] perf scripts: Add a coverage table Arnaldo Carvalho de Melo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261122214025.16769-1-acme@kernel.org \
--to=acme@kernel.org \
--cc=adrian.hunter@intel.com \
--cc=irogers@google.com \
--cc=james.clark@linaro.org \
--cc=jolsa@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=namhyung@kernel.org \
--cc=tglx@linutronix.de \
--cc=williams@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox