The Linux Kernel Mailing List
 help / color / mirror / Atom feed
From: Arnaldo Carvalho de Melo <acme@kernel.org>
To: Namhyung Kim <namhyung@kernel.org>
Cc: Ingo Molnar <mingo@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	James Clark <james.clark@linaro.org>,
	Jiri Olsa <jolsa@kernel.org>, Ian Rogers <irogers@google.com>,
	Adrian Hunter <adrian.hunter@intel.com>,
	Clark Williams <williams@redhat.com>,
	linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org,
	Arnaldo Carvalho de Melo <acme@kernel.org>
Subject: [PATCH v11 00/19] perf coverage: A harness for perf, and the fixes it found
Date: Sun, 22 Nov 2026 18:40:06 -0300	[thread overview]
Message-ID: <20261122214025.16769-1-acme@kernel.org> (raw)

Hi,

Nothing measured how much of perf the test suite reaches.  Patches 18-19
add coverage.sh, which builds perf with clang source based coverage, runs
the full test suite and reports line and function coverage, and
coverage-table.sh, which renders the biggest uncovered gaps over the
whole tree, not only the files the tests already touch.

Patches 1-7 are the fixes the first runs found, 8-17 their tests and the
test fixes.  1 is a build fix for the GTK UI, which the coverage build
needs; 5 and 6 are parser changes, not regression fixes.

  - The GTK UI was not linked with EXTRA_CFLAGS, so the coverage build
    never saw its counters (1).

  - Absolute addressing was missing from the data type resolver (2);
    movabs was not parsed (6).

  - --insn-stat was printed before the samples were accounted (3).

  - FEATURE-DUMP did not record features checked outside FEATURE_TESTS
    (4).

  - The flexible array member row showed 0.00% while the member said
    100.00% (7).

  - Tests cover the x86 operand parser (8), DW_OP_addrx globals (9, test
    only: fixed upstream by c3991e42f58a3650), the flexible array path
    (10), a struct pointer kept in a register at -O2 (11) and annotate
    stats (12).
  - Intel PT virtual LBR, dlfilter zeroed IP and LBR inline unwinding
    test fixes (13-15).

The full root run used for this series, with clang DWARF 4 on this AMD
host, measured 112185 of 236044 lines (47.53%), 86374 of 174097 in
tools/perf/ (49.61%), 170 passed, 160 subtests, 27 skipped and one
failure (21.2, the known Tracepoint event kernel regression).  The total
includes libbpf and the dlfilters; the Rust workload is built and run but
is not instrumented.

That one failure makes the coverage target exit 1 after writing the
report and the baseline; the data above is from that completed run.

On aarch64 (Raspberry Pi 4, Cortex-A72, clang 19), the full suite as
root, run sequentially, measured 101119 of 232961 lines (43.41%), 75800
of 171070 in tools/perf/ (44.31%), 142 passed, 133 subtests, 47 skipped,
two timed out and one failed (130, the stat test; 17 now selects an
absent CPU instead of an online one).  The parallel test mode starts
every test at once and times out on the four cores, hence --sequential
and --timeout (18).

The harness, the tests and the review fixes were developed with an AI
coding assistant (OpenCode): it drafted the scripts and tests, ran the
coverage probes, and proposed the fixes for the review findings.  The
latest edits were built and checked with the operand test and test-list
matching on toolbx.

What changed from v10:

  - 2: recognize absolute addressing in both 32- and 64-bit objects,
    while keeping the %gs:40 stack canary handling.

  - 4: record late feature probes independently of FEATURE_DISPLAY.

  - 5, 6: parse movabs and record x86 immediates in a 64-bit field, so
    GNU and LLVM spellings of the KASAN shadow offset track the same
    value.

  - 7: count flexible-array accesses at the struct boundary, including
    tail padding.

  - 8: exercise both absolute-address conversions and reject invalid
    operands; drop the call case the data type resolver does not follow.

  - 10: test a flexible array after struct tail padding in all output modes.

  - 11: restrict the annotation to touch and verify the register path is
    not resolved by instruction tracking.

  - 12: run instruction statistics without requiring memory events and
    check the totals against the rows.

  - 15: skip the LBR test only on the unsupported branch-stack error.

  - 16: skip trace-summary BPF tests unless the running kernel exposes BTF.

  - 17: avoid online CPUs in the offline stat test, while retaining
    the check for an absent CPU.

  - 18: preserve inherited CXXFLAGS, build with clang, rebuild dlfilters,
    trim -t fragments, and support sequential and timed runs.

  - 19: keep the coverage-table filtering and end-of-run table output
    checks, including decimal --min-lines values.

Best regards,

- Arnaldo

Arnaldo Carvalho de Melo (19):
  perf build: Link the GTK UI with EXTRA_CFLAGS
  perf annotate: Handle absolute addressing in data type profiling
  perf annotate: Print the stats after the annotations
  perf build: Record the features checked outside FEATURE_TESTS
  perf annotate-x86: Record immediates in a 64-bit field
  perf annotate-x86: Parse movabs like a mov
  perf annotate-data: Count flexible array accesses for the whole struct
  perf test: Exercise the x86 operand parser
  perf test: Exercise DW_OP_addrx global variables
  perf test: Exercise the flexible array member path
  perf test: Exercise data type resolution from a register
  perf test: Exercise the annotate stats output
  perf test: Fix the Intel PT virtual LBR check
  perf test: Drop samples without an IP in the dlfilter test
  perf test: Skip the LBR inline unwinding check when unsupported
  perf test: Skip the trace summary BPF tests without kernel BTF
  perf test: Avoid online CPUs in the offline stat test
  perf scripts: Add a coverage harness for perf
  perf scripts: Add a coverage table

 tools/perf/.gitignore                         |   1 +
 tools/perf/Documentation/perf-annotate.txt    |   2 +-
 tools/perf/Makefile                           |  15 +-
 tools/perf/Makefile.config                    |   8 +
 tools/perf/Makefile.perf                      |  12 +-
 tools/perf/builtin-annotate.c                 |  20 +-
 tools/perf/scripts/coverage-table.sh          | 337 ++++++++++++
 tools/perf/scripts/coverage.sh                | 493 ++++++++++++++++++
 tools/perf/tests/Build                        |   1 +
 tools/perf/tests/annotate-ops.c               | 172 ++++++
 tools/perf/tests/builtin-test.c               |   4 +
 tools/perf/tests/shell/addr2line_inlines.sh   |  19 +-
 tools/perf/tests/shell/annotate_stats.sh      | 172 ++++++
 .../perf/tests/shell/data_type_flex_array.sh  | 135 +++++
 tools/perf/tests/shell/data_type_global.sh    |  73 +++
 tools/perf/tests/shell/data_type_register.sh  |  86 +++
 tools/perf/tests/shell/script_dlfilter.sh     |   2 +-
 tools/perf/tests/shell/stat.sh                |  21 +-
 tools/perf/tests/shell/test_intel_pt.sh       |  10 +-
 tools/perf/tests/shell/trace_summary.sh       |   9 +
 tools/perf/tests/tests.h                      |   4 +
 tools/perf/tests/workloads/Build              |   6 +
 tools/perf/tests/workloads/flex_array.c       |  71 +++
 tools/perf/tests/workloads/global_data.c      |  57 ++
 tools/perf/tests/workloads/regptr.c           |  60 +++
 tools/perf/ui/browsers/annotate-data.c        |   7 +-
 tools/perf/util/annotate-arch/annotate-x86.c  |  51 +-
 tools/perf/util/annotate-data.c               |  27 +-
 tools/perf/util/annotate-data.h               |   8 +
 tools/perf/util/annotate.c                    |  31 +-
 tools/perf/util/annotate.h                    |  13 +-
 31 files changed, 1881 insertions(+), 46 deletions(-)
 create mode 100755 tools/perf/scripts/coverage-table.sh
 create mode 100755 tools/perf/scripts/coverage.sh
 create mode 100644 tools/perf/tests/annotate-ops.c
 create mode 100755 tools/perf/tests/shell/annotate_stats.sh
 create mode 100755 tools/perf/tests/shell/data_type_flex_array.sh
 create mode 100755 tools/perf/tests/shell/data_type_global.sh
 create mode 100755 tools/perf/tests/shell/data_type_register.sh
 create mode 100644 tools/perf/tests/workloads/flex_array.c
 create mode 100644 tools/perf/tests/workloads/global_data.c
 create mode 100644 tools/perf/tests/workloads/regptr.c

-- 
2.56.1


             reply	other threads:[~2026-11-22 21:40 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-22 21:40 Arnaldo Carvalho de Melo [this message]
2026-11-22 21:40 ` [PATCH v11 01/19] perf build: Link the GTK UI with EXTRA_CFLAGS Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 02/19] perf annotate: Handle absolute addressing in data type profiling Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 03/19] perf annotate: Print the stats after the annotations Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 04/19] perf build: Record the features checked outside FEATURE_TESTS Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 05/19] perf annotate-x86: Record immediates in a 64-bit field Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 06/19] perf annotate-x86: Parse movabs like a mov Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 07/19] perf annotate-data: Count flexible array accesses for the whole struct Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 08/19] perf test: Exercise the x86 operand parser Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 09/19] perf test: Exercise DW_OP_addrx global variables Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 10/19] perf test: Exercise the flexible array member path Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 11/19] perf test: Exercise data type resolution from a register Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 12/19] perf test: Exercise the annotate stats output Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 13/19] perf test: Fix the Intel PT virtual LBR check Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 14/19] perf test: Drop samples without an IP in the dlfilter test Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 15/19] perf test: Skip the LBR inline unwinding check when unsupported Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 16/19] perf test: Skip the trace summary BPF tests without kernel BTF Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 17/19] perf test: Avoid online CPUs in the offline stat test Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 18/19] perf scripts: Add a coverage harness for perf Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 19/19] perf scripts: Add a coverage table Arnaldo Carvalho de Melo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261122214025.16769-1-acme@kernel.org \
    --to=acme@kernel.org \
    --cc=adrian.hunter@intel.com \
    --cc=irogers@google.com \
    --cc=james.clark@linaro.org \
    --cc=jolsa@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=mingo@kernel.org \
    --cc=namhyung@kernel.org \
    --cc=tglx@linutronix.de \
    --cc=williams@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox