The Linux Kernel Mailing List
 help / color / mirror / Atom feed
* [PATCH v11 00/19] perf coverage: A harness for perf, and the fixes it found
@ 2026-11-22 21:40 Arnaldo Carvalho de Melo
  2026-11-22 21:40 ` [PATCH v11 01/19] perf build: Link the GTK UI with EXTRA_CFLAGS Arnaldo Carvalho de Melo
                   ` (18 more replies)
  0 siblings, 19 replies; 20+ messages in thread
From: Arnaldo Carvalho de Melo @ 2026-11-22 21:40 UTC (permalink / raw)
  To: Namhyung Kim
  Cc: Ingo Molnar, Thomas Gleixner, James Clark, Jiri Olsa, Ian Rogers,
	Adrian Hunter, Clark Williams, linux-kernel, linux-perf-users,
	Arnaldo Carvalho de Melo

Hi,

Nothing measured how much of perf the test suite reaches.  Patches 18-19
add coverage.sh, which builds perf with clang source based coverage, runs
the full test suite and reports line and function coverage, and
coverage-table.sh, which renders the biggest uncovered gaps over the
whole tree, not only the files the tests already touch.

Patches 1-7 are the fixes the first runs found, 8-17 their tests and the
test fixes.  1 is a build fix for the GTK UI, which the coverage build
needs; 5 and 6 are parser changes, not regression fixes.

  - The GTK UI was not linked with EXTRA_CFLAGS, so the coverage build
    never saw its counters (1).

  - Absolute addressing was missing from the data type resolver (2);
    movabs was not parsed (6).

  - --insn-stat was printed before the samples were accounted (3).

  - FEATURE-DUMP did not record features checked outside FEATURE_TESTS
    (4).

  - The flexible array member row showed 0.00% while the member said
    100.00% (7).

  - Tests cover the x86 operand parser (8), DW_OP_addrx globals (9, test
    only: fixed upstream by c3991e42f58a3650), the flexible array path
    (10), a struct pointer kept in a register at -O2 (11) and annotate
    stats (12).
  - Intel PT virtual LBR, dlfilter zeroed IP and LBR inline unwinding
    test fixes (13-15).

The full root run used for this series, with clang DWARF 4 on this AMD
host, measured 112185 of 236044 lines (47.53%), 86374 of 174097 in
tools/perf/ (49.61%), 170 passed, 160 subtests, 27 skipped and one
failure (21.2, the known Tracepoint event kernel regression).  The total
includes libbpf and the dlfilters; the Rust workload is built and run but
is not instrumented.

That one failure makes the coverage target exit 1 after writing the
report and the baseline; the data above is from that completed run.

On aarch64 (Raspberry Pi 4, Cortex-A72, clang 19), the full suite as
root, run sequentially, measured 101119 of 232961 lines (43.41%), 75800
of 171070 in tools/perf/ (44.31%), 142 passed, 133 subtests, 47 skipped,
two timed out and one failed (130, the stat test; 17 now selects an
absent CPU instead of an online one).  The parallel test mode starts
every test at once and times out on the four cores, hence --sequential
and --timeout (18).

The harness, the tests and the review fixes were developed with an AI
coding assistant (OpenCode): it drafted the scripts and tests, ran the
coverage probes, and proposed the fixes for the review findings.  The
latest edits were built and checked with the operand test and test-list
matching on toolbx.

What changed from v10:

  - 2: recognize absolute addressing in both 32- and 64-bit objects,
    while keeping the %gs:40 stack canary handling.

  - 4: record late feature probes independently of FEATURE_DISPLAY.

  - 5, 6: parse movabs and record x86 immediates in a 64-bit field, so
    GNU and LLVM spellings of the KASAN shadow offset track the same
    value.

  - 7: count flexible-array accesses at the struct boundary, including
    tail padding.

  - 8: exercise both absolute-address conversions and reject invalid
    operands; drop the call case the data type resolver does not follow.

  - 10: test a flexible array after struct tail padding in all output modes.

  - 11: restrict the annotation to touch and verify the register path is
    not resolved by instruction tracking.

  - 12: run instruction statistics without requiring memory events and
    check the totals against the rows.

  - 15: skip the LBR test only on the unsupported branch-stack error.

  - 16: skip trace-summary BPF tests unless the running kernel exposes BTF.

  - 17: avoid online CPUs in the offline stat test, while retaining
    the check for an absent CPU.

  - 18: preserve inherited CXXFLAGS, build with clang, rebuild dlfilters,
    trim -t fragments, and support sequential and timed runs.

  - 19: keep the coverage-table filtering and end-of-run table output
    checks, including decimal --min-lines values.

Best regards,

- Arnaldo

Arnaldo Carvalho de Melo (19):
  perf build: Link the GTK UI with EXTRA_CFLAGS
  perf annotate: Handle absolute addressing in data type profiling
  perf annotate: Print the stats after the annotations
  perf build: Record the features checked outside FEATURE_TESTS
  perf annotate-x86: Record immediates in a 64-bit field
  perf annotate-x86: Parse movabs like a mov
  perf annotate-data: Count flexible array accesses for the whole struct
  perf test: Exercise the x86 operand parser
  perf test: Exercise DW_OP_addrx global variables
  perf test: Exercise the flexible array member path
  perf test: Exercise data type resolution from a register
  perf test: Exercise the annotate stats output
  perf test: Fix the Intel PT virtual LBR check
  perf test: Drop samples without an IP in the dlfilter test
  perf test: Skip the LBR inline unwinding check when unsupported
  perf test: Skip the trace summary BPF tests without kernel BTF
  perf test: Avoid online CPUs in the offline stat test
  perf scripts: Add a coverage harness for perf
  perf scripts: Add a coverage table

 tools/perf/.gitignore                         |   1 +
 tools/perf/Documentation/perf-annotate.txt    |   2 +-
 tools/perf/Makefile                           |  15 +-
 tools/perf/Makefile.config                    |   8 +
 tools/perf/Makefile.perf                      |  12 +-
 tools/perf/builtin-annotate.c                 |  20 +-
 tools/perf/scripts/coverage-table.sh          | 337 ++++++++++++
 tools/perf/scripts/coverage.sh                | 493 ++++++++++++++++++
 tools/perf/tests/Build                        |   1 +
 tools/perf/tests/annotate-ops.c               | 172 ++++++
 tools/perf/tests/builtin-test.c               |   4 +
 tools/perf/tests/shell/addr2line_inlines.sh   |  19 +-
 tools/perf/tests/shell/annotate_stats.sh      | 172 ++++++
 .../perf/tests/shell/data_type_flex_array.sh  | 135 +++++
 tools/perf/tests/shell/data_type_global.sh    |  73 +++
 tools/perf/tests/shell/data_type_register.sh  |  86 +++
 tools/perf/tests/shell/script_dlfilter.sh     |   2 +-
 tools/perf/tests/shell/stat.sh                |  21 +-
 tools/perf/tests/shell/test_intel_pt.sh       |  10 +-
 tools/perf/tests/shell/trace_summary.sh       |   9 +
 tools/perf/tests/tests.h                      |   4 +
 tools/perf/tests/workloads/Build              |   6 +
 tools/perf/tests/workloads/flex_array.c       |  71 +++
 tools/perf/tests/workloads/global_data.c      |  57 ++
 tools/perf/tests/workloads/regptr.c           |  60 +++
 tools/perf/ui/browsers/annotate-data.c        |   7 +-
 tools/perf/util/annotate-arch/annotate-x86.c  |  51 +-
 tools/perf/util/annotate-data.c               |  27 +-
 tools/perf/util/annotate-data.h               |   8 +
 tools/perf/util/annotate.c                    |  31 +-
 tools/perf/util/annotate.h                    |  13 +-
 31 files changed, 1881 insertions(+), 46 deletions(-)
 create mode 100755 tools/perf/scripts/coverage-table.sh
 create mode 100755 tools/perf/scripts/coverage.sh
 create mode 100644 tools/perf/tests/annotate-ops.c
 create mode 100755 tools/perf/tests/shell/annotate_stats.sh
 create mode 100755 tools/perf/tests/shell/data_type_flex_array.sh
 create mode 100755 tools/perf/tests/shell/data_type_global.sh
 create mode 100755 tools/perf/tests/shell/data_type_register.sh
 create mode 100644 tools/perf/tests/workloads/flex_array.c
 create mode 100644 tools/perf/tests/workloads/global_data.c
 create mode 100644 tools/perf/tests/workloads/regptr.c

-- 
2.56.1


^ permalink raw reply	[flat|nested] 20+ messages in thread

end of thread, other threads:[~2026-11-22 21:41 UTC | newest]

Thread overview: 20+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-22 21:40 [PATCH v11 00/19] perf coverage: A harness for perf, and the fixes it found Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 01/19] perf build: Link the GTK UI with EXTRA_CFLAGS Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 02/19] perf annotate: Handle absolute addressing in data type profiling Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 03/19] perf annotate: Print the stats after the annotations Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 04/19] perf build: Record the features checked outside FEATURE_TESTS Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 05/19] perf annotate-x86: Record immediates in a 64-bit field Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 06/19] perf annotate-x86: Parse movabs like a mov Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 07/19] perf annotate-data: Count flexible array accesses for the whole struct Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 08/19] perf test: Exercise the x86 operand parser Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 09/19] perf test: Exercise DW_OP_addrx global variables Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 10/19] perf test: Exercise the flexible array member path Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 11/19] perf test: Exercise data type resolution from a register Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 12/19] perf test: Exercise the annotate stats output Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 13/19] perf test: Fix the Intel PT virtual LBR check Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 14/19] perf test: Drop samples without an IP in the dlfilter test Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 15/19] perf test: Skip the LBR inline unwinding check when unsupported Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 16/19] perf test: Skip the trace summary BPF tests without kernel BTF Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 17/19] perf test: Avoid online CPUs in the offline stat test Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 18/19] perf scripts: Add a coverage harness for perf Arnaldo Carvalho de Melo
2026-11-22 21:40 ` [PATCH v11 19/19] perf scripts: Add a coverage table Arnaldo Carvalho de Melo

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox