The Linux Kernel Mailing List
 help / color / mirror / Atom feed
From: Namhyung Kim <namhyung@kernel.org>
To: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Ingo Molnar <mingo@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	James Clark <james.clark@linaro.org>,
	Jiri Olsa <jolsa@kernel.org>, Ian Rogers <irogers@google.com>,
	Adrian Hunter <adrian.hunter@intel.com>,
	Clark Williams <williams@redhat.com>,
	linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org,
	Arnaldo Carvalho de Melo <acme@redhat.com>
Subject: Re: [PATCH v6 02/13] perf scripts: Add a coverage harness for perf
Date: Tue, 17 Nov 2026 22:56:59 -0800	[thread overview]
Message-ID: <av1MO74yqgr2OJt7@google.com> (raw)
In-Reply-To: <20261117200433.45174-3-acme@kernel.org>

Hi Arnaldo,

On Tue, Nov 17, 2026 at 05:04:20PM -0300, Arnaldo Carvalho de Melo wrote:
> From: Arnaldo Carvalho de Melo <acme@redhat.com>
> 
> Build perf with clang source based coverage, run the complete perf test
> suite, merge the raw profiles and report line and function coverage,
> optionally against a baseline.  This is pahole's coverage/coverage.sh +
> coverage-diff.sh merged into one script, wired as
> 'make -C tools/perf coverage' and listed in 'make help'.
> 
> Without -f the report lists the files with the most uncovered lines,
> which is where new tests should try to get more coverage; -t narrows
> the tests, -f focuses on specific files, --html writes a browsable
> report and --table/-T, also available as 'make -C tools/perf
> coverage-table', prints the last run as an ASCII table with the
> per-file line and function coverage, like pahole's coverage_table.py.

It'd be nice if you could list all options.

> 
> The build can be split from the test run: 'make -C tools/perf
> coverage-prep' builds the instrumented perf and records the build
> directory, and a later run as root, which needs no make, checks that

Ok, so IIUC it's optional and just running 'make coverage' would all
necessary jobs one stop, right?


> build, runs the tests and renders the report, or prints the table with
> --table.  This is for running the tests that need root on the bare
> metal host while the instrumented perf is built in the development
> container.

What is the table?

> 
> The report covers the whole tree, not only the paths the tests walk, so
> the per-file table is a map of what the suite does not reach in a given
> run.  The uncovered line list and the failures the runs exposed are what
> drove the rest of this series.
> 
>   $ make -C tools/perf coverage-prep O=~/coverage-build
>   ...

What is the actual command line here?  I guess it's..

    $ sudo ~/coverage-build/perf test


>   $ sudo scripts/coverage.sh
>   ...
>   Total: 104787 of 205810 lines covered (50.91%)
>     perf: 85674 of 173247 lines covered (49.45%)
> 
>     largest uncovered files (>= 100 lines)
>     file                                            lines     cover    missed  fn-cover
>     util/intel-pt.c                                  3237     0.00%      3237     0.00%
>     util/intel-pt-decoder/intel-pt-decoder.c         3208     0.00%      3208     0.00%
>     ui/browsers/hists.c                              2730     0.00%      2730     0.00%
>     builtin-sched.c                                  3737    41.26%      2195    43.97%
>     builtin-script.c                                 3212    33.50%      2136    37.90%
>     util/auxtrace.c                                  2013     4.47%      1923    11.11%
>     util/session.c                                   2686    41.14%      1581    48.80%
>     ...
>   Report:  /home/acme/coverage-build/cov/report.txt
> 
> The total also counts the tools/ libraries and headers linked into
> perf, the perf line is the tools/perf/ files alone.  The numbers above
> are from the split run: coverage-prep built the instrumented perf as the
> developer, the tests and the report ran as root.  This machine has no
> Intel PT, so the PT tests skip and leave intel-pt.c and
> intel-pt-decoder.c at 0.00%; the TUI browser is in the list because the
> suite does not drive it.
> 
> 'perf test' exits 0 even when tests fail or time out, so the script
> takes its status from the "Failed tests :" and "Timed out tests :" lines
> that perf test prints since commit 33f20342ba525ef7 ("perf test: Add
> summary reporting").
> 
> 168 tests passed, 28 skipped, one failure (20.2: Tracepoint event), in
> code this series does not change.

Looks useful, thanks!

Namhyung

> 
> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> ---

  reply	other threads:[~2026-11-18  6:57 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-17 20:04 [PATCH v6 0/13] perf coverage: A harness for perf, and the fixes it found Arnaldo Carvalho de Melo
2026-11-17 20:04 ` [PATCH v6 01/13] perf jvmti: Link the agent with EXTRA_CFLAGS Arnaldo Carvalho de Melo
2026-11-17 20:04 ` [PATCH v6 02/13] perf scripts: Add a coverage harness for perf Arnaldo Carvalho de Melo
2026-11-18  6:56   ` Namhyung Kim [this message]
2026-11-17 20:04 ` [PATCH v6 03/13] perf annotate: Handle absolute addressing in data type profiling Arnaldo Carvalho de Melo
2026-11-18  7:08   ` Namhyung Kim
2026-11-17 20:04 ` [PATCH v6 04/13] perf tests: Exercise DW_OP_addrx global variables Arnaldo Carvalho de Melo
2026-11-17 20:04 ` [PATCH v6 05/13] perf tests: Exercise the flexible array member path Arnaldo Carvalho de Melo
2026-11-17 20:04 ` [PATCH v6 06/13] perf annotate-data: Count flexible array accesses for the whole struct Arnaldo Carvalho de Melo
2026-11-17 20:04 ` [PATCH v6 07/13] perf tests: Exercise data type resolution from a register Arnaldo Carvalho de Melo
2026-11-18  7:16   ` Namhyung Kim
2026-11-17 20:04 ` [PATCH v6 08/13] perf annotate: Print the stats after the annotations Arnaldo Carvalho de Melo
2026-11-17 20:04 ` [PATCH v6 09/13] perf tests: Silence the libbfd probe in build-test Arnaldo Carvalho de Melo
2026-11-17 20:04 ` [PATCH v6 10/13] perf build: Record the display-only features in FEATURE-DUMP Arnaldo Carvalho de Melo
2026-11-17 20:04 ` [PATCH v6 11/13] perf tests: Fix the Intel PT virtual LBR check Arnaldo Carvalho de Melo
2026-11-17 20:04 ` [PATCH v6 12/13] perf test: Drop samples without an IP in the dlfilter test Arnaldo Carvalho de Melo
2026-11-17 20:04 ` [PATCH v6 13/13] perf test: Skip the LBR inline unwinding check when unsupported Arnaldo Carvalho de Melo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=av1MO74yqgr2OJt7@google.com \
    --to=namhyung@kernel.org \
    --cc=acme@kernel.org \
    --cc=acme@redhat.com \
    --cc=adrian.hunter@intel.com \
    --cc=irogers@google.com \
    --cc=james.clark@linaro.org \
    --cc=jolsa@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=mingo@kernel.org \
    --cc=tglx@linutronix.de \
    --cc=williams@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox