All of lore.kernel.org
 help / color / mirror / Atom feed
From: Namhyung Kim <namhyung@kernel.org>
To: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Ingo Molnar <mingo@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	James Clark <james.clark@linaro.org>,
	Jiri Olsa <jolsa@kernel.org>, Ian Rogers <irogers@google.com>,
	Adrian Hunter <adrian.hunter@intel.com>,
	Clark Williams <williams@redhat.com>,
	linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org,
	Arnaldo Carvalho de Melo <acme@redhat.com>
Subject: Re: [PATCH v6 02/13] perf scripts: Add a coverage harness for perf
Date: Tue, 17 Nov 2026 22:56:59 -0800	[thread overview]
Message-ID: <av1MO74yqgr2OJt7@google.com> (raw)
In-Reply-To: <20261117200433.45174-3-acme@kernel.org>

Hi Arnaldo,

On Tue, Nov 17, 2026 at 05:04:20PM -0300, Arnaldo Carvalho de Melo wrote:
> From: Arnaldo Carvalho de Melo <acme@redhat.com>
> 
> Build perf with clang source based coverage, run the complete perf test
> suite, merge the raw profiles and report line and function coverage,
> optionally against a baseline.  This is pahole's coverage/coverage.sh +
> coverage-diff.sh merged into one script, wired as
> 'make -C tools/perf coverage' and listed in 'make help'.
> 
> Without -f the report lists the files with the most uncovered lines,
> which is where new tests should try to get more coverage; -t narrows
> the tests, -f focuses on specific files, --html writes a browsable
> report and --table/-T, also available as 'make -C tools/perf
> coverage-table', prints the last run as an ASCII table with the
> per-file line and function coverage, like pahole's coverage_table.py.

It'd be nice if you could list all options.

> 
> The build can be split from the test run: 'make -C tools/perf
> coverage-prep' builds the instrumented perf and records the build
> directory, and a later run as root, which needs no make, checks that

Ok, so IIUC it's optional and just running 'make coverage' would all
necessary jobs one stop, right?


> build, runs the tests and renders the report, or prints the table with
> --table.  This is for running the tests that need root on the bare
> metal host while the instrumented perf is built in the development
> container.

What is the table?

> 
> The report covers the whole tree, not only the paths the tests walk, so
> the per-file table is a map of what the suite does not reach in a given
> run.  The uncovered line list and the failures the runs exposed are what
> drove the rest of this series.
> 
>   $ make -C tools/perf coverage-prep O=~/coverage-build
>   ...

What is the actual command line here?  I guess it's..

    $ sudo ~/coverage-build/perf test


>   $ sudo scripts/coverage.sh
>   ...
>   Total: 104787 of 205810 lines covered (50.91%)
>     perf: 85674 of 173247 lines covered (49.45%)
> 
>     largest uncovered files (>= 100 lines)
>     file                                            lines     cover    missed  fn-cover
>     util/intel-pt.c                                  3237     0.00%      3237     0.00%
>     util/intel-pt-decoder/intel-pt-decoder.c         3208     0.00%      3208     0.00%
>     ui/browsers/hists.c                              2730     0.00%      2730     0.00%
>     builtin-sched.c                                  3737    41.26%      2195    43.97%
>     builtin-script.c                                 3212    33.50%      2136    37.90%
>     util/auxtrace.c                                  2013     4.47%      1923    11.11%
>     util/session.c                                   2686    41.14%      1581    48.80%
>     ...
>   Report:  /home/acme/coverage-build/cov/report.txt
> 
> The total also counts the tools/ libraries and headers linked into
> perf, the perf line is the tools/perf/ files alone.  The numbers above
> are from the split run: coverage-prep built the instrumented perf as the
> developer, the tests and the report ran as root.  This machine has no
> Intel PT, so the PT tests skip and leave intel-pt.c and
> intel-pt-decoder.c at 0.00%; the TUI browser is in the list because the
> suite does not drive it.
> 
> 'perf test' exits 0 even when tests fail or time out, so the script
> takes its status from the "Failed tests :" and "Timed out tests :" lines
> that perf test prints since commit 33f20342ba525ef7 ("perf test: Add
> summary reporting").
> 
> 168 tests passed, 28 skipped, one failure (20.2: Tracepoint event), in
> code this series does not change.

Looks useful, thanks!

Namhyung

> 
> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> ---

  parent reply	other threads:[~2026-11-18  6:57 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-17 20:04 [PATCH v6 0/13] perf coverage: A harness for perf, and the fixes it found Arnaldo Carvalho de Melo
2026-11-17 20:04 ` [PATCH v6 01/13] perf jvmti: Link the agent with EXTRA_CFLAGS Arnaldo Carvalho de Melo
2026-11-17 20:10   ` sashiko-bot
2026-11-17 20:04 ` [PATCH v6 02/13] perf scripts: Add a coverage harness for perf Arnaldo Carvalho de Melo
2026-11-17 20:15   ` sashiko-bot
2026-11-18  6:56   ` Namhyung Kim [this message]
2026-11-17 20:04 ` [PATCH v6 03/13] perf annotate: Handle absolute addressing in data type profiling Arnaldo Carvalho de Melo
2026-11-17 20:35   ` sashiko-bot
2026-11-18  7:08   ` Namhyung Kim
2026-11-17 20:04 ` [PATCH v6 04/13] perf tests: Exercise DW_OP_addrx global variables Arnaldo Carvalho de Melo
2026-11-17 20:13   ` sashiko-bot
2026-11-17 20:04 ` [PATCH v6 05/13] perf tests: Exercise the flexible array member path Arnaldo Carvalho de Melo
2026-11-17 20:11   ` sashiko-bot
2026-11-17 20:04 ` [PATCH v6 06/13] perf annotate-data: Count flexible array accesses for the whole struct Arnaldo Carvalho de Melo
2026-11-17 20:10   ` sashiko-bot
2026-11-17 20:04 ` [PATCH v6 07/13] perf tests: Exercise data type resolution from a register Arnaldo Carvalho de Melo
2026-11-17 20:20   ` sashiko-bot
2026-11-18  7:16   ` Namhyung Kim
2026-11-17 20:04 ` [PATCH v6 08/13] perf annotate: Print the stats after the annotations Arnaldo Carvalho de Melo
2026-11-17 20:11   ` sashiko-bot
2026-11-17 20:04 ` [PATCH v6 09/13] perf tests: Silence the libbfd probe in build-test Arnaldo Carvalho de Melo
2026-11-17 20:08   ` sashiko-bot
2026-11-17 20:04 ` [PATCH v6 10/13] perf build: Record the display-only features in FEATURE-DUMP Arnaldo Carvalho de Melo
2026-11-17 20:15   ` sashiko-bot
2026-11-17 20:04 ` [PATCH v6 11/13] perf tests: Fix the Intel PT virtual LBR check Arnaldo Carvalho de Melo
2026-11-17 20:18   ` sashiko-bot
2026-11-17 20:04 ` [PATCH v6 12/13] perf test: Drop samples without an IP in the dlfilter test Arnaldo Carvalho de Melo
2026-11-17 20:12   ` sashiko-bot
2026-11-17 20:04 ` [PATCH v6 13/13] perf test: Skip the LBR inline unwinding check when unsupported Arnaldo Carvalho de Melo
2026-11-17 20:29   ` sashiko-bot

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=av1MO74yqgr2OJt7@google.com \
    --to=namhyung@kernel.org \
    --cc=acme@kernel.org \
    --cc=acme@redhat.com \
    --cc=adrian.hunter@intel.com \
    --cc=irogers@google.com \
    --cc=james.clark@linaro.org \
    --cc=jolsa@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=mingo@kernel.org \
    --cc=tglx@linutronix.de \
    --cc=williams@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.