From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=100.103.45.18 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1794883997; b=vjtbPdBW4JJ7d0qsT6q+3h1J+glR++t6t/r7szfPR96/nGjGodDsG9EPYY0jlFUEI0td IRio3AFLbS/YT9iqp/XEALMUfd496Ikg8hu1TIryyCDsUTyU9SMIAdH1AMV7SxxOBjwf2 +fFguopMJZYEfKyrBgfHQxbuBtkdbKa67vP1WUDmD2toEyfuOOmUAm1bC8WA0WI8P9ivl SagUKWOquXSCzNGMUqnoxGYgEapLQpaeKA1a5zELyeH/kKq3+HTO/y1j7nQfTQdR5y4LB kCZpVygZaOds5avDd5WvwQ0OhISn9gRq0F8T3+zTtkoDsV3nkTPvga1VmaGGSIc670g== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1794883997; h=Received:DKIM-Signature:From:To:Cc:Subject:Date:Message-ID:X-Mailer: In-Reply-To:References:MIME-Version:Content-Type: Content-Transfer-Encoding; bh=xKtuf93XNJ2mdKgmEusi+siqNg2jILUBeycLm7evfj0=; b=BcpnMvcRs0KaVQNuCw5bJMup8WnQcJl2sKBQLlI/PhNAOM2sno7QI96a/0s05gNEORDO fVaClWmC440wURRzhtLBjnRtRqjIGsd2tD/yH/6jGwytgXPbSPehRp1T//+9m6HGee0EQ bt4I5+Lgcaw0NjLT9hnIUerJ0JfbqmKKyIoVRHEH31CyUg5RO/be9/UpzA425LVOAPpsK QwM58GzodPpl7+i2aO9x68uy92find0tQrnE7w0MVZYuKkvoYq9NfzGveYAVWsJR7btgQ BtkJHX44ckxBh0EwfNX8+A0Im2UJnMs3XbsYv/T+lPyL6ETfssW8Er65XM/GiWvnkZQ== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=lcqx1Fmm; arc=none smtp.remote-ip=100.103.45.18 Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 36024606A333; Tue, 17 Nov 2026 02:53:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=lcqx1Fmm Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4071E1F000C5; Tue, 17 Nov 2026 02:53:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1794883996; bh=xKtuf93XNJ2mdKgmEusi+siqNg2jILUBeycLm7evfj0=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=lcqx1FmmS3lxxW1mggJLVHXkEjVb4wmcRslwJ1HdCq1KAvT7h2G5m0YKKwbs8Yd7+ 473Xs6HXmjkPWznki+KgppDx9Lz0fYt3r+Ya4vKKrx23jgIU1p9nH05QXpvlwfvcXj ghGdMZ7MoLVr84LkFjDFuGgYlQehHZTJ1DyP7WrMV1E7heg2qJ8hqwPPyR2hD8Uw2K /4FRwOpwD3NcSs7176sWJrAof7yLMKerFYu8kmJUglMfLlGBBr/6g9H6JaOXHNZSJW qn9fPTmoJkAV+clSpbKjnJo2lLFJ/clAreyNubhBsh2TVC9QI8ATc8ghFkbtGhhDps 4R7rYc7k9VxLA== From: Arnaldo Carvalho de Melo To: Namhyung Kim Cc: Ingo Molnar , Thomas Gleixner , James Clark , Jiri Olsa , Ian Rogers , Adrian Hunter , Clark Williams , linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Arnaldo Carvalho de Melo Subject: [PATCH v5 01/12] perf scripts: Add a coverage harness for perf Date: Mon, 16 Nov 2026 23:52:53 -0300 Message-ID: <20261117025305.3958882-2-acme@kernel.org> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261117025305.3958882-1-acme@kernel.org> References: <20261117025305.3958882-1-acme@kernel.org> X-Mailing-List: linux-perf-users@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit From: Arnaldo Carvalho de Melo Build perf with clang source based coverage, run the complete perf test suite, merge the raw profiles and report line and function coverage for the whole tree, optionally against a baseline. This is pahole's coverage/coverage.sh + coverage-diff.sh merged into one script, wired as 'make -C tools/perf coverage' and listed in 'make help'. Without -f the report lists the files with the most uncovered lines, which is where new tests should try to get more coverage; -t narrows the tests, -f focuses on specific files, --html writes a browsable report and --table/-T, also available as 'make -C tools/perf coverage-table', prints the last run as an ASCII table with the per-file line and function coverage, like pahole's coverage_table.py. The build can be split from the test run: 'make -C tools/perf coverage-prep' builds the instrumented perf and records the build directory, and a later run as root, which needs no make, checks that build, runs the tests and renders the report, or prints the table with --table. This is for running the tests that need root on the bare metal host while the instrumented perf is built in the development container. The report covers the whole tree, not only the paths the tests walk, so the per-file table is a map of what the suite does not reach in a given run. The uncovered line list and the failures the runs exposed are what drove the rest of this series. $ make -C tools/perf coverage-prep O=~/coverage-build ... $ sudo scripts/coverage.sh ... Total: 104292 of 205810 lines covered (50.67%) largest uncovered files (>= 100 lines) file lines cover missed fn-cover util/intel-pt.c 3237 0.00% 3237 0.00% util/intel-pt-decoder/intel-pt-decoder.c 3208 0.00% 3208 0.00% ui/browsers/hists.c 2730 0.00% 2730 0.00% builtin-sched.c 3737 41.29% 2194 43.97% builtin-script.c 3212 33.50% 2136 37.90% util/auxtrace.c 2013 4.47% 1923 11.11% util/session.c 2686 41.14% 1581 48.80% ... Report: /home/acme/coverage-build/cov/report.txt The numbers above are from the split run: coverage-prep built the instrumented perf as the developer, the tests and the report ran as root. This machine has no Intel PT, so the PT tests skip and leave intel-pt.c and intel-pt-decoder.c at 0.0%; the TUI browser is in the list because the suite does not drive it. 168 tests passed, 28 skipped, one failure (20.2: Tracepoint event), in code this series does not change. Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/.gitignore | 1 + tools/perf/Makefile | 11 +- tools/perf/Makefile.perf | 4 + tools/perf/scripts/coverage.sh | 738 +++++++++++++++++++++++++++++++++ 4 files changed, 753 insertions(+), 1 deletion(-) create mode 100755 tools/perf/scripts/coverage.sh diff --git a/tools/perf/.gitignore b/tools/perf/.gitignore index 3b968c5158b822ce..9052f25597ac9489 100644 --- a/tools/perf/.gitignore +++ b/tools/perf/.gitignore @@ -27,6 +27,7 @@ stats-*.csv cscope* config.mak config.mak.autogen +.coverage-build-dir *-bison.* *-flex.* *.pyc diff --git a/tools/perf/Makefile b/tools/perf/Makefile index 00baf90e5bda7123..8df8e9ed37f0a8f5 100644 --- a/tools/perf/Makefile +++ b/tools/perf/Makefile @@ -140,6 +140,15 @@ check-headers: sync-headers: @./scripts/sync-headers.sh $(SYNC_HEADERS_ARGS) +coverage-prep: + @./scripts/coverage.sh --prep $(if $(FULL_O),--build-dir=$(FULL_O)) $(COVERAGE_ARGS) + +coverage: + @./scripts/coverage.sh $(if $(FULL_O),--build-dir=$(FULL_O)) $(COVERAGE_ARGS) + +coverage-table: + @./scripts/coverage.sh --table $(if $(FULL_O),--build-dir=$(FULL_O)) $(COVERAGE_ARGS) + # # All other targets get passed through: # @@ -147,4 +156,4 @@ sync-headers: $(print_msg) $(make) -.PHONY: all clean tags TAGS FORCE Makefile build-test build-test-tarball check-headers sync-headers +.PHONY: all clean tags TAGS FORCE Makefile build-test build-test-tarball check-headers sync-headers coverage coverage-prep coverage-table diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf index b74b534e1153ffcb..782929b0ba10d8f3 100644 --- a/tools/perf/Makefile.perf +++ b/tools/perf/Makefile.perf @@ -873,6 +873,10 @@ help: @echo ' check-headers - check which header copies are out of sync with the kernel' @echo ' sync-headers - sync those header copies with the kernel, creating commits' @echo ' (SYNC_HEADERS_ARGS="..." passes options to the script)' + @echo ' coverage-prep - build perf with clang coverage, ready for a root test run' + @echo ' coverage - build perf with clang coverage, run the tests and report' + @echo ' coverage-table - print an ASCII table of the last coverage run' + @echo ' (COVERAGE_ARGS="..." passes options to the script)' DOC_TARGETS := doc man html info pdf diff --git a/tools/perf/scripts/coverage.sh b/tools/perf/scripts/coverage.sh new file mode 100755 index 0000000000000000..aa2649cf08c4ad74 --- /dev/null +++ b/tools/perf/scripts/coverage.sh @@ -0,0 +1,738 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# +# Copyright (C) 2026, Red Hat Inc, Arnaldo Carvalho de Melo +# +# LLVM source-based coverage for perf: build with clang instrumentation, +# run the complete perf test suite and report line/function coverage for +# perf, optionally comparing it against a previously saved report. -t +# narrows the tests and -f adds files to the per-file table; without -f +# the report lists the files with the most uncovered lines, which is +# where new tests are needed. --table prints the last run as an ASCII +# coverage table, like pahole's coverage_table.py. The build can be +# split from the test run: --prep builds the instrumented perf and +# records the build directory, and a later run as root uses that build, +# skips the build and goes straight to the tests. The sampling based +# tests move the percentages by a percent or two between identical runs. + +set -euo pipefail + +# awk prints dots and printf/sort parse them: keep the numeric locale fixed +# so --table doesn't abort where the decimal separator is a comma. +export LC_ALL=C + +PERF_DIR=$(cd "$(dirname "$0")/.." && pwd) + +DEFAULT_BUILD_DIR=/tmp/build/perf-coverage +BUILD_DIR=${BUILD_DIR:-} +COV_DIR=${COV_DIR:-} +BUILD_DIR_SET=0 +COV_DIR_SET=0 +[ -n "$BUILD_DIR" ] && BUILD_DIR_SET=1 +[ -n "$COV_DIR" ] && COV_DIR_SET=1 +JOBS=${JOBS:-$(nproc 2>/dev/null || echo 1)} +LLVM_PROFDATA=${LLVM_PROFDATA:-llvm-profdata} +LLVM_COV=${LLVM_COV:-llvm-cov} + +DEBUG_BUILD=1 +SKIP_BUILD=0 +REPORT_ONLY=0 +PREP=0 +TABLE=0 +HTML=0 +USE_BASELINE=1 +BASELINE= +UNCOVERED=${UNCOVERED:-20} +TOP=${TOP:-20} +MIN_LINES=${MIN_LINES:-100} +DWARF_VERSION=${DWARF_VERSION:-4} +TESTS=() + +FOCUS=() + +ROOT=0 +[ "$(id -u)" = 0 ] && ROOT=1 + +say() { printf '\033[1;34m==>\033[0m %s\n' "$*"; } +warn() { printf '\033[0;33mwarning:\033[0m %s\n' "$*" >&2; } +die() { printf '\033[0;31merror:\033[0m %s\n' "$*" >&2; exit 1; } + +usage() +{ + cat < /dev/null || + die "$LLVM_COV not found (set LLVM_COV)" + fi + if [ "$SKIP_BUILD" = 0 ]; then + command -v clang > /dev/null || + die "clang not found, it is the only compiler with -fcoverage-mapping" + fi + if [ "$PREP" = 0 ] && [ "$REPORT_ONLY" = 0 ]; then + command -v "$LLVM_PROFDATA" > /dev/null || + die "$LLVM_PROFDATA not found (set LLVM_PROFDATA)" + fi + mkdir -p "$COV_DIR" +fi + +# +# Build: clang, DEBUG=1 gives -O0 -g like pahole's coverage build, which +# keeps the line mapping honest at the cost of slower test runs. DWARF 4 +# is the default; --dwarf-version 5 describes the workloads linked into +# perf with DW_OP_addrx, the encoding clang uses for globals in DWARF 5. +# +check_coverage_build() +{ + [ -x "$BUILD_DIR/perf" ] || + die "no $BUILD_DIR/perf, run 'make -C $PERF_DIR O=$BUILD_DIR coverage-prep' first" + if command -v nm > /dev/null; then + # No -q: grep reads all of nm's output, so pipefail does + # not see a SIGPIPE from an early exit. + nm "$BUILD_DIR/perf" 2>/dev/null | + grep '__llvm_profile' > /dev/null || + die "$BUILD_DIR/perf was not built for coverage, run coverage-prep" + else + warn "nm not found, cannot verify that $BUILD_DIR/perf is instrumented" + fi +} + +write_state() +{ + mkdir -p "$(dirname "$STATE_FILE")" || + die "could not create $(dirname "$STATE_FILE")" + { + printf 'BUILD_DIR=%s\n' "$BUILD_DIR" + printf 'COV_DIR=%s\n' "$COV_DIR" + printf 'DEBUG_BUILD=%s\n' "$DEBUG_BUILD" + printf 'DWARF_VERSION=%s\n' "$DWARF_VERSION" + } > "$STATE_FILE" || die "could not write $STATE_FILE" + + if ! printf '%s\n' "$BUILD_DIR" > "$POINTER" 2>/dev/null; then + warn "could not write $POINTER, root runs will need -b $BUILD_DIR" + fi +} + +build_perf() +{ + if [ "$SKIP_BUILD" = 1 ]; then + check_coverage_build + return + fi + + say "Building perf in $BUILD_DIR" + mkdir -p "$BUILD_DIR" + local debug=() + [ "$DEBUG_BUILD" = 1 ] && debug=(DEBUG=1) + + if ! make -C "$PERF_DIR" O="$BUILD_DIR" CC=clang \ + "${debug[@]+"${debug[@]}"}" \ + EXTRA_CFLAGS="-fprofile-instr-generate -fcoverage-mapping -gdwarf-$DWARF_VERSION" \ + -j"$JOBS" > "$BUILD_LOG" 2>&1; then + tail -20 "$BUILD_LOG" >&2 + die "build failed, full log in $BUILD_LOG" + fi + tail -1 "$BUILD_LOG" + write_state +} + +# +# Run the selected tests with the instrumented perf in PATH: the shell +# tests, the workloads and 'perf test' itself all come from there. +# +run_tests() +{ + if [ ${#TESTS[@]} -eq 0 ]; then + say "Running the complete perf test suite" + else + say "Running perf tests: ${TESTS[*]}" + fi + rm -rf "$COV_DIR/raw" + mkdir -p "$COV_DIR/raw" + + set +e + # The shell tests only get discovered from the kernel tree root. + (cd "$PERF_DIR/../.." && + PATH="$BUILD_DIR:$PATH" \ + LLVM_PROFILE_FILE="$COV_DIR/raw/perf-%p-%m.profraw" \ + "$BUILD_DIR/perf" test -v \ + "${TESTS[@]+"${TESTS[@]}"}") 2>&1 | tee "$TEST_LOG" + test_status=${PIPESTATUS[0]} + set -e + + # 'perf test' exits 0 even when tests fail, only the summary tells. + if [ "$test_status" = 0 ] && grep -qE '^Failed tests +: +[1-9]' "$TEST_LOG"; then + test_status=1 + fi + + grep -E '^(Passed|Skipped|Failed) .*:' "$TEST_LOG" || true + [ "$test_status" = 0 ] || + warn "some tests failed or were skipped, coverage is still collected" + if [ "$test_status" != 0 ] && grep -q 'Access to performance monitoring' "$TEST_LOG"; then + warn "perf_event_paranoid=$(cat /proc/sys/kernel/perf_event_paranoid 2>/dev/null)" \ + "denied events to tests recording with callchains;" \ + "run as root or lower it for the full suite" + fi +} + +# +# Merge the per-process profiles and render the text report. +# +merge_profiles() +{ + local count size unusable=0 + count=$(find "$COV_DIR/raw" -name '*.profraw' | wc -l) + size=$(du -sh "$COV_DIR/raw" 2>/dev/null | cut -f1) + [ "$count" -gt 0 ] || + die "no .profraw files, was perf rebuilt with -fprofile-instr-generate?" + + say "Merging $count profile files ($size)" + if ! "$LLVM_PROFDATA" merge -sparse "$COV_DIR"/raw/*.profraw \ + -o "$PROFDATA" 2> "$COV_DIR/merge.log"; then + # A file left half written, by a perf killed at exit or by a full + # tmpfs, makes llvm-profdata drop the whole merge and with it the + # run's measurement: retry keeping what can be read and say how + # much was lost. --failure-mode=warn exists since LLVM 14, the + # oldest llvm-profdata the test images ship. + warn "some profile files are unusable, merging the rest" + "$LLVM_PROFDATA" merge -sparse --failure-mode=warn \ + "$COV_DIR"/raw/*.profraw -o "$PROFDATA" \ + 2> "$COV_DIR/merge.log" || + die "llvm-profdata merge failed, see $COV_DIR/merge.log" + unusable=$(grep -c '^warning:' "$COV_DIR/merge.log" || true) + [ -n "$unusable" ] || unusable=0 + [ "$unusable" -lt "$count" ] || + die "no usable profile among $count, see $COV_DIR/merge.log" + warn "merged $((count - unusable)) profile files, ignored $unusable unusable" + fi +} + +llvm_cov_objects() +{ + echo "$BUILD_DIR/perf" + local so + for so in perf.so libperf.so libperf-jvmti.so; do + [ -f "$BUILD_DIR/$so" ] && echo "--object=$BUILD_DIR/$so" + done + # The Python extension is named after the interpreter ABI and lives + # in its own directory. + for so in "$BUILD_DIR"/python/perf*.so; do + [ -f "$so" ] && echo "--object=$so" + done +} + +render_report() +{ + local objects + mapfile -t objects < <(llvm_cov_objects) + + say "Generating report" + "$LLVM_COV" report "${objects[@]}" --instr-profile="$PROFDATA" > "$REPORT" +} + +# +# Line coverage of a full report for one file: "lines missed cover fn%". +# Report paths are relative to the build directory, so match the requested +# path as a suffix: a build outside the source tree gives +# ".../tools/perf/..." and one inside gives "tools/perf/...". +# +# -f takes the path as it shows in the report, or a tools/perf/ relative one. +focus_path() +{ + case "$1" in + *tools/perf/*) printf '%s\n' "${1#*tools/perf/}" ;; + *) printf '%s\n' "$1" ;; + esac +} + +file_cov() +{ + awk -v f="$2" ' + function perf_relpath(p) { + if (p ~ /(^|\/)tools\/perf\//) + sub(/^.*tools\/perf\//, "", p) + return p + } + { + g = f + sub(/^.*tools\/perf\//, "", g) + rel = perf_relpath($1) + if (rel == f || rel == g || $1 == f) { + print $8, $9, $10, $7 + exit + } + }' "$1" +} + +total_cov() +{ + awk '$1 == "TOTAL" { print $8, $9, $10 }' "$1" +} + +total_delta() +{ + local prev=$1 cur=$2 old cur_cov + [ -f "$prev" ] || return 0 + old=$(total_cov "$prev" | awk '{print $3}') + cur_cov=$(total_cov "$cur" | awk '{print $3}') + [ -n "$old" ] && [ -n "$cur_cov" ] || return 0 + awk -v cur="$cur_cov" -v old="$old" 'BEGIN { + sub(/%$/, "", cur); sub(/%$/, "", old) + printf "%+.2f%%", cur - old }' +} + +report_delta() +{ + local prev=$1 cur=$2 label + local cur_cov old_cov delta + label=$(focus_path "$3") + cur_cov=$(file_cov "$cur" "$label" | awk '{print $3}') + [ -n "$cur_cov" ] || return 0 + + if [ -f "$prev" ]; then + old_cov=$(file_cov "$prev" "$label" | awk '{print $3}') + if [ -n "$old_cov" ]; then + delta=$(awk -v cur="$cur_cov" -v old="$old_cov" 'BEGIN { + sub(/%$/, "", cur); sub(/%$/, "", old) + printf "%+.2f%%", cur - old }') + else + delta="(new)" + fi + else + old_cov="-" + delta="-" + fi + printf " %-36s %8s %8s %9s\n" "$label" "${old_cov:--}" "$cur_cov" "$delta" +} + +# +# With no -f, show where the uncovered lines are: the files with the most +# missed lines are where new tests should try to get more coverage. +# +print_worst_files() +{ + [ "$TOP" -gt 0 ] || return 0 + local rows missed lines cover fn path + rows=$(awk -v min="$MIN_LINES" ' + $1 ~ /(^|\/)tools\/perf\// && $8 >= min && $9 > 0 { + path = $1 + sub(/^.*tools\/perf\//, "", path) + printf "%s %s %s %s %s\n", $9, $8, $10, $7, path + }' "$REPORT" | sort -k1,1nr -k2,2nr | sed -n "1,${TOP}p") + [ -n "$rows" ] || return 0 + + printf " largest uncovered files (>= %s lines)\n" "$MIN_LINES" + printf " %-44s %8s %8s %8s %8s\n" \ + "file" "lines" "cover" "missed" "fn-cover" + while read -r missed lines cover fn path; do + printf " %-44s %8s %8s %8s %8s\n" \ + "$path" "$lines" "$cover" "$missed" "$fn" + done <<< "$rows" +} + +print_summary() +{ + local lines missed cover delta file cov + read -r lines missed cover < <(total_cov "$REPORT") + printf "\n\033[1mTotal:\033[0m %s of %s lines covered (%s)" \ + "$((lines - missed))" "$lines" "$cover" + if [ "$USE_BASELINE" = 1 ] && [ -f "$BASELINE" ] && + ! cmp -s "$BASELINE" "$REPORT"; then + delta=$(total_delta "$BASELINE" "$REPORT") + [ -n "$delta" ] && printf " (previous %s, %s)" \ + "$(total_cov "$BASELINE" | awk '{print $3}')" "$delta" + fi + printf "\n\n" + + if [ ${#FOCUS[@]} -eq 0 ]; then + print_worst_files + return + fi + + printf " \033[1m%-36s\033[0m %8s %6s %6s %8s\n" \ + "file" "lines" "cover" "missed" "fn-cover" + for file in "${FOCUS[@]}"; do + file=$(focus_path "$file") + cov=$(file_cov "$REPORT" "$file") + if [ -z "$cov" ]; then + printf " %-36s %8s\n" "$file" "not covered" + continue + fi + printf " %-36s %8s %6s %6s %8s\n" "$file" \ + "$(echo "$cov" | awk '{print $1}')" \ + "$(echo "$cov" | awk '{print $3}')" \ + "$(echo "$cov" | awk '{print $2}')" \ + "$(echo "$cov" | awk '{print $4}')" + done + + if [ "$USE_BASELINE" = 1 ] && [ -f "$BASELINE" ] && + ! cmp -s "$BASELINE" "$REPORT"; then + printf "\n per file delta since %s\n" "$BASELINE" + printf " %-36s %8s %8s %9s\n" "file" "previous" "current" "delta" + for file in "${FOCUS[@]}"; do + report_delta "$BASELINE" "$REPORT" "$file" + done + fi +} + +# +# --table: render the report of the last run as an ASCII table, like +# pahole's coverage_table.py. Without -f the rows are the TOP files with +# the most uncovered lines, sorted by line coverage; --top 0 lists every +# file with at least MIN_LINES lines and uncovered lines. The total is +# the whole report, which also counts the libraries perf links with. +# +bar() +{ + local pct=$1 width=$2 filled i out= + filled=$(awk -v p="$pct" -v w="$width" 'BEGIN { printf "%d", p * w / 100 }') + for ((i = 0; i < filled; i++)); do out+=█; done + for ((i = filled; i < width; i++)); do out+=░; done + printf '%s' "$out" +} + +center() +{ + local left=$(( ($2 - ${#1}) / 2 )) + printf '%*s%s%*s' "$left" '' "$1" "$(( $2 - ${#1} - left ))" '' +} + +dashes() +{ + local s + printf -v s '%*s' "$1" '' + printf '%s' "${s// /─}" +} + +table_row() +{ + awk -v f="$2" ' + function perf_relpath(p) { + if (p ~ /(^|\/)tools\/perf\//) + sub(/^.*tools\/perf\//, "", p) + return p + } + { + g = f + sub(/^.*tools\/perf\//, "", g) + path = perf_relpath($1) + if (path == f || path == g || $1 == f) { + printf "%s\t%s\t%d\t%d\t%.2f\t%s\t%d\t%s\n", + path, $8, $8 - $9, $9, $10, $5, $5 - $6, $7 + exit + } + }' "$1" +} + +table_rows() +{ + if [ ${#FOCUS[@]} -gt 0 ]; then + local file + for file in "${FOCUS[@]}"; do + table_row "$REPORT" "$file" + done | sort -t$'\t' -k5,5gr + return + fi + + awk -v min="$MIN_LINES" ' + $1 ~ /(^|\/)tools\/perf\// && $8 >= min && $9 > 0 { + path = $1 + sub(/^.*tools\/perf\//, "", path) + printf "%s\t%s\t%d\t%d\t%.2f\t%s\t%d\t%s\n", + path, $8, $8 - $9, $9, $10, $5, $5 - $6, $7 + }' "$REPORT" | + sort -t$'\t' -k4,4nr | + if [ "$TOP" -gt 0 ]; then sed -n "1,${TOP}p"; else cat; fi | + sort -t$'\t' -k5,5gr +} + +print_table() +{ + local rows total path lines hit missed pct funcs funcs_hit func_cov + local row_fmt w d_file d29 d13 d10 d8 + + rows=$(table_rows) + [ -n "$rows" ] || + die "no coverage data in $REPORT for the requested files" + total=$(awk '$1 == "TOTAL" { + printf "%s\t%d\t%d\t%.2f\t%s\t%d\t%s\n", + $8, $8 - $9, $9, $10, $5, $5 - $6, $7 + }' "$REPORT") + + w=4 + while IFS=$'\t' read -r path _; do + if [ ${#path} -gt "$w" ]; then + w=${#path} + fi + done <<< "$rows" + + d8=$(dashes 8) + d10=$(dashes 10) + d13=$(dashes 13) + d29=$(dashes 29) + d_file=$(dashes $((w + 2))) + + printf '┌%s┬%s┬%s┬%s┬%s┬%s┬%s┐\n' \ + "$d_file" "$d8" "$d8" "$d8" "$d29" "$d13" "$d10" + printf '│ %-*s │ %6s │ %6s │ %6s │ %s │ %11s │ %8s │\n' \ + "$w" "File" "Lines" "Hit" "Miss" \ + "$(center 'Line Coverage' 27)" "Funcs" "FuncCov" + printf '├%s┼%s┼%s┼%s┼%s┼%s┼%s┤\n' \ + "$d_file" "$d8" "$d8" "$d8" "$d29" "$d13" "$d10" + + row_fmt=$(printf '│ %%-%ds │ %%6s │ %%6s │ %%6s │ %%s %%5.1f%%%%' "$w") + row_fmt+=' │ %5s/%-5s │ %8s │\n' + + while IFS=$'\t' read -r path lines hit missed pct funcs funcs_hit func_cov; do + printf "$row_fmt" "$path" "$lines" "$hit" "$missed" \ + "$(bar "$pct" 20)" "$pct" "$funcs_hit" "$funcs" "$func_cov" + done <<< "$rows" + + printf '├%s┼%s┼%s┼%s┼%s┼%s┼%s┤\n' \ + "$d_file" "$d8" "$d8" "$d8" "$d29" "$d13" "$d10" + while IFS=$'\t' read -r lines hit missed pct funcs funcs_hit func_cov; do + printf "$row_fmt" "TOTAL" "$lines" "$hit" "$missed" \ + "$(bar "$pct" 20)" "$pct" "$funcs_hit" "$funcs" "$func_cov" + done <<< "$total" + printf '└%s┴%s┴%s┴%s┴%s┴%s┴%s┘\n' \ + "$d_file" "$d8" "$d8" "$d8" "$d29" "$d13" "$d10" +} + +print_uncovered() +{ + [ "$UNCOVERED" -gt 0 ] || return 0 + [ ${#FOCUS[@]} -gt 0 ] || return 0 + local objects file out lines header=0 + mapfile -t objects < <(llvm_cov_objects) + + for file in "${FOCUS[@]}"; do + file=$(focus_path "$file") + # No report row means the file was not built: print_summary() + # already said so, llvm-cov show would dump the whole profile. + [ -n "$(file_cov "$REPORT" "$file")" ] || continue + out=$("$LLVM_COV" show "${objects[@]}" --instr-profile="$PROFDATA" \ + "$PERF_DIR/$file" 2>/dev/null | + awk -F'|' '$2 ~ /^[[:space:]]*0[[:space:]]*$/ { + gsub(/[[:space:]]/, "", $1); printf "%s ", $1 }') + lines=$(echo "$out" | wc -w) + [ "$lines" -gt 0 ] || continue + if [ "$header" = 0 ]; then + printf "\n uncovered lines (first %s per file)\n" "$UNCOVERED" + header=1 + fi + printf " %-24s %4s lines: %s\n" "$file" "$lines" \ + "$(echo "$out" | cut -d' ' -f1-"$UNCOVERED")" + done +} + +save_html() +{ + [ "$HTML" = 1 ] || return 0 + local objects + mapfile -t objects < <(llvm_cov_objects) + say "Writing HTML report to $COV_DIR/html" + "$LLVM_COV" show "${objects[@]}" --instr-profile="$PROFDATA" \ + --format=html --output-dir="$COV_DIR/html" \ + --show-line-counts-or-regions --show-branches=count +} + +check_stale() +{ + [ "$REPORT_ONLY" = 1 ] || return 0 + if [ ! -f "$PROFDATA" ]; then + [ "$TABLE" = 1 ] && return 0 + die "no $PROFDATA, run without --report-only first" + fi + local newer + newer=$(find "$PERF_DIR" -name '*.c' -newer "$PROFDATA" -print -quit 2>/dev/null) + if [ -n "$newer" ]; then + warn "profiles are older than source changes (e.g. $newer), results may be stale" + fi +} + +update_baseline() +{ + [ "$USE_BASELINE" = 1 ] || return 0 + [ "$REPORT_ONLY" = 1 ] && return 0 + [ -f "$BASELINE" ] && mv "$BASELINE" "$BASELINE.prev" + cp "$REPORT" "$BASELINE" + printf "\nBaseline saved: %s\n" "$BASELINE" +} + +test_status=0 + +if [ "$TABLE" = 1 ]; then + [ -f "$REPORT" ] || die "no $REPORT, run coverage first" + check_stale + print_table + exit 0 +fi + +if [ "$PREP" = 1 ]; then + build_perf + write_state + say "Coverage build ready: $BUILD_DIR/perf" + printf "As root: %s/scripts/coverage.sh\n" "$PERF_DIR" + exit 0 +fi + +if [ "$REPORT_ONLY" = 0 ]; then + build_perf + run_tests + merge_profiles +else + check_coverage_build + check_stale +fi + +[ -f "$PROFDATA" ] || die "no $PROFDATA, run without --report-only first" + +render_report +print_summary +print_uncovered +save_html +update_baseline + +printf "\nReport: %s\n" "$REPORT" +[ "$HTML" = 1 ] && printf "HTML: %s\n" "$COV_DIR/html/index.html" + +exit "$test_status" -- 2.56.0