From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=100.103.45.18 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1794872828; b=0n1o4I1FBJCXPC9dVxmk2HDxZzDyLDIMux9rd13ol9xPKU4NT7wNAGdgvyS5zspv6m83 +M9RM9Q80rRy5HPEkO426V+wXHhSKPvoklQLtwBd9jVSPHmdh6IaBiEFmAvyePws6tVnC V1+VFzg+MsKBCIVZIRtf4doO3ieSBG4patQuWZZugJ8odwxV5tKKmxM9Yv5hzuV0+z6cO RvN5iyGMeaUAMoRQKbxmLkkPO6a6fjGowb8BSB919bLKPxqngEtOGyKtb+c1RFeu4dC7j VOIynzY3E7VyPdmA1iGGSGjFdqgWwcMnO2K1E9Aq5hjULNB1rVRytk3JgcaLHgyuo9A== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1794872828; h=Received:DKIM-Signature:From:To:Cc:Subject:Date:Message-ID:X-Mailer: In-Reply-To:References:MIME-Version:Content-Type: Content-Transfer-Encoding; bh=QjVhvPQpjtKK/JDjod0UrlX0CtDLWUrbfB7ivXIkWEU=; b=xmL47MytJO1V2edT5m4d71tgqR4SYrGdS5cSAQYkJoTPHi97Mrbr7kwUa9fD5iog5EZL xJNKxL96+7aJ513D7FV1opXhCCw/v1gEJ1Q52EctO/xlM/Tfete30Xd2q3ulBE8ESi30F vIOizAaX2faDYR8S1SLHtwHcBKtdsOm2wXNtvsIcYtA6PQG6SOPiSftHgup4CVQ5pkJ0t xbF4M9YlgdiEFsQt519JWuJqDskqPesY2NG1bfYujHX7pFWUhmOUDr1ucqVLsWTSlTNHz b4MDW9euB3w3c9927/rPHMcUFPpZK0mYgLioHTLIm2nmRTx0OkmUIGeMl+33cFQi4/Q== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=moulO7e0; arc=none smtp.remote-ip=100.103.45.18 Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5139A606A323; Mon, 16 Nov 2026 23:47:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=moulO7e0 Received: by smtp.kernel.org (Postfix) with ESMTPSA id B41C51F000C4; Mon, 16 Nov 2026 23:47:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1794872828; bh=QjVhvPQpjtKK/JDjod0UrlX0CtDLWUrbfB7ivXIkWEU=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=moulO7e0+8OhD5QCsurAzA1NGetT/wl4CulrWmzNpvLCnGyJsZC3IYzOst99loF09 ihifnZSAD6tL6iiNObcdDcPO+DgQrruyKhLvTjeVfsWOFwnfaMYFhTzV1JcnFXkcM0 OidKjd0zUp1ZW5DRfUmxm/6mnDn09cO1UTISJ5wsaGqXxOhkQUjTL1v6cpbVWvqDjx pauvyj69zzQiAaU+6ZG5LM31C74ASCF9tOYYjPfrEpj92pLJeeoRIPXJOIGGUPU3ju fep1v8Ed5xPNtSGjE9CBKVN00hLrpqn3hpk1D0tek2mUNU7+sP7jayp7yRMEtgWD8d 48HEfrVaGBDNQ== From: Arnaldo Carvalho de Melo To: Namhyung Kim Cc: Ingo Molnar , Thomas Gleixner , James Clark , Jiri Olsa , Ian Rogers , Adrian Hunter , Clark Williams , linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Arnaldo Carvalho de Melo Subject: [PATCH v4 01/12] perf scripts: Add a coverage harness for perf Date: Mon, 16 Nov 2026 20:46:47 -0300 Message-ID: <20261116234659.3950531-2-acme@kernel.org> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261116234659.3950531-1-acme@kernel.org> References: <20261116234659.3950531-1-acme@kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit From: Arnaldo Carvalho de Melo Build perf with clang source based coverage, run the complete perf test suite, merge the raw profiles and report line and function coverage for the whole tree, optionally against a baseline. This is pahole's coverage/coverage.sh + coverage-diff.sh merged into one script, wired as 'make -C tools/perf coverage' and listed in 'make help'. Without -f the report lists the files with the most uncovered lines, which is where new tests should try to get more coverage; -t narrows the tests, -f focuses on specific files, --html writes a browsable report and --table/-T, also available as 'make -C tools/perf coverage-table', prints the last run as an ASCII table with the per-file line and function coverage, like pahole's coverage_table.py. The build can be split from the test run: 'make -C tools/perf coverage-prep' builds the instrumented perf and records the build directory, and a later run as root, which needs no make, checks that build, runs the tests and renders the report, or prints the table with --table. This is for running the tests that need root on the bare metal host while the instrumented perf is built in the development container. The report covers the whole tree, not only the paths the tests walk, so the per-file table is a map of what the suite does not reach in a given run. The uncovered line list and the failures the runs exposed are what drove the rest of this series. $ make -C tools/perf coverage-prep O=~/coverage-build ... $ sudo scripts/coverage.sh ... Total: 104798 of 205007 lines covered (51.12%) largest uncovered files (>= 100 lines) file lines cover missed fn-cover util/intel-pt.c 3237 0.00% 3237 0.00% util/intel-pt-decoder/intel-pt-decoder.c 3208 0.00% 3208 0.00% ui/browsers/hists.c 2730 0.00% 2730 0.00% builtin-sched.c 3737 41.24% 2196 43.97% builtin-script.c 3212 33.50% 2136 37.90% util/auxtrace.c 2013 4.47% 1923 11.11% util/session.c 2686 41.14% 1581 48.80% ... Report: /home/acme/coverage-build/cov/report.txt The numbers above are from the split run: coverage-prep built the instrumented perf as the developer, the tests and the report ran as root. This machine has no Intel PT, so the PT tests skip and leave intel-pt.c and intel-pt-decoder.c at 0.0%; the TUI browser is in the list because the suite does not drive it. 169 tests passed, 27 skipped, one failure (20.2: Tracepoint event), in code this series does not change. Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/.gitignore | 1 + tools/perf/Makefile | 11 +- tools/perf/Makefile.perf | 4 + tools/perf/scripts/coverage.sh | 738 +++++++++++++++++++++++++++++++++ 4 files changed, 753 insertions(+), 1 deletion(-) create mode 100755 tools/perf/scripts/coverage.sh diff --git a/tools/perf/.gitignore b/tools/perf/.gitignore index 3b968c5158b822ce..9052f25597ac9489 100644 --- a/tools/perf/.gitignore +++ b/tools/perf/.gitignore @@ -27,6 +27,7 @@ stats-*.csv cscope* config.mak config.mak.autogen +.coverage-build-dir *-bison.* *-flex.* *.pyc diff --git a/tools/perf/Makefile b/tools/perf/Makefile index 00baf90e5bda7123..8df8e9ed37f0a8f5 100644 --- a/tools/perf/Makefile +++ b/tools/perf/Makefile @@ -140,6 +140,15 @@ check-headers: sync-headers: @./scripts/sync-headers.sh $(SYNC_HEADERS_ARGS) +coverage-prep: + @./scripts/coverage.sh --prep $(if $(FULL_O),--build-dir=$(FULL_O)) $(COVERAGE_ARGS) + +coverage: + @./scripts/coverage.sh $(if $(FULL_O),--build-dir=$(FULL_O)) $(COVERAGE_ARGS) + +coverage-table: + @./scripts/coverage.sh --table $(if $(FULL_O),--build-dir=$(FULL_O)) $(COVERAGE_ARGS) + # # All other targets get passed through: # @@ -147,4 +156,4 @@ sync-headers: $(print_msg) $(make) -.PHONY: all clean tags TAGS FORCE Makefile build-test build-test-tarball check-headers sync-headers +.PHONY: all clean tags TAGS FORCE Makefile build-test build-test-tarball check-headers sync-headers coverage coverage-prep coverage-table diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf index b74b534e1153ffcb..782929b0ba10d8f3 100644 --- a/tools/perf/Makefile.perf +++ b/tools/perf/Makefile.perf @@ -873,6 +873,10 @@ help: @echo ' check-headers - check which header copies are out of sync with the kernel' @echo ' sync-headers - sync those header copies with the kernel, creating commits' @echo ' (SYNC_HEADERS_ARGS="..." passes options to the script)' + @echo ' coverage-prep - build perf with clang coverage, ready for a root test run' + @echo ' coverage - build perf with clang coverage, run the tests and report' + @echo ' coverage-table - print an ASCII table of the last coverage run' + @echo ' (COVERAGE_ARGS="..." passes options to the script)' DOC_TARGETS := doc man html info pdf diff --git a/tools/perf/scripts/coverage.sh b/tools/perf/scripts/coverage.sh new file mode 100755 index 0000000000000000..aa2649cf08c4ad74 --- /dev/null +++ b/tools/perf/scripts/coverage.sh @@ -0,0 +1,738 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# +# Copyright (C) 2026, Red Hat Inc, Arnaldo Carvalho de Melo +# +# LLVM source-based coverage for perf: build with clang instrumentation, +# run the complete perf test suite and report line/function coverage for +# perf, optionally comparing it against a previously saved report. -t +# narrows the tests and -f adds files to the per-file table; without -f +# the report lists the files with the most uncovered lines, which is +# where new tests are needed. --table prints the last run as an ASCII +# coverage table, like pahole's coverage_table.py. The build can be +# split from the test run: --prep builds the instrumented perf and +# records the build directory, and a later run as root uses that build, +# skips the build and goes straight to the tests. The sampling based +# tests move the percentages by a percent or two between identical runs. + +set -euo pipefail + +# awk prints dots and printf/sort parse them: keep the numeric locale fixed +# so --table doesn't abort where the decimal separator is a comma. +export LC_ALL=C + +PERF_DIR=$(cd "$(dirname "$0")/.." && pwd) + +DEFAULT_BUILD_DIR=/tmp/build/perf-coverage +BUILD_DIR=${BUILD_DIR:-} +COV_DIR=${COV_DIR:-} +BUILD_DIR_SET=0 +COV_DIR_SET=0 +[ -n "$BUILD_DIR" ] && BUILD_DIR_SET=1 +[ -n "$COV_DIR" ] && COV_DIR_SET=1 +JOBS=${JOBS:-$(nproc 2>/dev/null || echo 1)} +LLVM_PROFDATA=${LLVM_PROFDATA:-llvm-profdata} +LLVM_COV=${LLVM_COV:-llvm-cov} + +DEBUG_BUILD=1 +SKIP_BUILD=0 +REPORT_ONLY=0 +PREP=0 +TABLE=0 +HTML=0 +USE_BASELINE=1 +BASELINE= +UNCOVERED=${UNCOVERED:-20} +TOP=${TOP:-20} +MIN_LINES=${MIN_LINES:-100} +DWARF_VERSION=${DWARF_VERSION:-4} +TESTS=() + +FOCUS=() + +ROOT=0 +[ "$(id -u)" = 0 ] && ROOT=1 + +say() { printf '\033[1;34m==>\033[0m %s\n' "$*"; } +warn() { printf '\033[0;33mwarning:\033[0m %s\n' "$*" >&2; } +die() { printf '\033[0;31merror:\033[0m %s\n' "$*" >&2; exit 1; } + +usage() +{ + cat < /dev/null || + die "$LLVM_COV not found (set LLVM_COV)" + fi + if [ "$SKIP_BUILD" = 0 ]; then + command -v clang > /dev/null || + die "clang not found, it is the only compiler with -fcoverage-mapping" + fi + if [ "$PREP" = 0 ] && [ "$REPORT_ONLY" = 0 ]; then + command -v "$LLVM_PROFDATA" > /dev/null || + die "$LLVM_PROFDATA not found (set LLVM_PROFDATA)" + fi + mkdir -p "$COV_DIR" +fi + +# +# Build: clang, DEBUG=1 gives -O0 -g like pahole's coverage build, which +# keeps the line mapping honest at the cost of slower test runs. DWARF 4 +# is the default; --dwarf-version 5 describes the workloads linked into +# perf with DW_OP_addrx, the encoding clang uses for globals in DWARF 5. +# +check_coverage_build() +{ + [ -x "$BUILD_DIR/perf" ] || + die "no $BUILD_DIR/perf, run 'make -C $PERF_DIR O=$BUILD_DIR coverage-prep' first" + if command -v nm > /dev/null; then + # No -q: grep reads all of nm's output, so pipefail does + # not see a SIGPIPE from an early exit. + nm "$BUILD_DIR/perf" 2>/dev/null | + grep '__llvm_profile' > /dev/null || + die "$BUILD_DIR/perf was not built for coverage, run coverage-prep" + else + warn "nm not found, cannot verify that $BUILD_DIR/perf is instrumented" + fi +} + +write_state() +{ + mkdir -p "$(dirname "$STATE_FILE")" || + die "could not create $(dirname "$STATE_FILE")" + { + printf 'BUILD_DIR=%s\n' "$BUILD_DIR" + printf 'COV_DIR=%s\n' "$COV_DIR" + printf 'DEBUG_BUILD=%s\n' "$DEBUG_BUILD" + printf 'DWARF_VERSION=%s\n' "$DWARF_VERSION" + } > "$STATE_FILE" || die "could not write $STATE_FILE" + + if ! printf '%s\n' "$BUILD_DIR" > "$POINTER" 2>/dev/null; then + warn "could not write $POINTER, root runs will need -b $BUILD_DIR" + fi +} + +build_perf() +{ + if [ "$SKIP_BUILD" = 1 ]; then + check_coverage_build + return + fi + + say "Building perf in $BUILD_DIR" + mkdir -p "$BUILD_DIR" + local debug=() + [ "$DEBUG_BUILD" = 1 ] && debug=(DEBUG=1) + + if ! make -C "$PERF_DIR" O="$BUILD_DIR" CC=clang \ + "${debug[@]+"${debug[@]}"}" \ + EXTRA_CFLAGS="-fprofile-instr-generate -fcoverage-mapping -gdwarf-$DWARF_VERSION" \ + -j"$JOBS" > "$BUILD_LOG" 2>&1; then + tail -20 "$BUILD_LOG" >&2 + die "build failed, full log in $BUILD_LOG" + fi + tail -1 "$BUILD_LOG" + write_state +} + +# +# Run the selected tests with the instrumented perf in PATH: the shell +# tests, the workloads and 'perf test' itself all come from there. +# +run_tests() +{ + if [ ${#TESTS[@]} -eq 0 ]; then + say "Running the complete perf test suite" + else + say "Running perf tests: ${TESTS[*]}" + fi + rm -rf "$COV_DIR/raw" + mkdir -p "$COV_DIR/raw" + + set +e + # The shell tests only get discovered from the kernel tree root. + (cd "$PERF_DIR/../.." && + PATH="$BUILD_DIR:$PATH" \ + LLVM_PROFILE_FILE="$COV_DIR/raw/perf-%p-%m.profraw" \ + "$BUILD_DIR/perf" test -v \ + "${TESTS[@]+"${TESTS[@]}"}") 2>&1 | tee "$TEST_LOG" + test_status=${PIPESTATUS[0]} + set -e + + # 'perf test' exits 0 even when tests fail, only the summary tells. + if [ "$test_status" = 0 ] && grep -qE '^Failed tests +: +[1-9]' "$TEST_LOG"; then + test_status=1 + fi + + grep -E '^(Passed|Skipped|Failed) .*:' "$TEST_LOG" || true + [ "$test_status" = 0 ] || + warn "some tests failed or were skipped, coverage is still collected" + if [ "$test_status" != 0 ] && grep -q 'Access to performance monitoring' "$TEST_LOG"; then + warn "perf_event_paranoid=$(cat /proc/sys/kernel/perf_event_paranoid 2>/dev/null)" \ + "denied events to tests recording with callchains;" \ + "run as root or lower it for the full suite" + fi +} + +# +# Merge the per-process profiles and render the text report. +# +merge_profiles() +{ + local count size unusable=0 + count=$(find "$COV_DIR/raw" -name '*.profraw' | wc -l) + size=$(du -sh "$COV_DIR/raw" 2>/dev/null | cut -f1) + [ "$count" -gt 0 ] || + die "no .profraw files, was perf rebuilt with -fprofile-instr-generate?" + + say "Merging $count profile files ($size)" + if ! "$LLVM_PROFDATA" merge -sparse "$COV_DIR"/raw/*.profraw \ + -o "$PROFDATA" 2> "$COV_DIR/merge.log"; then + # A file left half written, by a perf killed at exit or by a full + # tmpfs, makes llvm-profdata drop the whole merge and with it the + # run's measurement: retry keeping what can be read and say how + # much was lost. --failure-mode=warn exists since LLVM 14, the + # oldest llvm-profdata the test images ship. + warn "some profile files are unusable, merging the rest" + "$LLVM_PROFDATA" merge -sparse --failure-mode=warn \ + "$COV_DIR"/raw/*.profraw -o "$PROFDATA" \ + 2> "$COV_DIR/merge.log" || + die "llvm-profdata merge failed, see $COV_DIR/merge.log" + unusable=$(grep -c '^warning:' "$COV_DIR/merge.log" || true) + [ -n "$unusable" ] || unusable=0 + [ "$unusable" -lt "$count" ] || + die "no usable profile among $count, see $COV_DIR/merge.log" + warn "merged $((count - unusable)) profile files, ignored $unusable unusable" + fi +} + +llvm_cov_objects() +{ + echo "$BUILD_DIR/perf" + local so + for so in perf.so libperf.so libperf-jvmti.so; do + [ -f "$BUILD_DIR/$so" ] && echo "--object=$BUILD_DIR/$so" + done + # The Python extension is named after the interpreter ABI and lives + # in its own directory. + for so in "$BUILD_DIR"/python/perf*.so; do + [ -f "$so" ] && echo "--object=$so" + done +} + +render_report() +{ + local objects + mapfile -t objects < <(llvm_cov_objects) + + say "Generating report" + "$LLVM_COV" report "${objects[@]}" --instr-profile="$PROFDATA" > "$REPORT" +} + +# +# Line coverage of a full report for one file: "lines missed cover fn%". +# Report paths are relative to the build directory, so match the requested +# path as a suffix: a build outside the source tree gives +# ".../tools/perf/..." and one inside gives "tools/perf/...". +# +# -f takes the path as it shows in the report, or a tools/perf/ relative one. +focus_path() +{ + case "$1" in + *tools/perf/*) printf '%s\n' "${1#*tools/perf/}" ;; + *) printf '%s\n' "$1" ;; + esac +} + +file_cov() +{ + awk -v f="$2" ' + function perf_relpath(p) { + if (p ~ /(^|\/)tools\/perf\//) + sub(/^.*tools\/perf\//, "", p) + return p + } + { + g = f + sub(/^.*tools\/perf\//, "", g) + rel = perf_relpath($1) + if (rel == f || rel == g || $1 == f) { + print $8, $9, $10, $7 + exit + } + }' "$1" +} + +total_cov() +{ + awk '$1 == "TOTAL" { print $8, $9, $10 }' "$1" +} + +total_delta() +{ + local prev=$1 cur=$2 old cur_cov + [ -f "$prev" ] || return 0 + old=$(total_cov "$prev" | awk '{print $3}') + cur_cov=$(total_cov "$cur" | awk '{print $3}') + [ -n "$old" ] && [ -n "$cur_cov" ] || return 0 + awk -v cur="$cur_cov" -v old="$old" 'BEGIN { + sub(/%$/, "", cur); sub(/%$/, "", old) + printf "%+.2f%%", cur - old }' +} + +report_delta() +{ + local prev=$1 cur=$2 label + local cur_cov old_cov delta + label=$(focus_path "$3") + cur_cov=$(file_cov "$cur" "$label" | awk '{print $3}') + [ -n "$cur_cov" ] || return 0 + + if [ -f "$prev" ]; then + old_cov=$(file_cov "$prev" "$label" | awk '{print $3}') + if [ -n "$old_cov" ]; then + delta=$(awk -v cur="$cur_cov" -v old="$old_cov" 'BEGIN { + sub(/%$/, "", cur); sub(/%$/, "", old) + printf "%+.2f%%", cur - old }') + else + delta="(new)" + fi + else + old_cov="-" + delta="-" + fi + printf " %-36s %8s %8s %9s\n" "$label" "${old_cov:--}" "$cur_cov" "$delta" +} + +# +# With no -f, show where the uncovered lines are: the files with the most +# missed lines are where new tests should try to get more coverage. +# +print_worst_files() +{ + [ "$TOP" -gt 0 ] || return 0 + local rows missed lines cover fn path + rows=$(awk -v min="$MIN_LINES" ' + $1 ~ /(^|\/)tools\/perf\// && $8 >= min && $9 > 0 { + path = $1 + sub(/^.*tools\/perf\//, "", path) + printf "%s %s %s %s %s\n", $9, $8, $10, $7, path + }' "$REPORT" | sort -k1,1nr -k2,2nr | sed -n "1,${TOP}p") + [ -n "$rows" ] || return 0 + + printf " largest uncovered files (>= %s lines)\n" "$MIN_LINES" + printf " %-44s %8s %8s %8s %8s\n" \ + "file" "lines" "cover" "missed" "fn-cover" + while read -r missed lines cover fn path; do + printf " %-44s %8s %8s %8s %8s\n" \ + "$path" "$lines" "$cover" "$missed" "$fn" + done <<< "$rows" +} + +print_summary() +{ + local lines missed cover delta file cov + read -r lines missed cover < <(total_cov "$REPORT") + printf "\n\033[1mTotal:\033[0m %s of %s lines covered (%s)" \ + "$((lines - missed))" "$lines" "$cover" + if [ "$USE_BASELINE" = 1 ] && [ -f "$BASELINE" ] && + ! cmp -s "$BASELINE" "$REPORT"; then + delta=$(total_delta "$BASELINE" "$REPORT") + [ -n "$delta" ] && printf " (previous %s, %s)" \ + "$(total_cov "$BASELINE" | awk '{print $3}')" "$delta" + fi + printf "\n\n" + + if [ ${#FOCUS[@]} -eq 0 ]; then + print_worst_files + return + fi + + printf " \033[1m%-36s\033[0m %8s %6s %6s %8s\n" \ + "file" "lines" "cover" "missed" "fn-cover" + for file in "${FOCUS[@]}"; do + file=$(focus_path "$file") + cov=$(file_cov "$REPORT" "$file") + if [ -z "$cov" ]; then + printf " %-36s %8s\n" "$file" "not covered" + continue + fi + printf " %-36s %8s %6s %6s %8s\n" "$file" \ + "$(echo "$cov" | awk '{print $1}')" \ + "$(echo "$cov" | awk '{print $3}')" \ + "$(echo "$cov" | awk '{print $2}')" \ + "$(echo "$cov" | awk '{print $4}')" + done + + if [ "$USE_BASELINE" = 1 ] && [ -f "$BASELINE" ] && + ! cmp -s "$BASELINE" "$REPORT"; then + printf "\n per file delta since %s\n" "$BASELINE" + printf " %-36s %8s %8s %9s\n" "file" "previous" "current" "delta" + for file in "${FOCUS[@]}"; do + report_delta "$BASELINE" "$REPORT" "$file" + done + fi +} + +# +# --table: render the report of the last run as an ASCII table, like +# pahole's coverage_table.py. Without -f the rows are the TOP files with +# the most uncovered lines, sorted by line coverage; --top 0 lists every +# file with at least MIN_LINES lines and uncovered lines. The total is +# the whole report, which also counts the libraries perf links with. +# +bar() +{ + local pct=$1 width=$2 filled i out= + filled=$(awk -v p="$pct" -v w="$width" 'BEGIN { printf "%d", p * w / 100 }') + for ((i = 0; i < filled; i++)); do out+=█; done + for ((i = filled; i < width; i++)); do out+=░; done + printf '%s' "$out" +} + +center() +{ + local left=$(( ($2 - ${#1}) / 2 )) + printf '%*s%s%*s' "$left" '' "$1" "$(( $2 - ${#1} - left ))" '' +} + +dashes() +{ + local s + printf -v s '%*s' "$1" '' + printf '%s' "${s// /─}" +} + +table_row() +{ + awk -v f="$2" ' + function perf_relpath(p) { + if (p ~ /(^|\/)tools\/perf\//) + sub(/^.*tools\/perf\//, "", p) + return p + } + { + g = f + sub(/^.*tools\/perf\//, "", g) + path = perf_relpath($1) + if (path == f || path == g || $1 == f) { + printf "%s\t%s\t%d\t%d\t%.2f\t%s\t%d\t%s\n", + path, $8, $8 - $9, $9, $10, $5, $5 - $6, $7 + exit + } + }' "$1" +} + +table_rows() +{ + if [ ${#FOCUS[@]} -gt 0 ]; then + local file + for file in "${FOCUS[@]}"; do + table_row "$REPORT" "$file" + done | sort -t$'\t' -k5,5gr + return + fi + + awk -v min="$MIN_LINES" ' + $1 ~ /(^|\/)tools\/perf\// && $8 >= min && $9 > 0 { + path = $1 + sub(/^.*tools\/perf\//, "", path) + printf "%s\t%s\t%d\t%d\t%.2f\t%s\t%d\t%s\n", + path, $8, $8 - $9, $9, $10, $5, $5 - $6, $7 + }' "$REPORT" | + sort -t$'\t' -k4,4nr | + if [ "$TOP" -gt 0 ]; then sed -n "1,${TOP}p"; else cat; fi | + sort -t$'\t' -k5,5gr +} + +print_table() +{ + local rows total path lines hit missed pct funcs funcs_hit func_cov + local row_fmt w d_file d29 d13 d10 d8 + + rows=$(table_rows) + [ -n "$rows" ] || + die "no coverage data in $REPORT for the requested files" + total=$(awk '$1 == "TOTAL" { + printf "%s\t%d\t%d\t%.2f\t%s\t%d\t%s\n", + $8, $8 - $9, $9, $10, $5, $5 - $6, $7 + }' "$REPORT") + + w=4 + while IFS=$'\t' read -r path _; do + if [ ${#path} -gt "$w" ]; then + w=${#path} + fi + done <<< "$rows" + + d8=$(dashes 8) + d10=$(dashes 10) + d13=$(dashes 13) + d29=$(dashes 29) + d_file=$(dashes $((w + 2))) + + printf '┌%s┬%s┬%s┬%s┬%s┬%s┬%s┐\n' \ + "$d_file" "$d8" "$d8" "$d8" "$d29" "$d13" "$d10" + printf '│ %-*s │ %6s │ %6s │ %6s │ %s │ %11s │ %8s │\n' \ + "$w" "File" "Lines" "Hit" "Miss" \ + "$(center 'Line Coverage' 27)" "Funcs" "FuncCov" + printf '├%s┼%s┼%s┼%s┼%s┼%s┼%s┤\n' \ + "$d_file" "$d8" "$d8" "$d8" "$d29" "$d13" "$d10" + + row_fmt=$(printf '│ %%-%ds │ %%6s │ %%6s │ %%6s │ %%s %%5.1f%%%%' "$w") + row_fmt+=' │ %5s/%-5s │ %8s │\n' + + while IFS=$'\t' read -r path lines hit missed pct funcs funcs_hit func_cov; do + printf "$row_fmt" "$path" "$lines" "$hit" "$missed" \ + "$(bar "$pct" 20)" "$pct" "$funcs_hit" "$funcs" "$func_cov" + done <<< "$rows" + + printf '├%s┼%s┼%s┼%s┼%s┼%s┼%s┤\n' \ + "$d_file" "$d8" "$d8" "$d8" "$d29" "$d13" "$d10" + while IFS=$'\t' read -r lines hit missed pct funcs funcs_hit func_cov; do + printf "$row_fmt" "TOTAL" "$lines" "$hit" "$missed" \ + "$(bar "$pct" 20)" "$pct" "$funcs_hit" "$funcs" "$func_cov" + done <<< "$total" + printf '└%s┴%s┴%s┴%s┴%s┴%s┴%s┘\n' \ + "$d_file" "$d8" "$d8" "$d8" "$d29" "$d13" "$d10" +} + +print_uncovered() +{ + [ "$UNCOVERED" -gt 0 ] || return 0 + [ ${#FOCUS[@]} -gt 0 ] || return 0 + local objects file out lines header=0 + mapfile -t objects < <(llvm_cov_objects) + + for file in "${FOCUS[@]}"; do + file=$(focus_path "$file") + # No report row means the file was not built: print_summary() + # already said so, llvm-cov show would dump the whole profile. + [ -n "$(file_cov "$REPORT" "$file")" ] || continue + out=$("$LLVM_COV" show "${objects[@]}" --instr-profile="$PROFDATA" \ + "$PERF_DIR/$file" 2>/dev/null | + awk -F'|' '$2 ~ /^[[:space:]]*0[[:space:]]*$/ { + gsub(/[[:space:]]/, "", $1); printf "%s ", $1 }') + lines=$(echo "$out" | wc -w) + [ "$lines" -gt 0 ] || continue + if [ "$header" = 0 ]; then + printf "\n uncovered lines (first %s per file)\n" "$UNCOVERED" + header=1 + fi + printf " %-24s %4s lines: %s\n" "$file" "$lines" \ + "$(echo "$out" | cut -d' ' -f1-"$UNCOVERED")" + done +} + +save_html() +{ + [ "$HTML" = 1 ] || return 0 + local objects + mapfile -t objects < <(llvm_cov_objects) + say "Writing HTML report to $COV_DIR/html" + "$LLVM_COV" show "${objects[@]}" --instr-profile="$PROFDATA" \ + --format=html --output-dir="$COV_DIR/html" \ + --show-line-counts-or-regions --show-branches=count +} + +check_stale() +{ + [ "$REPORT_ONLY" = 1 ] || return 0 + if [ ! -f "$PROFDATA" ]; then + [ "$TABLE" = 1 ] && return 0 + die "no $PROFDATA, run without --report-only first" + fi + local newer + newer=$(find "$PERF_DIR" -name '*.c' -newer "$PROFDATA" -print -quit 2>/dev/null) + if [ -n "$newer" ]; then + warn "profiles are older than source changes (e.g. $newer), results may be stale" + fi +} + +update_baseline() +{ + [ "$USE_BASELINE" = 1 ] || return 0 + [ "$REPORT_ONLY" = 1 ] && return 0 + [ -f "$BASELINE" ] && mv "$BASELINE" "$BASELINE.prev" + cp "$REPORT" "$BASELINE" + printf "\nBaseline saved: %s\n" "$BASELINE" +} + +test_status=0 + +if [ "$TABLE" = 1 ]; then + [ -f "$REPORT" ] || die "no $REPORT, run coverage first" + check_stale + print_table + exit 0 +fi + +if [ "$PREP" = 1 ]; then + build_perf + write_state + say "Coverage build ready: $BUILD_DIR/perf" + printf "As root: %s/scripts/coverage.sh\n" "$PERF_DIR" + exit 0 +fi + +if [ "$REPORT_ONLY" = 0 ]; then + build_perf + run_tests + merge_profiles +else + check_coverage_build + check_stale +fi + +[ -f "$PROFDATA" ] || die "no $PROFDATA, run without --report-only first" + +render_report +print_summary +print_uncovered +save_html +update_baseline + +printf "\nReport: %s\n" "$REPORT" +[ "$HTML" = 1 ] && printf "HTML: %s\n" "$COV_DIR/html/index.html" + +exit "$test_status" -- 2.56.0