From: Ian Rogers <irogers@google.com>
To: Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
Arnaldo Carvalho de Melo <acme@kernel.org>,
Namhyung Kim <namhyung@kernel.org>, Jiri Olsa <jolsa@kernel.org>,
Ian Rogers <irogers@google.com>,
Adrian Hunter <adrian.hunter@intel.com>,
James Clark <james.clark@linaro.org>,
Thomas Falcon <thomas.falcon@intel.com>,
Leo Yan <leo.yan@arm.com>,
Thomas Richter <tmricht@linux.ibm.com>,
linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org
Subject: [PATCH v1 11/12] perf tests: Speed up lock contention analysis shell test
Date: Mon, 15 Jun 2026 18:27:43 -0700 [thread overview]
Message-ID: <20260616012744.4049193-12-irogers@google.com> (raw)
In-Reply-To: <20260616012744.4049193-1-irogers@google.com>
The lock contention analysis test suite (`lock_contention.sh`) performs
a series of 13 separate profiling checks to verify various aggregation
and filtering parameters of `perf lock contention`. Each of these checks
runs the `perf bench sched messaging` messaging benchmark as its workload.
By default, `sched messaging` runs 10 groups of 40 processes (400 processes
total) generating substantial task scheduling, context switching, and IPC
message passing. When traced system-wide for lock events, the tracing overhead
(handling millions of lock acquisitions and releases) slows execution down
significantly, causing the test suite to take over 80 seconds.
Optimize this by introducing a scaled-down messaging benchmark workload:
`perf bench sched messaging -g 1 -p`. Running 1 group (40 processes) takes
only 0.01 seconds natively (instead of 0.08 seconds), drastically reduces the
sheer volume of lock acquire/release trace events, and reduces CPU context
switching during tracing while still generating sufficient lock events to fully
exercise the BPF/record filters.
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/tests/shell/lock_contention.sh | 30 +++++++++++++----------
1 file changed, 17 insertions(+), 13 deletions(-)
diff --git a/tools/perf/tests/shell/lock_contention.sh b/tools/perf/tests/shell/lock_contention.sh
index 52e8b9db9fbd..f589d387f6b3 100755
--- a/tools/perf/tests/shell/lock_contention.sh
+++ b/tools/perf/tests/shell/lock_contention.sh
@@ -9,6 +9,10 @@ perfdata=$(mktemp /tmp/__perf_test.perf.data.XXXXX)
result=$(mktemp /tmp/__perf_test.result.XXXXX)
errout=$(mktemp /tmp/__perf_test.errout.XXXXX)
+# Workload to generate lock contention.
+# Using 1 group (-g 1) keeps runtime low while generating sufficient lock events.
+msg_workload="perf bench sched messaging -g 1 -p"
+
cleanup() {
rm -f ${perfdata}
rm -f ${result}
@@ -50,7 +54,7 @@ check() {
test_record()
{
echo "Testing perf lock record and perf lock contention"
- perf lock record -o ${perfdata} -- perf bench sched messaging -p > /dev/null 2>&1
+ perf lock record -o ${perfdata} -- ${msg_workload} > /dev/null 2>&1
# the output goes to the stderr and we expect only 1 output (-E 1)
perf lock contention -i ${perfdata} -E 1 -q 2> ${result}
if [ "$(cat "${result}" | wc -l)" != "1" ]; then
@@ -70,7 +74,7 @@ test_bpf()
fi
# the perf lock contention output goes to the stderr
- perf lock con -a -b -E 1 -q -- perf bench sched messaging -p > /dev/null 2> ${result}
+ perf lock con -a -b -E 1 -q -- ${msg_workload} > /dev/null 2> ${result}
if [ "$(cat "${result}" | wc -l)" != "1" ]; then
echo "[Fail] BPF result count is not 1:" "$(cat "${result}" | wc -l)"
err=1
@@ -81,7 +85,7 @@ test_bpf()
test_record_concurrent()
{
echo "Testing perf lock record and perf lock contention at the same time"
- perf lock record -o- -- perf bench sched messaging -p 2> ${errout} | \
+ perf lock record -o- -- ${msg_workload} 2> ${errout} | \
perf lock contention -i- -E 1 -q 2> ${result}
if [ "$(cat "${result}" | wc -l)" != "1" ]; then
echo "[Fail] Recorded result count is not 1:" "$(cat "${result}" | wc -l)"
@@ -107,7 +111,7 @@ test_aggr_task()
fi
# the perf lock contention output goes to the stderr
- perf lock con -a -b -t -E 1 -q -- perf bench sched messaging -p > /dev/null 2> ${result}
+ perf lock con -a -b -t -E 1 -q -- ${msg_workload} > /dev/null 2> ${result}
if [ "$(cat "${result}" | wc -l)" != "1" ]; then
echo "[Fail] BPF result count is not 1:" "$(cat "${result}" | wc -l)"
err=1
@@ -130,7 +134,7 @@ test_aggr_addr()
fi
# the perf lock contention output goes to the stderr
- perf lock con -a -b -l -E 1 -q -- perf bench sched messaging -p > /dev/null 2> ${result}
+ perf lock con -a -b -l -E 1 -q -- ${msg_workload} > /dev/null 2> ${result}
if [ "$(cat "${result}" | wc -l)" != "1" ]; then
echo "[Fail] BPF result count is not 1:" "$(cat "${result}" | wc -l)"
err=1
@@ -148,7 +152,7 @@ test_aggr_cgroup()
fi
# the perf lock contention output goes to the stderr
- perf lock con -a -b --lock-cgroup -E 1 -q -- perf bench sched messaging -p > /dev/null 2> ${result}
+ perf lock con -a -b --lock-cgroup -E 1 -q -- ${msg_workload} > /dev/null 2> ${result}
if [ "$(cat "${result}" | wc -l)" != "1" ]; then
echo "[Fail] BPF result count is not 1:" "$(cat "${result}" | wc -l)"
err=1
@@ -170,7 +174,7 @@ test_type_filter()
return
fi
- perf lock con -a -b -Y spinlock -q -- perf bench sched messaging -p > /dev/null 2> ${result}
+ perf lock con -a -b -Y spinlock -q -- ${msg_workload} > /dev/null 2> ${result}
if [ "$(grep -c -v spinlock "${result}")" != "0" ]; then
echo "[Fail] BPF result should not have non-spinlocks:" "$(cat "${result}")"
err=1
@@ -202,7 +206,7 @@ test_lock_filter()
return
fi
- perf lock con -a -b -L tasklist_lock -q -- perf bench sched messaging -p > /dev/null 2> ${result}
+ perf lock con -a -b -L tasklist_lock -q -- ${msg_workload} > /dev/null 2> ${result}
if [ "$(grep -c -v "${test_lock_filter_type}" "${result}")" != "0" ]; then
echo "[Fail] BPF result should not have non-${test_lock_filter_type} locks:" "$(cat "${result}")"
err=1
@@ -241,7 +245,7 @@ test_stack_filter()
return
fi
- perf lock con -a -b -S unix_stream -E 1 -q -- perf bench sched messaging -p > /dev/null 2> ${result}
+ perf lock con -a -b -S unix_stream -E 1 -q -- ${msg_workload} > /dev/null 2> ${result}
if [ "$(cat "${result}" | wc -l)" != "1" ]; then
echo "[Fail] BPF result should have a lock from unix_stream:" "$(cat "${result}")"
err=1
@@ -269,7 +273,7 @@ test_aggr_task_stack_filter()
return
fi
- perf lock con -a -b -t -S unix_stream -E 1 -q -- perf bench sched messaging -p > /dev/null 2> ${result}
+ perf lock con -a -b -t -S unix_stream -E 1 -q -- ${msg_workload} > /dev/null 2> ${result}
if [ "$(cat "${result}" | wc -l)" != "1" ]; then
echo "[Fail] BPF result should have a task from unix_stream:" "$(cat "${result}")"
err=1
@@ -285,7 +289,7 @@ test_cgroup_filter()
return
fi
- perf lock con -a -b --lock-cgroup -E 1 -F wait_total -q -- perf bench sched messaging -p > /dev/null 2> ${result}
+ perf lock con -a -b --lock-cgroup -E 1 -F wait_total -q -- ${msg_workload} > /dev/null 2> ${result}
if [ "$(cat "${result}" | wc -l)" != "1" ]; then
echo "[Fail] BPF result should have a cgroup result:" "$(cat "${result}")"
err=1
@@ -293,7 +297,7 @@ test_cgroup_filter()
fi
cgroup=$(cat "${result}" | awk '{ print $3 }')
- perf lock con -a -b --lock-cgroup -E 1 -G "${cgroup}" -q -- perf bench sched messaging -p > /dev/null 2> ${result}
+ perf lock con -a -b --lock-cgroup -E 1 -G "${cgroup}" -q -- ${msg_workload} > /dev/null 2> ${result}
if [ "$(cat "${result}" | wc -l)" != "1" ]; then
echo "[Fail] BPF result should have a result with cgroup filter:" "$(cat "${cgroup}")"
err=1
@@ -328,7 +332,7 @@ test_csv_output()
fi
# the perf lock contention output goes to the stderr
- perf lock con -a -b -E 1 -x , --output ${result} -- perf bench sched messaging -p > /dev/null 2>&1
+ perf lock con -a -b -E 1 -x , --output ${result} -- ${msg_workload} > /dev/null 2>&1
output=$(grep -v "^#" ${result} | tr -d -c , | wc -c)
if [ "${header}" != "${output}" ]; then
echo "[Fail] BPF result does not match the number of commas: ${header} != ${output}"
--
2.54.0.1136.gdb2ca164c4-goog
next prev parent reply other threads:[~2026-06-16 1:28 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-06-16 1:27 [PATCH v1 00/12] perf tests: Enhancements, speedups, and flakiness fixes Ian Rogers
2026-06-16 1:27 ` [PATCH v1 01/12] perf parse-events: Restrict core PMU bypass to --cputype option Ian Rogers
2026-06-16 1:44 ` sashiko-bot
2026-06-16 1:27 ` [PATCH v1 02/12] perf test: Truncate test description to fit terminal width Ian Rogers
2026-06-16 1:38 ` sashiko-bot
2026-06-16 1:27 ` [PATCH v1 03/12] perf tests workloads: Support sub-second durations in noploop and thloop Ian Rogers
2026-06-16 1:35 ` sashiko-bot
2026-06-16 1:27 ` [PATCH v1 04/12] perf tests: Add robust record retry helper and use subsecond workloads Ian Rogers
2026-06-16 1:38 ` sashiko-bot
2026-06-16 1:27 ` [PATCH v1 05/12] perf tests: Skip metrics validation if system-wide recording lacks permission Ian Rogers
2026-06-16 1:41 ` sashiko-bot
2026-06-16 1:27 ` [PATCH v1 06/12] perf tests: Fix Python JIT dump profiling test failure Ian Rogers
2026-06-16 1:39 ` sashiko-bot
2026-06-16 1:27 ` [PATCH v1 07/12] perf tests: Fix flakiness in trace record and replay test Ian Rogers
2026-06-16 1:42 ` sashiko-bot
2026-06-16 1:27 ` [PATCH v1 08/12] perf tests: Fix flakiness in BPF counters test on hybrid systems Ian Rogers
2026-06-16 1:35 ` sashiko-bot
2026-06-16 1:27 ` [PATCH v1 09/12] perf tests: Fix flakiness in branch stack sampling tests Ian Rogers
2026-06-16 1:27 ` [PATCH v1 10/12] perf tests: Speed up off-cpu profiling tests Ian Rogers
2026-06-16 1:41 ` sashiko-bot
2026-06-16 1:27 ` Ian Rogers [this message]
2026-06-16 1:27 ` [PATCH v1 12/12] perf tests: Speed up metrics checking shell tests Ian Rogers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260616012744.4049193-12-irogers@google.com \
--to=irogers@google.com \
--cc=acme@kernel.org \
--cc=adrian.hunter@intel.com \
--cc=james.clark@linaro.org \
--cc=jolsa@kernel.org \
--cc=leo.yan@arm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=namhyung@kernel.org \
--cc=peterz@infradead.org \
--cc=thomas.falcon@intel.com \
--cc=tmricht@linux.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox