Linux Perf Users
 help / color / mirror / Atom feed
From: Namhyung Kim <namhyung@kernel.org>
To: Ian Rogers <irogers@google.com>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>,
	adrian.hunter@intel.com, james.clark@linaro.org,
	jolsa@kernel.org, leo.yan@arm.com, linux-kernel@vger.kernel.org,
	linux-perf-users@vger.kernel.org, mingo@redhat.com,
	peterz@infradead.org, thomas.falcon@intel.com,
	tmricht@linux.ibm.com
Subject: Re: [PATCH v3 04/13] perf tests: Add robust record retry helper and use subsecond workloads
Date: Mon, 22 Jun 2026 23:06:12 -0700	[thread overview]
Message-ID: <ajoiVNOuDTzGxVDM@z2> (raw)
In-Reply-To: <CAP-5=fVwvX01JpjJarBBYUnKwWmQ-1WKPBAj3ZDL9cwTCHvqpQ@mail.gmail.com>

On Mon, Jun 22, 2026 at 04:59:15PM -0700, Ian Rogers wrote:
> On Thu, Jun 18, 2026 at 6:25 AM Arnaldo Carvalho de Melo
> <acme@kernel.org> wrote:
> >
> > On Wed, Jun 17, 2026 at 03:37:06PM -0700, Namhyung Kim wrote:
> > > On Tue, Jun 16, 2026 at 09:48:09AM -0700, Ian Rogers wrote:
> > > > Introduce `perf_record_with_retry` and `perf_record_cleanup` in a shared
> > > > library `tests/shell/lib/perf_record.sh` to prevent record test failures
> > > > caused by transient recording or workload delays.
> > > >
> > > > Update `record.sh`, `record_lbr.sh`, `pipe_test.sh`, `kvm.sh`, and
> > > > `stat_all_pfm.sh` to use this robust record retry logic. These tests now
> > > > start with very short durations (e.g. 0.01 seconds) and scale up if the
> > > > initial recording failed to capture samples, significantly improving test
> > > > execution speed on success while remaining resilient to slow systems.
> > > >
> > > > Assisted-by: Antigravity:gemini-3.1-pro
> > > > Signed-off-by: Ian Rogers <irogers@google.com>
> > > > ---
> > > [SNIP]
> > > > @@ -60,22 +71,29 @@ test_kvm_stat() {
> > > >  test_kvm_record_report() {
> > > >     echo "Testing perf kvm record/report"
> > > >
> > > > -   echo "Recording kvm profile for pid ${qemu_pid}..."
> > > > -   # Use --host to avoid needing guest symbols/mounts for this simple test
> > > > -   # We just want to verify the command runs and produces data
> > > > -   # We run in background and kill it because 'perf kvm record' appends options
> > > > -   # after the command, which breaks 'sleep' (e.g. it gets '-e cycles').
> > > > -   perf kvm --host record -p "${qemu_pid}" -o "${perfdata}" &
> > > > -   rec_pid=$!
> > > > -   sleep 1
> > > > -   kill -INT "${rec_pid}"
> > > > -   wait "${rec_pid}" || true
> > > > +   local duration
> > > > +   local success=false
> > > > +   for duration in 1 2 4 8; do
> > > > +           echo "Recording kvm profile for pid ${qemu_pid} (duration ${duration}s)..."
> > > > +           rm -f "${perfdata}" "${perfdata}".old
> > > > +
> > > > +           perf kvm --host record -p "${qemu_pid}" -o "${perfdata}" &
> > > > +           local rec_pid=$!
> > > > +           sleep ${duration}
> > > > +           kill -INT "${rec_pid}"
> > > > +           wait "${rec_pid}" || true
> > >
> > > Can this be just like below?
> > >
> > >               perf kvm --host record -p "${qemu_pid}" -o "${perfdata}" sleep ${duration}
> >
> > Right, looks equivalent and simpler,
> >
> > I was making notes to address the flakiness that is making me use the
> > end summary of entries that fail (great feature) to run them in
> > isolation to check if they work that way, which most do :-\
> 
> Here is an explanation from antigravity (sorry I have some typing
> issues atm), but it comes down to how the command line is built up:
> 
>   Normally, we could just write:
> 
>     perf kvm --host record -p "qemuₚid" - o"{perfdata}" sleep
> ${duration}
> 
>   However, on Intel x86 architectures, if no  -e  option is supplied
> to  perf kvm record , it calls architecture-specific setup logic (
> __kvm_add_default_arch_event_x86() ) which automatically appends  "-e"
>  and  "cycles"  to the end of the  argv  array.
>   Because  perf record  option parsing stops at the first non-option
> (which is  "sleep" ), any arguments appended after the workload name
> are forwarded directly to the workload command. This turns the
> executed command into:
> 
>     sleep 1 -e cycles
> 
>   Which causes  sleep  to crash with:
>    sleep: invalid option -- 'e'
> 
> I'll revise this in v4 to be clearer, but we should probably also fix
> the option parsing issue.

Thanks for the explanation!  I found it's in the deleted comment.

Let me think about the option parsing issue.

Thanks,
Namhyung


  reply	other threads:[~2026-06-23  6:06 UTC|newest]

Thread overview: 79+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-06-16  1:27 [PATCH v1 00/12] perf tests: Enhancements, speedups, and flakiness fixes Ian Rogers
2026-06-16  1:27 ` [PATCH v1 01/12] perf parse-events: Restrict core PMU bypass to --cputype option Ian Rogers
2026-06-16  1:44   ` sashiko-bot
2026-06-16  1:27 ` [PATCH v1 02/12] perf test: Truncate test description to fit terminal width Ian Rogers
2026-06-16  1:38   ` sashiko-bot
2026-06-16  1:27 ` [PATCH v1 03/12] perf tests workloads: Support sub-second durations in noploop and thloop Ian Rogers
2026-06-16  1:35   ` sashiko-bot
2026-06-16  1:27 ` [PATCH v1 04/12] perf tests: Add robust record retry helper and use subsecond workloads Ian Rogers
2026-06-16  1:38   ` sashiko-bot
2026-06-16  1:27 ` [PATCH v1 05/12] perf tests: Skip metrics validation if system-wide recording lacks permission Ian Rogers
2026-06-16  1:41   ` sashiko-bot
2026-06-16  1:27 ` [PATCH v1 06/12] perf tests: Fix Python JIT dump profiling test failure Ian Rogers
2026-06-16  1:39   ` sashiko-bot
2026-06-16  1:27 ` [PATCH v1 07/12] perf tests: Fix flakiness in trace record and replay test Ian Rogers
2026-06-16  1:42   ` sashiko-bot
2026-06-16  1:27 ` [PATCH v1 08/12] perf tests: Fix flakiness in BPF counters test on hybrid systems Ian Rogers
2026-06-16  1:35   ` sashiko-bot
2026-06-16  1:27 ` [PATCH v1 09/12] perf tests: Fix flakiness in branch stack sampling tests Ian Rogers
2026-06-16  1:27 ` [PATCH v1 10/12] perf tests: Speed up off-cpu profiling tests Ian Rogers
2026-06-16  1:41   ` sashiko-bot
2026-06-16  1:27 ` [PATCH v1 11/12] perf tests: Speed up lock contention analysis shell test Ian Rogers
2026-06-16  1:27 ` [PATCH v1 12/12] perf tests: Speed up metrics checking shell tests Ian Rogers
2026-06-16  6:13 ` [PATCH v2 00/12] perf tests: Enhance robustness, speed up execution, and fix flakiness Ian Rogers
2026-06-16  6:13   ` [PATCH v2 01/12] perf parse-events: Restrict core PMU bypass to --cputype option Ian Rogers
2026-06-16  6:31     ` sashiko-bot
2026-06-16 15:14     ` Arnaldo Carvalho de Melo
2026-06-16 15:17     ` Arnaldo Carvalho de Melo
2026-06-16  6:13   ` [PATCH v2 02/12] perf test: Truncate test description to fit terminal width Ian Rogers
2026-06-16  6:24     ` sashiko-bot
2026-06-16 15:25     ` Arnaldo Carvalho de Melo
2026-06-16  6:13   ` [PATCH v2 03/12] perf tests workloads: Support sub-second durations in noploop and thloop Ian Rogers
2026-06-16  6:22     ` sashiko-bot
2026-06-16  6:13   ` [PATCH v2 04/12] perf tests: Add robust record retry helper and use subsecond workloads Ian Rogers
2026-06-16  6:27     ` sashiko-bot
2026-06-16  6:13   ` [PATCH v2 05/12] perf tests: Skip metrics validation if system-wide recording lacks permission Ian Rogers
2026-06-16  6:13   ` [PATCH v2 06/12] perf tests: Fix Python JIT dump profiling test failure Ian Rogers
2026-06-16  6:27     ` sashiko-bot
2026-06-16  6:13   ` [PATCH v2 07/12] perf tests: Fix flakiness in trace record and replay test Ian Rogers
2026-06-16  6:27     ` sashiko-bot
2026-06-16  6:14   ` [PATCH v2 08/12] perf tests: Fix flakiness in BPF counters test on hybrid systems Ian Rogers
2026-06-16  6:14   ` [PATCH v2 09/12] perf tests: Fix flakiness in branch stack sampling tests Ian Rogers
2026-06-16  6:14   ` [PATCH v2 10/12] perf tests: Speed up off-cpu profiling tests Ian Rogers
2026-06-16  6:25     ` sashiko-bot
2026-06-16  6:14   ` [PATCH v2 11/12] perf tests: Speed up lock contention analysis shell test Ian Rogers
2026-06-16  6:14   ` [PATCH v2 12/12] perf tests: Speed up metrics checking shell tests Ian Rogers
2026-06-16 16:48   ` [PATCH v3 00/13] perf tests: Robustness and performance improvements Ian Rogers
2026-06-16 16:48     ` [PATCH v3 01/13] perf parse-events: Restrict core PMU bypass to --cputype option Ian Rogers
2026-06-16 16:48     ` [PATCH v3 02/13] perf test: Truncate test description to fit terminal width Ian Rogers
2026-06-17 22:33       ` Namhyung Kim
2026-06-23  0:51         ` Ian Rogers
2026-06-16 16:48     ` [PATCH v3 03/13] perf tests workloads: Support sub-second durations in noploop and thloop Ian Rogers
2026-06-16 16:48     ` [PATCH v3 04/13] perf tests: Add robust record retry helper and use subsecond workloads Ian Rogers
2026-06-17 22:37       ` Namhyung Kim
2026-06-18 13:24         ` Arnaldo Carvalho de Melo
2026-06-22 23:59           ` Ian Rogers
2026-06-23  6:06             ` Namhyung Kim [this message]
2026-06-16 16:48     ` [PATCH v3 05/13] perf tests: Skip metrics validation if system-wide recording lacks permission Ian Rogers
2026-06-16 16:48     ` [PATCH v3 06/13] perf tests: Fix Python JIT dump profiling test failure Ian Rogers
2026-06-16 16:48     ` [PATCH v3 07/13] perf tests: Fix flakiness in trace record and replay test Ian Rogers
2026-06-16 16:48     ` [PATCH v3 08/13] perf tests: Fix flakiness in BPF counters test on hybrid systems Ian Rogers
2026-06-16 16:48     ` [PATCH v3 09/13] perf tests: Fix flakiness in branch stack sampling tests Ian Rogers
2026-06-16 16:48     ` [PATCH v3 10/13] perf tests: Speed up off-cpu profiling tests Ian Rogers
2026-06-16 16:48     ` [PATCH v3 11/13] perf tests: Speed up lock contention analysis shell test Ian Rogers
2026-06-16 16:48     ` [PATCH v3 12/13] perf tests: Speed up metrics checking shell tests Ian Rogers
2026-06-16 16:48     ` [PATCH v3 13/13] perf tests: Include error output for skipped tests in JUnit XML Ian Rogers
2026-06-23  1:27     ` [PATCH v4 00/13] perf tests: Robustness and performance improvements Ian Rogers
2026-06-23  1:27       ` [PATCH v4 01/13] perf parse-events: Restrict core PMU bypass to --cputype option Ian Rogers
2026-06-23  1:27       ` [PATCH v4 02/13] perf test: Truncate test description to fit terminal width Ian Rogers
2026-06-23  1:27       ` [PATCH v4 03/13] perf tests workloads: Support sub-second durations in noploop and thloop Ian Rogers
2026-06-23  1:27       ` [PATCH v4 04/13] perf tests: Add robust record retry helper and use subsecond workloads Ian Rogers
2026-06-23  1:27       ` [PATCH v4 05/13] perf tests: Skip metrics validation if system-wide recording lacks permission Ian Rogers
2026-06-23  1:27       ` [PATCH v4 06/13] perf tests: Fix Python JIT dump profiling test failure Ian Rogers
2026-06-23  1:27       ` [PATCH v4 07/13] perf tests: Fix flakiness in trace record and replay test Ian Rogers
2026-06-23  1:27       ` [PATCH v4 08/13] perf tests: Fix flakiness in BPF counters test on hybrid systems Ian Rogers
2026-06-23  1:27       ` [PATCH v4 09/13] perf tests: Fix flakiness in branch stack sampling tests Ian Rogers
2026-06-23  1:27       ` [PATCH v4 10/13] perf tests: Speed up off-cpu profiling tests Ian Rogers
2026-06-23  1:27       ` [PATCH v4 11/13] perf tests: Speed up lock contention analysis shell test Ian Rogers
2026-06-23  1:27       ` [PATCH v4 12/13] perf tests: Speed up metrics checking shell tests Ian Rogers
2026-06-23  1:27       ` [PATCH v4 13/13] perf tests: Include error output for skipped tests in JUnit XML Ian Rogers

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ajoiVNOuDTzGxVDM@z2 \
    --to=namhyung@kernel.org \
    --cc=acme@kernel.org \
    --cc=adrian.hunter@intel.com \
    --cc=irogers@google.com \
    --cc=james.clark@linaro.org \
    --cc=jolsa@kernel.org \
    --cc=leo.yan@arm.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=mingo@redhat.com \
    --cc=peterz@infradead.org \
    --cc=thomas.falcon@intel.com \
    --cc=tmricht@linux.ibm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox