From: Arnaldo Carvalho de Melo <acme@kernel.org>
To: Leo Yan <leo.yan@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org>,
Ian Rogers <irogers@google.com>,
Adrian Hunter <adrian.hunter@intel.com>,
"Liang, Kan" <kan.liang@linux.intel.com>,
James Clark <james.clark@linaro.org>,
linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH] perf tests switch-tracking: Fix timestamp comparison
Date: Fri, 23 May 2025 14:12:58 -0300 [thread overview]
Message-ID: <aDCsmjb7Fex1ccOW@x1> (raw)
In-Reply-To: <20250523081036.GB2566836@e132581.arm.com>
On Fri, May 23, 2025 at 09:10:36AM +0100, Leo Yan wrote:
> On Thu, May 22, 2025 at 10:57:41PM -0300, Arnaldo Carvalho de Melo wrote:
> > On Thu, May 22, 2025 at 10:55:46PM -0300, Arnaldo Carvalho de Melo wrote:
> > > On Mon, Mar 31, 2025 at 06:27:59PM +0100, Leo Yan wrote:
> > > > The test might fail on the Arm64 platform with the error:
> > > > perf test -vvv "Track with sched_switch"
> > > > Missing sched_switch events
> > > > The issue is caused by incorrect handling of timestamp comparisons. The
> > > > comparison result, a signed 64-bit value, was being directly cast to an
> > > > int, leading to incorrect sorting for sched events.
> > > > Fix this by explicitly returning 0, 1, or -1 based on whether the result
> > > > is zero, positive, or negative.
> > > > Fixes: d44bc5582972 ("perf tests: Add a test for tracking with sched_switch")
> > > > Signed-off-by: Leo Yan <leo.yan@arm.com>
> > > How can I reproduce this?
> > > Testing on a rpi5, 64-bit debian, this test passes:
>
> Sorry that I did not give precise info for reproducing the failure.
> The case does not fail everytime, usually I can trigger the failure
> after run 20 ~ 30 times:
>
> # while true; do perf test "Track with sched_switch"; done
> 106: Track with sched_switch : Ok
> 106: Track with sched_switch : Ok
> 106: Track with sched_switch : Ok
> 106: Track with sched_switch : Ok
> 106: Track with sched_switch : Ok
> 106: Track with sched_switch : Ok
> 106: Track with sched_switch : Ok
> 106: Track with sched_switch : Ok
> 106: Track with sched_switch : Ok
> 106: Track with sched_switch : Ok
> 106: Track with sched_switch : Ok
> 106: Track with sched_switch : Ok
> 106: Track with sched_switch : Ok
> 106: Track with sched_switch : Ok
> 106: Track with sched_switch : FAILED!
> 106: Track with sched_switch : Ok
> 106: Track with sched_switch : Ok
> 106: Track with sched_switch : Ok
> 106: Track with sched_switch : Ok
> 106: Track with sched_switch : Ok
> 106: Track with sched_switch : Ok
> 106: Track with sched_switch : Ok
> 106: Track with sched_switch : Ok
> 106: Track with sched_switch : FAILED!
> 106: Track with sched_switch : Ok
> 106: Track with sched_switch : Ok
> I used cross compiler to build Perf tool on my host machine and tested on
> Debian / Juno board. Generally, I think this issue is not very specific
> to GCC versions. As both internal CI and my local env can reproduce the
> issue.
> Please let me know if need any more info. Thanks!
> ---8<---
> My Host Build compiler:
> # aarch64-linux-gnu-gcc --version
> aarch64-linux-gnu-gcc (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0
> Juno Board:
> # lsb_release -a
> No LSB modules are available.
> Distributor ID: Debian
> Description: Debian GNU/Linux 12 (bookworm)
> Release: 12
> Codename: bookworm
Thanks for the extra info, I'll add it to the commit log message, and
perhaps we could make this test exclusive and use stress-ng to generate
some background noise in the form of a good number of processes, see:
root@x1:~# stress-ng --switch $(($(nproc) * 2)) --timeout 30s & for a in $(seq 50) ; do perf test switch ; done
[1] 1773322
stress-ng: info: [1773322] setting to a 30 secs run per stressor
77: Track with sched_switch : Running (1 active)
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : Running (1 active)
stress-ng: info: [1773322] skipped: 0
stress-ng: info: [1773322] passed: 24: switch (24)
stress-ng: info: [1773322] failed: 0
stress-ng: info: [1773322] metrics untrustworthy: 0
77: Track with sched_switch : FAILED!
[1]+ Done stress-ng --switch $(($(nproc) * 2)) --timeout 30s
77: Track with sched_switch : Ok
77: Track with sched_switch : Ok
77: Track with sched_switch : Ok
77: Track with sched_switch : Ok
77: Track with sched_switch : FAILED!
77: Track with sched_switch : Ok
77: Track with sched_switch : Ok
77: Track with sched_switch : FAILED!
77: Track with sched_switch : FAILED!
77: Track with sched_switch : Ok
root@x1:~#
Now with your patch it also fails, so its for another reason:
--- start ---
test child forked, pid 1777071
Using CPUID GenuineIntel-6-BA-3
mmap size 528384B
45221 events recorded
Missing comm events
---- end(-1) ----
113: Track with sched_switch : FAILED!
Lots of short lived processes makes it fail as well :-\
Oh well...
I was just trying to improve this test case so that we would show it
failing before your patch and passing after it, but I ran out of time
:-\
Your patch is correct, so I'll probably just add your comments and go
with it.
- Arnaldo
next prev parent reply other threads:[~2025-05-23 17:13 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-03-31 17:27 [PATCH] perf tests switch-tracking: Fix timestamp comparison Leo Yan
2025-03-31 20:18 ` Ian Rogers
2025-04-01 9:14 ` Leo Yan
2025-04-01 19:54 ` Ian Rogers
2025-04-02 9:05 ` Leo Yan
2025-05-16 15:31 ` Leo Yan
2025-05-20 1:52 ` Namhyung Kim
2025-05-23 1:49 ` Arnaldo Carvalho de Melo
2025-05-23 1:51 ` Arnaldo Carvalho de Melo
2025-05-23 1:55 ` Arnaldo Carvalho de Melo
2025-05-23 1:57 ` Arnaldo Carvalho de Melo
2025-05-23 8:10 ` Leo Yan
2025-05-23 16:52 ` Ian Rogers
2025-05-23 17:12 ` Arnaldo Carvalho de Melo [this message]
2025-05-27 9:49 ` Leo Yan
2025-05-28 13:08 ` Arnaldo Carvalho de Melo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=aDCsmjb7Fex1ccOW@x1 \
--to=acme@kernel.org \
--cc=adrian.hunter@intel.com \
--cc=irogers@google.com \
--cc=james.clark@linaro.org \
--cc=kan.liang@linux.intel.com \
--cc=leo.yan@arm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=namhyung@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox