linux-perf-users.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: vmolnaro@redhat.com
To: linux-perf-users@vger.kernel.org, acme@kernel.org, acme@redhat.com
Cc: mpetlan@redhat.com
Subject: [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors
Date: Tue, 12 Sep 2023 14:19:26 +0200	[thread overview]
Message-ID: <20230912121926.12111-2-vmolnaro@redhat.com> (raw)
In-Reply-To: <20230912121926.12111-1-vmolnaro@redhat.com>

From: Veronika Molnarova <vmolnaro@redhat.com>

The test was failing in specific scenarios due to imperfection of FP
arithmetics. The `bc` command wasn't correctly rounding the result
of division causing the failure.

Replace the `bc` with `awk` which should work with more decimal
places and add a threshold to catch any possible rounding errors.
The acceptable rounding error is set to 0.01 when the test
passes with a warning message.
---
 tools/perf/tests/shell/stat+shadow_stat.sh | 30 +++++++++++++++++-----
 1 file changed, 24 insertions(+), 6 deletions(-)

diff --git a/tools/perf/tests/shell/stat+shadow_stat.sh b/tools/perf/tests/shell/stat+shadow_stat.sh
index 0e9cba84e75..73c990fe5ee 100755
--- a/tools/perf/tests/shell/stat+shadow_stat.sh
+++ b/tools/perf/tests/shell/stat+shadow_stat.sh
@@ -4,6 +4,8 @@
 
 set -e
 
+TRESHOLD=0.015
+
 # skip if system-wide mode is forbidden
 perf stat -a true > /dev/null 2>&1 || exit 2
 
@@ -33,10 +35,18 @@ test_global_aggr()
 		fi
 
 		# use printf for rounding and a leading zero
-		res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"`
+		res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
 		if [ "$ipc" != "$res" ]; then
-			echo "IPC is different: $res != $ipc  ($num / $cyc)"
-			exit 1
+			# check the difference from the real result for FP imperfections
+			diff=`echo $ipc $res $TRESHOLD | \
+			awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
+
+			if [ $diff -eq 1 ]; then
+				echo "IPC is different: $res != $ipc  ($num / $cyc)"
+				exit 1
+			fi
+
+			echo "Warning: Difference of IPC is under the treshold"
 		fi
 	done
 }
@@ -67,10 +77,18 @@ test_no_aggr()
 		fi
 
 		# use printf for rounding and a leading zero
-		res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"`
+		res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
 		if [ "$ipc" != "$res" ]; then
-			echo "IPC is different for $cpu: $res != $ipc  ($num / $cyc)"
-			exit 1
+			# check difference from the real result for FP imperfections
+			diff=`echo $ipc $res $TRESHOLD | \
+			awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
+
+			if [ $diff -eq 1 ]; then
+				echo "IPC is different: $res != $ipc  ($num / $cyc)"
+				exit 1
+			fi
+
+			echo "Warning: Difference of IPC is under the treshold"
 		fi
 	done
 }
-- 
2.41.0


  reply	other threads:[~2023-09-12 12:20 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-09-12 12:19 [PATCH 1/2] perf test lock_contention.sh: Skip test if the number of CPUs is low vmolnaro
2023-09-12 12:19 ` vmolnaro [this message]
2023-09-12 12:22   ` [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors Michael Petlan
2023-09-12 21:01   ` Arnaldo Carvalho de Melo
2023-09-12 12:21 ` [PATCH 1/2] perf test lock_contention.sh: Skip test if the number of CPUs is low Michael Petlan
2023-09-12 20:57 ` Arnaldo Carvalho de Melo
  -- strict thread matches above, loose matches on Subject: below --
2023-08-21  9:00 vmolnaro
2023-08-21  9:00 ` [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors vmolnaro
2023-08-23  9:02   ` Michael Petlan
2023-09-06 16:51   ` Arnaldo Carvalho de Melo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230912121926.12111-2-vmolnaro@redhat.com \
    --to=vmolnaro@redhat.com \
    --cc=acme@kernel.org \
    --cc=acme@redhat.com \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=mpetlan@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).