* [PATCH 1/2] perf test lock_contention.sh: Skip test if the number of CPUs is low
@ 2023-09-12 12:19 vmolnaro
2023-09-12 12:19 ` [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors vmolnaro
` (2 more replies)
0 siblings, 3 replies; 9+ messages in thread
From: vmolnaro @ 2023-09-12 12:19 UTC (permalink / raw)
To: linux-perf-users, acme, acme; +Cc: mpetlan
From: Veronika Molnarova <vmolnaro@redhat.com>
Machines with less then 4 CPUs weren't consistently triggering lock
events required for the test.
Skip the test on those machines. The limit of 4 CPUs is set as it
generates around 100 lock events for a test.
---
tools/perf/tests/shell/lock_contention.sh | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/tools/perf/tests/shell/lock_contention.sh b/tools/perf/tests/shell/lock_contention.sh
index f2cc187b618..1d3598ff6a0 100755
--- a/tools/perf/tests/shell/lock_contention.sh
+++ b/tools/perf/tests/shell/lock_contention.sh
@@ -31,6 +31,12 @@ check() {
echo "[Skip] No lock contention tracepoints"
err=2
exit
+ fi
+
+ if [ `nproc` -lt 4 ]; then
+ echo "[Skip] Low number of CPUs (`nproc`), lock event cannot be triggered certainly"
+ err=2
+ exit
fi
}
--
2.41.0
^ permalink raw reply related [flat|nested] 9+ messages in thread* [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors
2023-09-12 12:19 [PATCH 1/2] perf test lock_contention.sh: Skip test if the number of CPUs is low vmolnaro
@ 2023-09-12 12:19 ` vmolnaro
2023-09-12 12:22 ` Michael Petlan
2023-09-12 21:01 ` Arnaldo Carvalho de Melo
2023-09-12 12:21 ` [PATCH 1/2] perf test lock_contention.sh: Skip test if the number of CPUs is low Michael Petlan
2023-09-12 20:57 ` Arnaldo Carvalho de Melo
2 siblings, 2 replies; 9+ messages in thread
From: vmolnaro @ 2023-09-12 12:19 UTC (permalink / raw)
To: linux-perf-users, acme, acme; +Cc: mpetlan
From: Veronika Molnarova <vmolnaro@redhat.com>
The test was failing in specific scenarios due to imperfection of FP
arithmetics. The `bc` command wasn't correctly rounding the result
of division causing the failure.
Replace the `bc` with `awk` which should work with more decimal
places and add a threshold to catch any possible rounding errors.
The acceptable rounding error is set to 0.01 when the test
passes with a warning message.
---
tools/perf/tests/shell/stat+shadow_stat.sh | 30 +++++++++++++++++-----
1 file changed, 24 insertions(+), 6 deletions(-)
diff --git a/tools/perf/tests/shell/stat+shadow_stat.sh b/tools/perf/tests/shell/stat+shadow_stat.sh
index 0e9cba84e75..73c990fe5ee 100755
--- a/tools/perf/tests/shell/stat+shadow_stat.sh
+++ b/tools/perf/tests/shell/stat+shadow_stat.sh
@@ -4,6 +4,8 @@
set -e
+TRESHOLD=0.015
+
# skip if system-wide mode is forbidden
perf stat -a true > /dev/null 2>&1 || exit 2
@@ -33,10 +35,18 @@ test_global_aggr()
fi
# use printf for rounding and a leading zero
- res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"`
+ res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
if [ "$ipc" != "$res" ]; then
- echo "IPC is different: $res != $ipc ($num / $cyc)"
- exit 1
+ # check the difference from the real result for FP imperfections
+ diff=`echo $ipc $res $TRESHOLD | \
+ awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
+
+ if [ $diff -eq 1 ]; then
+ echo "IPC is different: $res != $ipc ($num / $cyc)"
+ exit 1
+ fi
+
+ echo "Warning: Difference of IPC is under the treshold"
fi
done
}
@@ -67,10 +77,18 @@ test_no_aggr()
fi
# use printf for rounding and a leading zero
- res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"`
+ res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
if [ "$ipc" != "$res" ]; then
- echo "IPC is different for $cpu: $res != $ipc ($num / $cyc)"
- exit 1
+ # check difference from the real result for FP imperfections
+ diff=`echo $ipc $res $TRESHOLD | \
+ awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
+
+ if [ $diff -eq 1 ]; then
+ echo "IPC is different: $res != $ipc ($num / $cyc)"
+ exit 1
+ fi
+
+ echo "Warning: Difference of IPC is under the treshold"
fi
done
}
--
2.41.0
^ permalink raw reply related [flat|nested] 9+ messages in thread* Re: [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors
2023-09-12 12:19 ` [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors vmolnaro
@ 2023-09-12 12:22 ` Michael Petlan
2023-09-12 21:01 ` Arnaldo Carvalho de Melo
1 sibling, 0 replies; 9+ messages in thread
From: Michael Petlan @ 2023-09-12 12:22 UTC (permalink / raw)
To: vmolnaro; +Cc: linux-perf-users, acme, acme
On Tue, 12 Sep 2023, vmolnaro@redhat.com wrote:
> From: Veronika Molnarova <vmolnaro@redhat.com>
>
> The test was failing in specific scenarios due to imperfection of FP
> arithmetics. The `bc` command wasn't correctly rounding the result
> of division causing the failure.
>
> Replace the `bc` with `awk` which should work with more decimal
> places and add a threshold to catch any possible rounding errors.
> The acceptable rounding error is set to 0.01 when the test
> passes with a warning message.
Acked-by: Michael Petlan <mpetlan@redhat.com>
> ---
> tools/perf/tests/shell/stat+shadow_stat.sh | 30 +++++++++++++++++-----
> 1 file changed, 24 insertions(+), 6 deletions(-)
>
> diff --git a/tools/perf/tests/shell/stat+shadow_stat.sh b/tools/perf/tests/shell/stat+shadow_stat.sh
> index 0e9cba84e75..73c990fe5ee 100755
> --- a/tools/perf/tests/shell/stat+shadow_stat.sh
> +++ b/tools/perf/tests/shell/stat+shadow_stat.sh
> @@ -4,6 +4,8 @@
>
> set -e
>
> +TRESHOLD=0.015
> +
> # skip if system-wide mode is forbidden
> perf stat -a true > /dev/null 2>&1 || exit 2
>
> @@ -33,10 +35,18 @@ test_global_aggr()
> fi
>
> # use printf for rounding and a leading zero
> - res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"`
> + res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
> if [ "$ipc" != "$res" ]; then
> - echo "IPC is different: $res != $ipc ($num / $cyc)"
> - exit 1
> + # check the difference from the real result for FP imperfections
> + diff=`echo $ipc $res $TRESHOLD | \
> + awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
> +
> + if [ $diff -eq 1 ]; then
> + echo "IPC is different: $res != $ipc ($num / $cyc)"
> + exit 1
> + fi
> +
> + echo "Warning: Difference of IPC is under the treshold"
> fi
> done
> }
> @@ -67,10 +77,18 @@ test_no_aggr()
> fi
>
> # use printf for rounding and a leading zero
> - res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"`
> + res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
> if [ "$ipc" != "$res" ]; then
> - echo "IPC is different for $cpu: $res != $ipc ($num / $cyc)"
> - exit 1
> + # check difference from the real result for FP imperfections
> + diff=`echo $ipc $res $TRESHOLD | \
> + awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
> +
> + if [ $diff -eq 1 ]; then
> + echo "IPC is different: $res != $ipc ($num / $cyc)"
> + exit 1
> + fi
> +
> + echo "Warning: Difference of IPC is under the treshold"
> fi
> done
> }
> --
> 2.41.0
>
>
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors
2023-09-12 12:19 ` [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors vmolnaro
2023-09-12 12:22 ` Michael Petlan
@ 2023-09-12 21:01 ` Arnaldo Carvalho de Melo
1 sibling, 0 replies; 9+ messages in thread
From: Arnaldo Carvalho de Melo @ 2023-09-12 21:01 UTC (permalink / raw)
To: vmolnaro; +Cc: linux-perf-users, acme, mpetlan
Em Tue, Sep 12, 2023 at 02:19:26PM +0200, vmolnaro@redhat.com escreveu:
> From: Veronika Molnarova <vmolnaro@redhat.com>
>
> The test was failing in specific scenarios due to imperfection of FP
> arithmetics. The `bc` command wasn't correctly rounding the result
> of division causing the failure.
>
> Replace the `bc` with `awk` which should work with more decimal
> places and add a threshold to catch any possible rounding errors.
> The acceptable rounding error is set to 0.01 when the test
> passes with a warning message.
> ---
> tools/perf/tests/shell/stat+shadow_stat.sh | 30 +++++++++++++++++-----
> 1 file changed, 24 insertions(+), 6 deletions(-)
>
> diff --git a/tools/perf/tests/shell/stat+shadow_stat.sh b/tools/perf/tests/shell/stat+shadow_stat.sh
> index 0e9cba84e75..73c990fe5ee 100755
> --- a/tools/perf/tests/shell/stat+shadow_stat.sh
> +++ b/tools/perf/tests/shell/stat+shadow_stat.sh
> @@ -4,6 +4,8 @@
>
> set -e
>
> +TRESHOLD=0.015
You got it right on the patch description, I fixed it here as:
⬢[acme@toolbox perf-tools-next]$ git diff
diff --git a/tools/perf/tests/shell/stat+shadow_stat.sh b/tools/perf/tests/shell/stat+shadow_stat.sh
index 626d669158d2d089..a7e4a8f2ad186458 100755
--- a/tools/perf/tests/shell/stat+shadow_stat.sh
+++ b/tools/perf/tests/shell/stat+shadow_stat.sh
@@ -4,7 +4,7 @@
set -e
-TRESHOLD=0.015
+THRESHOLD=0.015
# skip if system-wide mode is forbidden
perf stat -a true > /dev/null 2>&1 || exit 2
@@ -38,7 +38,7 @@ test_global_aggr()
res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
if [ "$ipc" != "$res" ]; then
# check the difference from the real result for FP imperfections
- diff=`echo $ipc $res $TRESHOLD | \
+ diff=`echo $ipc $res $THRESHOLD | \
awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
if [ $diff -eq 1 ]; then
@@ -80,7 +80,7 @@ test_no_aggr()
res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
if [ "$ipc" != "$res" ]; then
# check difference from the real result for FP imperfections
- diff=`echo $ipc $res $TRESHOLD | \
+ diff=`echo $ipc $res $THRESHOLD | \
awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
if [ $diff -eq 1 ]; then
⬢[acme@toolbox perf-tools-next]$
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH 1/2] perf test lock_contention.sh: Skip test if the number of CPUs is low
2023-09-12 12:19 [PATCH 1/2] perf test lock_contention.sh: Skip test if the number of CPUs is low vmolnaro
2023-09-12 12:19 ` [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors vmolnaro
@ 2023-09-12 12:21 ` Michael Petlan
2023-09-12 20:57 ` Arnaldo Carvalho de Melo
2 siblings, 0 replies; 9+ messages in thread
From: Michael Petlan @ 2023-09-12 12:21 UTC (permalink / raw)
To: vmolnaro; +Cc: linux-perf-users, acme, acme
On Tue, 12 Sep 2023, vmolnaro@redhat.com wrote:
> From: Veronika Molnarova <vmolnaro@redhat.com>
>
> Machines with less then 4 CPUs weren't consistently triggering lock
> events required for the test.
>
> Skip the test on those machines. The limit of 4 CPUs is set as it
> generates around 100 lock events for a test.
Acked-by: Michael Petlan <mpetlan@redhat.com>
> ---
> tools/perf/tests/shell/lock_contention.sh | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/tools/perf/tests/shell/lock_contention.sh b/tools/perf/tests/shell/lock_contention.sh
> index f2cc187b618..1d3598ff6a0 100755
> --- a/tools/perf/tests/shell/lock_contention.sh
> +++ b/tools/perf/tests/shell/lock_contention.sh
> @@ -31,6 +31,12 @@ check() {
> echo "[Skip] No lock contention tracepoints"
> err=2
> exit
> + fi
> +
> + if [ `nproc` -lt 4 ]; then
> + echo "[Skip] Low number of CPUs (`nproc`), lock event cannot be triggered certainly"
> + err=2
> + exit
> fi
> }
>
> --
> 2.41.0
>
>
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [PATCH 1/2] perf test lock_contention.sh: Skip test if the number of CPUs is low
2023-09-12 12:19 [PATCH 1/2] perf test lock_contention.sh: Skip test if the number of CPUs is low vmolnaro
2023-09-12 12:19 ` [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors vmolnaro
2023-09-12 12:21 ` [PATCH 1/2] perf test lock_contention.sh: Skip test if the number of CPUs is low Michael Petlan
@ 2023-09-12 20:57 ` Arnaldo Carvalho de Melo
2 siblings, 0 replies; 9+ messages in thread
From: Arnaldo Carvalho de Melo @ 2023-09-12 20:57 UTC (permalink / raw)
To: vmolnaro; +Cc: linux-perf-users, acme, mpetlan
Em Tue, Sep 12, 2023 at 02:19:25PM +0200, vmolnaro@redhat.com escreveu:
> From: Veronika Molnarova <vmolnaro@redhat.com>
>
> Machines with less then 4 CPUs weren't consistently triggering lock
> events required for the test.
>
> Skip the test on those machines. The limit of 4 CPUs is set as it
> generates around 100 lock events for a test.
Better, but still some issues, fixing manually.
⬢[acme@toolbox perf-tools-next]$ b4 am -ctsl --cc-trailers 20230912121926.12111-1-vmolnaro@redhat.com
Grabbing thread from lore.kernel.org/all/20230912121926.12111-1-vmolnaro%40redhat.com/t.mbox.gz
Checking for newer revisions
Grabbing search results from lore.kernel.org
Analyzing 4 messages in the thread
Checking attestation on all messages, may take a moment...
---
✓ [PATCH 1/2] perf test lock_contention.sh: Skip test if the number of CPUs is low
+ Acked-by: Michael Petlan <mpetlan@redhat.com> (✓ DKIM/redhat.com)
+ Link: https://lore.kernel.org/r/20230912121926.12111-1-vmolnaro@redhat.com
+ Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
✓ [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors
+ Acked-by: Michael Petlan <mpetlan@redhat.com> (✓ DKIM/redhat.com)
+ Link: https://lore.kernel.org/r/20230912121926.12111-2-vmolnaro@redhat.com
+ Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
✓ Signed: DKIM/redhat.com
---
Total patches: 2
---
Link: https://lore.kernel.org/r/20230912121926.12111-1-vmolnaro@redhat.com
Base: not specified
git am ./20230912_vmolnaro_perf_test_lock_contention_sh_skip_test_if_the_number_of_cpus_is_low.mbx
⬢[acme@toolbox perf-tools-next]$ git diff
⬢[acme@toolbox perf-tools-next]$ git am ./20230912_vmolnaro_perf_test_lock_contention_sh_skip_test_if_the_number_of_cpus_is_low.mbx
Applying: perf test lock_contention.sh: Skip test if the number of CPUs is low
.git/rebase-apply/patch:13: trailing whitespace.
fi
warning: 1 line adds whitespace errors.
Applying: perf test stat+shadow_stat.sh: Add threshold for rounding errors
⬢[acme@toolbox perf-tools-next]$
> ---
> tools/perf/tests/shell/lock_contention.sh | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/tools/perf/tests/shell/lock_contention.sh b/tools/perf/tests/shell/lock_contention.sh
> index f2cc187b618..1d3598ff6a0 100755
> --- a/tools/perf/tests/shell/lock_contention.sh
> +++ b/tools/perf/tests/shell/lock_contention.sh
> @@ -31,6 +31,12 @@ check() {
> echo "[Skip] No lock contention tracepoints"
> err=2
> exit
> + fi
> +
> + if [ `nproc` -lt 4 ]; then
> + echo "[Skip] Low number of CPUs (`nproc`), lock event cannot be triggered certainly"
> + err=2
> + exit
> fi
> }
>
> --
> 2.41.0
>
--
- Arnaldo
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH 1/2] perf test lock_contention.sh: Skip test if the number of CPUs is low
@ 2023-08-21 9:00 vmolnaro
2023-08-21 9:00 ` [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors vmolnaro
0 siblings, 1 reply; 9+ messages in thread
From: vmolnaro @ 2023-08-21 9:00 UTC (permalink / raw)
To: linux-perf-users, acme, acme; +Cc: mpetlan
From: notValord <v.m.veverka@gmail.com>
Machines with less then 4 CPUs weren't consistently triggering lock
events required for the test.
Skip the test on those machines. The limit of 4 CPUs is set as it
generates around 100 lock events for a test.
---
tools/perf/tests/shell/lock_contention.sh | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/tools/perf/tests/shell/lock_contention.sh b/tools/perf/tests/shell/lock_contention.sh
index f2cc187b618..1d3598ff6a0 100755
--- a/tools/perf/tests/shell/lock_contention.sh
+++ b/tools/perf/tests/shell/lock_contention.sh
@@ -31,6 +31,12 @@ check() {
echo "[Skip] No lock contention tracepoints"
err=2
exit
+ fi
+
+ if [ `nproc` -lt 4 ]; then
+ echo "[Skip] Low number of CPUs (`nproc`), lock event cannot be triggered certainly"
+ err=2
+ exit
fi
}
--
2.41.0
^ permalink raw reply related [flat|nested] 9+ messages in thread* [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors
2023-08-21 9:00 vmolnaro
@ 2023-08-21 9:00 ` vmolnaro
2023-08-23 9:02 ` Michael Petlan
2023-09-06 16:51 ` Arnaldo Carvalho de Melo
0 siblings, 2 replies; 9+ messages in thread
From: vmolnaro @ 2023-08-21 9:00 UTC (permalink / raw)
To: linux-perf-users, acme, acme; +Cc: mpetlan
From: notValord <v.m.veverka@gmail.com>
The test was failing in specific scenarios due to imperfection of FP
arithmetics. The `bc` command wasn't correctly rounding the result
of division causing the failure.
Replace the `bc` with `awk` which should work with more decimal
places and add a threshold to catch any possible rounding errors.
The acceptable rounding error is set to 0.01 when the test
passes with a warning message.
---
tools/perf/tests/shell/stat+shadow_stat.sh | 30 +++++++++++++++++-----
1 file changed, 24 insertions(+), 6 deletions(-)
diff --git a/tools/perf/tests/shell/stat+shadow_stat.sh b/tools/perf/tests/shell/stat+shadow_stat.sh
index 0e9cba84e75..73c990fe5ee 100755
--- a/tools/perf/tests/shell/stat+shadow_stat.sh
+++ b/tools/perf/tests/shell/stat+shadow_stat.sh
@@ -4,6 +4,8 @@
set -e
+TRESHOLD=0.015
+
# skip if system-wide mode is forbidden
perf stat -a true > /dev/null 2>&1 || exit 2
@@ -33,10 +35,18 @@ test_global_aggr()
fi
# use printf for rounding and a leading zero
- res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"`
+ res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
if [ "$ipc" != "$res" ]; then
- echo "IPC is different: $res != $ipc ($num / $cyc)"
- exit 1
+ # check the difference from the real result for FP imperfections
+ diff=`echo $ipc $res $TRESHOLD | \
+ awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
+
+ if [ $diff -eq 1 ]; then
+ echo "IPC is different: $res != $ipc ($num / $cyc)"
+ exit 1
+ fi
+
+ echo "Warning: Difference of IPC is under the treshold"
fi
done
}
@@ -67,10 +77,18 @@ test_no_aggr()
fi
# use printf for rounding and a leading zero
- res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"`
+ res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
if [ "$ipc" != "$res" ]; then
- echo "IPC is different for $cpu: $res != $ipc ($num / $cyc)"
- exit 1
+ # check difference from the real result for FP imperfections
+ diff=`echo $ipc $res $TRESHOLD | \
+ awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
+
+ if [ $diff -eq 1 ]; then
+ echo "IPC is different: $res != $ipc ($num / $cyc)"
+ exit 1
+ fi
+
+ echo "Warning: Difference of IPC is under the treshold"
fi
done
}
--
2.41.0
^ permalink raw reply related [flat|nested] 9+ messages in thread* Re: [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors
2023-08-21 9:00 ` [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors vmolnaro
@ 2023-08-23 9:02 ` Michael Petlan
2023-09-06 16:51 ` Arnaldo Carvalho de Melo
1 sibling, 0 replies; 9+ messages in thread
From: Michael Petlan @ 2023-08-23 9:02 UTC (permalink / raw)
To: vmolnaro; +Cc: linux-perf-users, acme, acme
On Mon, 21 Aug 2023, vmolnaro@redhat.com wrote:
> From: notValord <v.m.veverka@gmail.com>
>
> The test was failing in specific scenarios due to imperfection of FP
> arithmetics. The `bc` command wasn't correctly rounding the result
> of division causing the failure.
>
> Replace the `bc` with `awk` which should work with more decimal
> places and add a threshold to catch any possible rounding errors.
> The acceptable rounding error is set to 0.01 when the test
> passes with a warning message.
Acked-by: Michael Petlan <mpetlan@redhat.com>
> ---
> tools/perf/tests/shell/stat+shadow_stat.sh | 30 +++++++++++++++++-----
> 1 file changed, 24 insertions(+), 6 deletions(-)
>
> diff --git a/tools/perf/tests/shell/stat+shadow_stat.sh b/tools/perf/tests/shell/stat+shadow_stat.sh
> index 0e9cba84e75..73c990fe5ee 100755
> --- a/tools/perf/tests/shell/stat+shadow_stat.sh
> +++ b/tools/perf/tests/shell/stat+shadow_stat.sh
> @@ -4,6 +4,8 @@
>
> set -e
>
> +TRESHOLD=0.015
> +
> # skip if system-wide mode is forbidden
> perf stat -a true > /dev/null 2>&1 || exit 2
>
> @@ -33,10 +35,18 @@ test_global_aggr()
> fi
>
> # use printf for rounding and a leading zero
> - res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"`
> + res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
> if [ "$ipc" != "$res" ]; then
> - echo "IPC is different: $res != $ipc ($num / $cyc)"
> - exit 1
> + # check the difference from the real result for FP imperfections
> + diff=`echo $ipc $res $TRESHOLD | \
> + awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
> +
> + if [ $diff -eq 1 ]; then
> + echo "IPC is different: $res != $ipc ($num / $cyc)"
> + exit 1
> + fi
> +
> + echo "Warning: Difference of IPC is under the treshold"
> fi
> done
> }
> @@ -67,10 +77,18 @@ test_no_aggr()
> fi
>
> # use printf for rounding and a leading zero
> - res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"`
> + res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
> if [ "$ipc" != "$res" ]; then
> - echo "IPC is different for $cpu: $res != $ipc ($num / $cyc)"
> - exit 1
> + # check difference from the real result for FP imperfections
> + diff=`echo $ipc $res $TRESHOLD | \
> + awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
> +
> + if [ $diff -eq 1 ]; then
> + echo "IPC is different: $res != $ipc ($num / $cyc)"
> + exit 1
> + fi
> +
> + echo "Warning: Difference of IPC is under the treshold"
> fi
> done
> }
> --
> 2.41.0
>
>
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors
2023-08-21 9:00 ` [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors vmolnaro
2023-08-23 9:02 ` Michael Petlan
@ 2023-09-06 16:51 ` Arnaldo Carvalho de Melo
1 sibling, 0 replies; 9+ messages in thread
From: Arnaldo Carvalho de Melo @ 2023-09-06 16:51 UTC (permalink / raw)
To: vmolnaro; +Cc: linux-perf-users, acme, mpetlan
Em Mon, Aug 21, 2023 at 11:00:53AM +0200, vmolnaro@redhat.com escreveu:
> From: notValord <v.m.veverka@gmail.com>
Hi, is the above the same person as vmolnaro@redhat.com? Can we please
have the full name and use that as the From and also the Signed-off-by?
Thanks,
- Arnaldo
> The test was failing in specific scenarios due to imperfection of FP
> arithmetics. The `bc` command wasn't correctly rounding the result
> of division causing the failure.
>
> Replace the `bc` with `awk` which should work with more decimal
> places and add a threshold to catch any possible rounding errors.
> The acceptable rounding error is set to 0.01 when the test
> passes with a warning message.
> ---
> tools/perf/tests/shell/stat+shadow_stat.sh | 30 +++++++++++++++++-----
> 1 file changed, 24 insertions(+), 6 deletions(-)
>
> diff --git a/tools/perf/tests/shell/stat+shadow_stat.sh b/tools/perf/tests/shell/stat+shadow_stat.sh
> index 0e9cba84e75..73c990fe5ee 100755
> --- a/tools/perf/tests/shell/stat+shadow_stat.sh
> +++ b/tools/perf/tests/shell/stat+shadow_stat.sh
> @@ -4,6 +4,8 @@
>
> set -e
>
> +TRESHOLD=0.015
> +
> # skip if system-wide mode is forbidden
> perf stat -a true > /dev/null 2>&1 || exit 2
>
> @@ -33,10 +35,18 @@ test_global_aggr()
> fi
>
> # use printf for rounding and a leading zero
> - res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"`
> + res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
> if [ "$ipc" != "$res" ]; then
> - echo "IPC is different: $res != $ipc ($num / $cyc)"
> - exit 1
> + # check the difference from the real result for FP imperfections
> + diff=`echo $ipc $res $TRESHOLD | \
> + awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
> +
> + if [ $diff -eq 1 ]; then
> + echo "IPC is different: $res != $ipc ($num / $cyc)"
> + exit 1
> + fi
> +
> + echo "Warning: Difference of IPC is under the treshold"
> fi
> done
> }
> @@ -67,10 +77,18 @@ test_no_aggr()
> fi
>
> # use printf for rounding and a leading zero
> - res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"`
> + res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
> if [ "$ipc" != "$res" ]; then
> - echo "IPC is different for $cpu: $res != $ipc ($num / $cyc)"
> - exit 1
> + # check difference from the real result for FP imperfections
> + diff=`echo $ipc $res $TRESHOLD | \
> + awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
> +
> + if [ $diff -eq 1 ]; then
> + echo "IPC is different: $res != $ipc ($num / $cyc)"
> + exit 1
> + fi
> +
> + echo "Warning: Difference of IPC is under the treshold"
> fi
> done
> }
> --
> 2.41.0
>
--
- Arnaldo
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2023-09-12 21:01 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-09-12 12:19 [PATCH 1/2] perf test lock_contention.sh: Skip test if the number of CPUs is low vmolnaro
2023-09-12 12:19 ` [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors vmolnaro
2023-09-12 12:22 ` Michael Petlan
2023-09-12 21:01 ` Arnaldo Carvalho de Melo
2023-09-12 12:21 ` [PATCH 1/2] perf test lock_contention.sh: Skip test if the number of CPUs is low Michael Petlan
2023-09-12 20:57 ` Arnaldo Carvalho de Melo
-- strict thread matches above, loose matches on Subject: below --
2023-08-21 9:00 vmolnaro
2023-08-21 9:00 ` [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors vmolnaro
2023-08-23 9:02 ` Michael Petlan
2023-09-06 16:51 ` Arnaldo Carvalho de Melo
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.