All of lore.kernel.org
 help / color / mirror / Atom feed
From: Thomas Gummerer <t.gummerer@gmail.com>
To: git@vger.kernel.org
Cc: Jeff King <peff@peff.net>, Thomas Gummerer <t.gummerer@gmail.com>
Subject: [PATCH] perf-lib: remove old result files before running tests
Date: Tue, 19 Nov 2019 18:50:47 +0000	[thread overview]
Message-ID: <20191119185047.8550-1-t.gummerer@gmail.com> (raw)

The perf tests write files recording the results of tests.  These
results are later aggregated by 'aggregate.perl'.  If the tests are
run multiple times, those results are overwritten by the new results.
This works just fine as long as there are only perf tests measuring
the times, whose results are stored in "$base".times files.

However 22bec79d1a ("t/perf: add infrastructure for measuring sizes",
2018-08-17) introduced a new type of test for measuring the size of
something.  The results of this are written to "$base".size files.

"$base" is essentially made up of the basename of the script plus the
test number.  So if test numbers shift because a new test was
introduced earlier in the script we might end up with both a ".times"
and a ".size" file for the same test.  In the aggregation script the
".times" file is preferred over the ".size" file, so some size tests
might end with performance numbers from a previous run of the test.

This is mainly relevant when writing perf tests that check both
performance and sizes, and can get quite confusing during
developement.

Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---

This came out of something different that I'm working on, but makes
most sense as a standalone patch, rather than part of that series, so
I'm sending this out separately.

 t/perf/perf-lib.sh | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/t/perf/perf-lib.sh b/t/perf/perf-lib.sh
index b58a43ea43..7e80251889 100644
--- a/t/perf/perf-lib.sh
+++ b/t/perf/perf-lib.sh
@@ -178,10 +178,11 @@ test_wrapper_ () {
 	export test_prereq
 	if ! test_skip "$@"
 	then
-		base=$(basename "$0" .sh)
-		echo "$test_count" >>"$perf_results_dir"/$base.subtests
-		echo "$1" >"$perf_results_dir"/$base.$test_count.descr
 		base="$perf_results_dir"/"$PERF_RESULTS_PREFIX$(basename "$0" .sh)"."$test_count"
+		rm -f "$base".*
+		no_prefix_base="$perf_results_dir"/$(basename "$0" .sh)
+		echo "$test_count" >>$no_prefix_base.subtests
+		echo "$1" >$no_prefix_base.$test_count.descr
 		"$test_wrapper_func_" "$@"
 	fi
 
-- 
2.24.0.155.gd9f6f3b619


             reply	other threads:[~2019-11-19 18:50 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-11-19 18:50 Thomas Gummerer [this message]
2019-11-20  4:12 ` [PATCH] perf-lib: remove old result files before running tests Junio C Hamano
2019-11-20  8:00   ` Thomas Gummerer
2019-11-21 10:20 ` Jeff King
2019-11-22  8:11   ` Thomas Gummerer
2019-11-25 14:09     ` Jeff King
2019-11-25 17:04       ` Thomas Gummerer

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20191119185047.8550-1-t.gummerer@gmail.com \
    --to=t.gummerer@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=peff@peff.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.