Re: [PATCH 2/2] perf-lib: add test_perf_cleanup target

[Thomas Gummerer <t.gummerer@gmail.com>]

Junio C Hamano <gitster@pobox.com> writes:

> Thomas Gummerer <t.gummerer@gmail.com> writes:
>
>> +For performance tests that need cleaning up after them that should not
>> +be timed, use
>> +
>> +	test_perf_cleanup 'descriptive string' '
>> +		command1 &&
>> +		command2
>> +	' '
>> +		cleanupcommand1 &&
>> +		cleanupcommand2
>> +	'
>> +
>
> Hmm, if "not timing the clean-up actions" is the primary goal, is it
> an option to reuse test_when_finished for this (you may have to make
> sure that the commands run inside it are not timed; I didn't check).

Maybe I wasn't clear enough.  The goal is to allow tests to have
everything cleaned up after every run.  This can then be used for
commands that change the index (or other things) and get back to the
original state after that.  For example something like this:

file=$(git ls-files | tail -n 30 | head -1)

test_expect_success "change a file" "
	echo 'something' >>$file
"

test_perf_cleanup "v5 update-index file" "
	git update-index $file
" "
	git reset
"

test_when_finished on the other hand only cleans up when the whole test
is finished.

> One thing I felt uneasy about the above construct is that it makes
> cleanupcommand2 responsible for handling cases where not just
> command2 but also command1 might have failed.
>
> Compared to that, test-when-finished allows you to control what
> clean-up to do depending on what have already been done, i.e.
>
>         test_when_finished 'undo what command1 would have done' &&
> 	command1 &&
>         test_when_finished 'undo what command2 would have done' &&
> 	command2 &&
>         ...
>
> The second "undo command2" must be prepared for the case where
> command2 may have failed in the middle, but it can at least rely on
> command1 having succeeded when it is run.

When one performance test fails, the testing is aborted and the cleanup
commands are not executed anymore, leaving the trash directory in the
failed state.  This consistent with the normal tests with the immediate
flag passed to them.  (All performance tests have the --immediate flag
on implicitly).