[PATCH v3 0/3] test-lib: improve missing prereq handling

[Fabian Stelzer <fs@gigacodes.de>]

The ssh signing feature was breaking tests when the broken openssh-8.7
was used. We have now fixed that by checking for this exact case in the
GPGSSH prereq and I will improve that check further in a future patch.
However we are now in a situation where a broken openssh in the future
will result in successfull tests but not a working git build afterwards
(either not compiling in the expected feature or like in the ssh case
runtime failures) resulting in a false sense of security in the tests.
This patches try to improve this situation by showing which prereqs
failed in the test summary and by adding an environment variable to
enforce certain prereqs to succeed or abort the test otherwise.

See also:
https://public-inbox.org/git/xmqqv916wh7t.fsf@gitster.g/

changes sinve v2:
 - use a space separated list for GIT_TES_REQUIRED_PREREQ like we do for
   GIT_SKIP_TESTS
 - use BAIL_OUT() insted of just error()
 - make BAIL_OUT() print errors even when used within prereq context

changes since v1:
 - use \012 instead of \n for possible portability reasons
 - fix typo in commit msg

Fabian Stelzer (3):
  test-lib: show missing prereq summary
  test-lib: introduce required prereq for test runs
  test-lib: make BAIL_OUT() work in tests and prereq

 t/README                |  6 ++++++
 t/aggregate-results.sh  | 17 +++++++++++++++++
 t/test-lib-functions.sh | 11 +++++++++++
 t/test-lib.sh           | 21 +++++++++++++++++----
 4 files changed, 51 insertions(+), 4 deletions(-)

Range-diff against v2:
1:  69e77cd854 ! 1:  35c92671e5 test-lib: show missing prereq summary
    @@ t/aggregate-results.sh: do
     +		tr -s "," "\n" |
     +		grep -v '^$' |
     +		sort -u |
    -+		paste -s -d ',')
    -+	if test -n $unique_missing_prereq
    ++		paste -s -d ' ')
    ++	if test -n "$unique_missing_prereq"
     +	then
     +		printf "\nmissing prereq: $unique_missing_prereq\n\n"
     +	fi
2:  12bd18c5ce ! 2:  d6a53f0980 test-lib: introduce required prereq for test runs
    @@ Commit message
         test-lib: introduce required prereq for test runs
     
         In certain environments or for specific test scenarios we might expect a
    -    specific prerequisite check to be succeed. Therefore we would like to
    -    trigger an error when running our tests if this is not the case.
    +    specific prerequisite check to succeed. Therefore we would like to abort
    +    running our tests if this is not the case.
     
         To remedy this we add the environment variable GIT_TEST_REQUIRE_PREREQ
    -    which can be set to a comma separated list of prereqs. If one of these
    +    which can be set to a space separated list of prereqs. If one of these
         prereq tests fail then the whole test run will abort.
     
         Signed-off-by: Fabian Stelzer <fs@gigacodes.de>
    @@ t/README: explicitly providing repositories when accessing submodule objects is
      complete or needs to be abandoned for whatever reason (in which case the
      migrated codepaths still retain their performance benefits).
      
    -+GIT_TEST_REQUIRE_PREREQ=<list> allows specifying a comma speparated list of
    ++GIT_TEST_REQUIRE_PREREQ=<list> allows specifying a space speparated list of
     +prereqs that are required to succeed. If a prereq in this list is triggered by
     +a test and then fails then the whole test run will abort. This can help to make
     +sure the expected tests are executed and not silently skipped when their
    @@ t/test-lib-functions.sh: test_have_prereq () {
      			prerequisite=${negative_prereq:+!}$prerequisite
     +
     +			# Abort if this prereq was marked as required
    -+			if test -n $GIT_TEST_REQUIRE_PREREQ
    ++			if test -n "$GIT_TEST_REQUIRE_PREREQ"
     +			then
    -+				case ",$GIT_TEST_REQUIRE_PREREQ," in
    -+				*,$prerequisite,*)
    -+					error "required prereq $prerequisite failed"
    ++				case " $GIT_TEST_REQUIRE_PREREQ " in
    ++				*" $prerequisite "*)
    ++					BAIL_OUT "required prereq $prerequisite failed"
     +					;;
     +				esac
     +			fi
-:  ---------- > 3:  de21c484d6 test-lib: make BAIL_OUT() work in tests and prereq

base-commit: cd3e606211bb1cf8bc57f7d76bab98cc17a150bc
-- 
2.31.1