[RFH] adding test coverage for contrib/ in CI jobs

[Junio C Hamano <gitster@pobox.com>]

Junio C Hamano <gitster@pobox.com> writes:

> Thanks.  I am tempted to propose us doing something like this, so
> that you guys do not have to every time you import my 'next'.
>
> --- >8 ---
> Subject: [PATCH] test: optionally test contrib in CI
>
> Recently it was reported that a topic merged to 'next' broke build
> and test for contrib/subtree part of the system.
>
> Instead of having those who run 'next' or 'master' to hit the build
> and test breakage and report to us, make sure we notice breakages in
> contrib/ area before they hit my tree at all, during their own
> presubmit testing.
>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
>
>  * There should be a cleaner way to make sure any new Makefile with
>    "test" target in contrib/* directores are added to the execution,
>    but for now this should do.

Unfortunately, this seems to reveal existing other problems with
subtree tests (t7900), in addition to diff-highlight tests (t9400)
in various configurations.

  https://github.com/git/git/actions/runs/21617099884

This Ci run is near the tip of 'seen', so there may be breakages
attributable to new topics in flight, but I suspect that many of
them are already in 'master', noticed by nobody because nobody ran
these tests in these configurations (like "breaking changes",
"sha256", "leaks", "reftable", "asan").

I didn't look into the details of any of these (yet).



>  Makefile                  |  6 ++++++
>  ci/run-build-and-tests.sh |  2 ++
>  contrib/Makefile          | 10 ++++++++++
>  3 files changed, 18 insertions(+)
>  create mode 100644 contrib/Makefile
>
> diff --git a/Makefile b/Makefile
> index 8aa489f3b6..d0ab8fdb04 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -342,6 +342,9 @@ include shared.mak
>  # If it isn't set, fallback to $LC_ALL, $LANG or use the first utf-8
>  # locale returned by "locale -a".
>  #
> +# Define TEST_CONTRIB_TOO to make "make test" run tests in contrib/
> +# directories.
> +#
>  # Define HAVE_CLOCK_GETTIME if your platform has clock_gettime.
>  #
>  # Define HAVE_CLOCK_MONOTONIC if your platform has CLOCK_MONOTONIC.
> @@ -3369,6 +3372,9 @@ export TEST_NO_MALLOC_CHECK
>  
>  test: all
>  	$(MAKE) -C t/ all
> +ifdef TEST_CONTRIB_TOO
> +	$(MAKE) -C contrib/ test
> +endif
>  
>  perf: all
>  	$(MAKE) -C t/perf/ all
> diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
> index 8bda62b921..b07b89f954 100755
> --- a/ci/run-build-and-tests.sh
> +++ b/ci/run-build-and-tests.sh
> @@ -5,6 +5,8 @@
>  
>  . ${0%/*}/lib.sh
>  
> +export TEST_CONTRIB_TOO=yes
> +
>  case "$jobname" in
>  fedora-breaking-changes-musl|linux-breaking-changes)
>  	export WITH_BREAKING_CHANGES=YesPlease
> diff --git a/contrib/Makefile b/contrib/Makefile
> new file mode 100644
> index 0000000000..787cd07f52
> --- /dev/null
> +++ b/contrib/Makefile
> @@ -0,0 +1,10 @@
> +all::
> +
> +test::
> +	$(MAKE) -C diff-highlight $@
> +	$(MAKE) -C subtree $@
> +
> +clean::
> +	$(MAKE) -C contacts $@
> +	$(MAKE) -C diff-highlight $@
> +	$(MAKE) -C subtree $@