[PATCH] ci: fix aggregation of test results with Meson

[PATCH] ci: fix aggregation of test results with Meson

[Patrick Steinhardt]

Our CI needs to be aware of the location of the test output directory so
that it knows where to find test results. Some of our CI jobs achieve
this by setting the `TEST_OUTPUT_DIRECTORY` environment variable, which
ensures that the output will be written to that directory. Other jobs,
especially on GitHub Workflows, don't set that environment variable and
instead expect test results to be located in the source directory in
"t/".

The latter logic does not work with Meson though, as the test results
are not written into the source directory by default, but instead into
the build directory. As such, any job that uses Meson without setting
the environment variable will be unable to locate and aggregate results.

Fix this by explicitly setting the test output directory when we set up
the Meson build directory. Like this, we can easily default to "t/" in
the source directory when the value hasn't been set explicitly.

Reported-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
Hi,

Johannes reported to me off-list that aggregation of test results
doesn't work on GitHub Workflow when using Meson, as can be seen e.g. in
[1].

As it turns out, the issue is that we don't set `TEST_OUTPUT_DIRECTORY`
for many of the GitHub Workflows jobs. And because Meson by default puts
test results into the build instead of into the source directory our
assumption that the results can be found in "t/" is broken. This isn't 

I never noticed this failure myself because in GitLab we always set the
above environment variable there. In any case, this patch fixes it as
can be seen at [2].

Thanks!

Patrick

[1]: https://github.com/git-for-windows/git/actions/runs/14806194960/job/41574766327#step:9:2125
[2]: https://github.com/git/git/actions/runs/14833082023/job/41638408968
---
 ci/run-build-and-tests.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index f99b7db2ee8..bd300a05db8 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -56,6 +56,7 @@ case "$jobname" in
 		--warnlevel 2 --werror \
 		--wrap-mode nofallback \
 		-Dfuzzers=true \
+		-Dtest_output_directory="${TEST_OUTPUT_DIRECTORY:-$(pwd)/t}" \
 		$MESONFLAGS
 	group "Build" meson compile -C build --
 	if test -n "$run_tests"

---
base-commit: 6c0bd1fc70efaf053abe4e57c976afdc72d15377
change-id: 20250505-b4-pks-meson-aggregate-results-60ef175dd424

Re: [PATCH] ci: fix aggregation of test results with Meson

[Johannes Schindelin]

Hi Patrick,

On Mon, 5 May 2025, Patrick Steinhardt wrote:

> Our CI needs to be aware of the location of the test output directory so
> that it knows where to find test results. Some of our CI jobs achieve
> this by setting the `TEST_OUTPUT_DIRECTORY` environment variable, which
> ensures that the output will be written to that directory. Other jobs,
> especially on GitHub Workflows, don't set that environment variable and
> instead expect test results to be located in the source directory in
> "t/".
> 
> The latter logic does not work with Meson though, as the test results
> are not written into the source directory by default, but instead into
> the build directory. As such, any job that uses Meson without setting
> the environment variable will be unable to locate and aggregate results.
> 
> Fix this by explicitly setting the test output directory when we set up
> the Meson build directory. Like this, we can easily default to "t/" in
> the source directory when the value hasn't been set explicitly.
> 
> Reported-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
> Hi,
> 
> Johannes reported to me off-list that aggregation of test results
> doesn't work on GitHub Workflow when using Meson, as can be seen e.g. in
> [1].
> 
> As it turns out, the issue is that we don't set `TEST_OUTPUT_DIRECTORY`
> for many of the GitHub Workflows jobs. And because Meson by default puts
> test results into the build instead of into the source directory our
> assumption that the results can be found in "t/" is broken. This isn't 
> 
> I never noticed this failure myself because in GitLab we always set the
> above environment variable there. In any case, this patch fixes it as
> can be seen at [2].
> 
> Thanks!

Thank _you_!
Johannes

> 
> Patrick
> 
> [1]: https://github.com/git-for-windows/git/actions/runs/14806194960/job/41574766327#step:9:2125
> [2]: https://github.com/git/git/actions/runs/14833082023/job/41638408968
> ---
>  ci/run-build-and-tests.sh | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
> index f99b7db2ee8..bd300a05db8 100755
> --- a/ci/run-build-and-tests.sh
> +++ b/ci/run-build-and-tests.sh
> @@ -56,6 +56,7 @@ case "$jobname" in
>  		--warnlevel 2 --werror \
>  		--wrap-mode nofallback \
>  		-Dfuzzers=true \
> +		-Dtest_output_directory="${TEST_OUTPUT_DIRECTORY:-$(pwd)/t}" \
>  		$MESONFLAGS
>  	group "Build" meson compile -C build --
>  	if test -n "$run_tests"
> 
> ---
> base-commit: 6c0bd1fc70efaf053abe4e57c976afdc72d15377
> change-id: 20250505-b4-pks-meson-aggregate-results-60ef175dd424
> 
> 

Re: [PATCH] ci: fix aggregation of test results with Meson

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> Johannes reported to me off-list that aggregation of test results
>> doesn't work on GitHub Workflow when using Meson, as can be seen e.g. in
>> [1].
>> ...
>> 
>> Thanks!
>
> Thank _you_!
> Johannes

Thank both of you ;-)