[PATCH 0/5] Some assorted fixes for GitLab CI

[PATCH 0/5] Some assorted fixes for GitLab CI

[Patrick Steinhardt]

Hi,

I recently had the pleasure of debugging a couple of failing
MSVC+Windows jobs in GitLab CI, which hasn't been quite fun because we
didn't know to print error logs, and neither did we upload the failed
test artifacts. This patch series is the result of this frustration and
fixes a couple of smaller issues in the context of our CI:

  - I noticed that test slicing is slightly wrong because of a
    difference between zero- and one-based indices, which causes us to
    skip the first test on GitLab.

  - I deduplicated how we run Meson tests so that both GitLab and GitHub
    use the same "run-test-slice-meson.sh" script.

  - I add logic to handle failing tests via "print-test-failures.sh".

The result can be found at [1]. Note that tests are failing, but those
failures are fixed in a separate patch series via [2]. In any case, I
guess those test failures also serve as a good demonstration how the
failing tests show up now.

Thanks!

Patrick

[1]: https://gitlab.com/gitlab-org/git/-/merge_requests/497
[2]: <20260209-b4-pks-ci-msvc-iconv-fixes-v1-0-1e3167cd8828@pks.im>

---
Patrick Steinhardt (5):
      ci: handle failures of test-slice helper
      ci: don't skip smallest test slice in GitLab
      ci: make test slicing consistent across Meson/Make
      gitlab-ci: use "run-test-slice-meson.sh"
      gitlab-ci: handle failed tests on MSVC+Meson job

 .github/workflows/main.yml |  4 ++--
 .gitlab-ci.yml             | 17 +++++++++++++++--
 ci/run-test-slice-meson.sh |  2 +-
 ci/run-test-slice.sh       |  6 +++---
 t/helper/test-path-utils.c | 18 ++++++++++++------
 5 files changed, 33 insertions(+), 14 deletions(-)


---
base-commit: 3e0db84c88c57e70ac8be8c196dfa92c5d656fbc
change-id: 20260209-b4-pks-ci-meson-improvements-93d8a1ffdd27

[PATCH 1/5] ci: handle failures of test-slice helper

[Patrick Steinhardt]

The "run-test-slice.sh" script executes the test helper to slice up
tests passed to it. As the execution is part of a pipe though, we end up
ignoring any potential error code returned by the helper.

Make the code more robust by storing the tests in a variable first so
that we can split up the pipeline.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 ci/run-test-slice.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh
index 0444c79c02..ff948e397f 100755
--- a/ci/run-test-slice.sh
+++ b/ci/run-test-slice.sh
@@ -5,9 +5,9 @@
 
 . ${0%/*}/lib.sh
 
-group "Run tests" make --quiet -C t T="$(cd t &&
-	./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
-	tr '\n' ' ')" ||
+TESTS=$(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh)
+
+group "Run tests" make --quiet -C t T="$(echo "$TESTS" | tr '\n' ' ')" ||
 handle_failed_tests
 
 # We only have one unit test at the moment, so run it in the first slice

-- 
2.53.0.295.g64333814d3.dirty

[PATCH 2/5] ci: don't skip smallest test slice in GitLab

[Patrick Steinhardt]

The "ci/run-test-slice.sh" script can be used to slice up all of our
tests into N pieces and then run each of them on a separate CI job.
This is used by both GitLab and GitHub CI to speed up Windows tests,
which would otherwise be painfully slow.

The infra itself is fueled by `test-tool path-utils slice-tests`. This
tool receives as input an "offset" and a "stride" that can be combined
to slice up tests. This framing can be misleading though: you are
expected to pass a zero-based index as "offset", and the complete number
of slices to the "stride". The latter makes sense, but it is somewhat
surprising that the offset needs to be zero-based. And this is in fact
biting us: while GitHub passes zero-based indices, GitLab passes
`$CI_NODE_INDEX`, which is a one-based indice.

Ideally, we should have verification that the parameters make sense.
And naturally, one would for example expect that it's an error to call
the binary with an offset larger than the stride. But with the current
framing as "offset" it's not even wrong to do so, as it is of course
well-defined to start at a larger offset than the stride.

This means that we get this wrong on GitLab's CI, as we pass a one based
index there, and this causes us to skip one of the tests. Interestingly,
it's not the lexicographically first test that we skip. Instead, as we
sort tests by size before slicing them, we skip the _smallest_ test.

Reframe the problem to instead talk about "slice number" and "total
number of slices". For all of our use cases this is semantically
equivalent, but it allows us to perform some verifications:

  - The total number of slices must be greater than 1.

  - The selected slice must be between 1 <= nr <= slices_total.

As the indices are now one-based it means that GitLab's CI is fixed.
The GitHub workflow is updated accordingly.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 .github/workflows/main.yml |  2 +-
 t/helper/test-path-utils.c | 18 ++++++++++++------
 2 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index f2e93f5461..2b175dc5c6 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -150,7 +150,7 @@ jobs:
     - uses: git-for-windows/setup-git-for-windows-sdk@v1
     - name: test
       shell: bash
-      run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
+      run: . /etc/profile && ci/run-test-slice.sh ${{ matrix.nr + 1 }} 10
     - name: print test failures
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
       shell: bash
diff --git a/t/helper/test-path-utils.c b/t/helper/test-path-utils.c
index f5f33751da..874542ec34 100644
--- a/t/helper/test-path-utils.c
+++ b/t/helper/test-path-utils.c
@@ -477,14 +477,20 @@ int cmd__path_utils(int argc, const char **argv)
 
 	if (argc > 5 && !strcmp(argv[1], "slice-tests")) {
 		int res = 0;
-		long offset, stride, i;
+		long slice, slices_total, i;
 		struct string_list list = STRING_LIST_INIT_NODUP;
 		struct stat st;
 
-		offset = strtol(argv[2], NULL, 10);
-		stride = strtol(argv[3], NULL, 10);
-		if (stride < 1)
-			stride = 1;
+		slices_total = strtol(argv[3], NULL, 10);
+		if (slices_total < 1)
+			die("there must be at least one slice, got '%s'",
+			    argv[3]);
+
+		slice = strtol(argv[2], NULL, 10);
+		if (1 > slice || slice > slices_total)
+			die("slice must be in the range 1 <= slice <= %ld, got '%s'",
+			    slices_total, argv[2]);
+
 		for (i = 4; i < argc; i++)
 			if (stat(argv[i], &st))
 				res = error_errno("Cannot stat '%s'", argv[i]);
@@ -492,7 +498,7 @@ int cmd__path_utils(int argc, const char **argv)
 				string_list_append(&list, argv[i])->util =
 					(void *)(intptr_t)st.st_size;
 		QSORT(list.items, list.nr, cmp_by_st_size);
-		for (i = offset; i < list.nr; i+= stride)
+		for (i = slice - 1; i < list.nr; i+= slices_total)
 			printf("%s\n", list.items[i].string);
 
 		return !!res;

-- 
2.53.0.295.g64333814d3.dirty

Re: [PATCH 2/5] ci: don't skip smallest test slice in GitLab

[Justin Tobler]

On 26/02/09 05:56PM, Patrick Steinhardt wrote:
> The "ci/run-test-slice.sh" script can be used to slice up all of our
> tests into N pieces and then run each of them on a separate CI job.
> This is used by both GitLab and GitHub CI to speed up Windows tests,
> which would otherwise be painfully slow.
> 
> The infra itself is fueled by `test-tool path-utils slice-tests`. This
> tool receives as input an "offset" and a "stride" that can be combined
> to slice up tests. This framing can be misleading though: you are
> expected to pass a zero-based index as "offset", and the complete number
> of slices to the "stride". The latter makes sense, but it is somewhat
> surprising that the offset needs to be zero-based. And this is in fact
> biting us: while GitHub passes zero-based indices, GitLab passes
> `$CI_NODE_INDEX`, which is a one-based indice.
> 
> Ideally, we should have verification that the parameters make sense.
> And naturally, one would for example expect that it's an error to call
> the binary with an offset larger than the stride. But with the current
> framing as "offset" it's not even wrong to do so, as it is of course
> well-defined to start at a larger offset than the stride.

It was also suprising for me to see that the "offset" could be set to a
value higher than the stride. I can't see any reason that we would want
this to be the case.

> This means that we get this wrong on GitLab's CI, as we pass a one based
> index there, and this causes us to skip one of the tests. Interestingly,
> it's not the lexicographically first test that we skip. Instead, as we
> sort tests by size before slicing them, we skip the _smallest_ test.
> 
> Reframe the problem to instead talk about "slice number" and "total
> number of slices". For all of our use cases this is semantically
> equivalent, but it allows us to perform some verifications:
> 
>   - The total number of slices must be greater than 1.
> 
>   - The selected slice must be between 1 <= nr <= slices_total.

This seems reasonable to me.

> As the indices are now one-based it means that GitLab's CI is fixed.
> The GitHub workflow is updated accordingly.
> 
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  .github/workflows/main.yml |  2 +-
>  t/helper/test-path-utils.c | 18 ++++++++++++------
>  2 files changed, 13 insertions(+), 7 deletions(-)
> 
> diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
> index f2e93f5461..2b175dc5c6 100644
> --- a/.github/workflows/main.yml
> +++ b/.github/workflows/main.yml
> @@ -150,7 +150,7 @@ jobs:
>      - uses: git-for-windows/setup-git-for-windows-sdk@v1
>      - name: test
>        shell: bash
> -      run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
> +      run: . /etc/profile && ci/run-test-slice.sh ${{ matrix.nr + 1 }} 10

Here the GitHub CI is updated to be one-based indexed. The GitLab CI is
already set up that way.

>      - name: print test failures
>        if: failure() && env.FAILED_TEST_ARTIFACTS != ''
>        shell: bash
> diff --git a/t/helper/test-path-utils.c b/t/helper/test-path-utils.c
> index f5f33751da..874542ec34 100644
> --- a/t/helper/test-path-utils.c
> +++ b/t/helper/test-path-utils.c
> @@ -477,14 +477,20 @@ int cmd__path_utils(int argc, const char **argv)
>  
>  	if (argc > 5 && !strcmp(argv[1], "slice-tests")) {
>  		int res = 0;
> -		long offset, stride, i;
> +		long slice, slices_total, i;
>  		struct string_list list = STRING_LIST_INIT_NODUP;
>  		struct stat st;
>  
> -		offset = strtol(argv[2], NULL, 10);
> -		stride = strtol(argv[3], NULL, 10);
> -		if (stride < 1)
> -			stride = 1;
> +		slices_total = strtol(argv[3], NULL, 10);
> +		if (slices_total < 1)
> +			die("there must be at least one slice, got '%s'",
> +			    argv[3]);

Here we validate the slices count is greater than one.

> +
> +		slice = strtol(argv[2], NULL, 10);
> +		if (1 > slice || slice > slices_total)
> +			die("slice must be in the range 1 <= slice <= %ld, got '%s'",
> +			    slices_total, argv[2]);

Here we validate the provided slice index is in the correct range.

> +
>  		for (i = 4; i < argc; i++)
>  			if (stat(argv[i], &st))
>  				res = error_errno("Cannot stat '%s'", argv[i]);
> @@ -492,7 +498,7 @@ int cmd__path_utils(int argc, const char **argv)
>  				string_list_append(&list, argv[i])->util =
>  					(void *)(intptr_t)st.st_size;
>  		QSORT(list.items, list.nr, cmp_by_st_size);
> -		for (i = offset; i < list.nr; i+= stride)
> +		for (i = slice - 1; i < list.nr; i+= slices_total)
>  			printf("%s\n", list.items[i].string);
>  
>  		return !!res;

This patch looks good.

-Justin

[PATCH 3/5] ci: make test slicing consistent across Meson/Make

[Patrick Steinhardt]

In the preceding commit we have adjusted test slicing to be one-based
when using the "ci/run-test-slice.sh" script. But we also have an
equivalent script for Meson that is still zero-based, which is of course
inconsistent.

Adapt the script to be one-based, as well, and adapt the GitHub workflow
accordingly. Note that GitLab doesn't yet use the script, so it does not
need to be adapted. This will change in the next commit though.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 .github/workflows/main.yml | 2 +-
 ci/run-test-slice-meson.sh | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 2b175dc5c6..1b7a16e1f1 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -298,7 +298,7 @@ jobs:
         path: build
     - name: Test
       shell: pwsh
-      run: ci/run-test-slice-meson.sh build ${{matrix.nr}} 10
+      run: ci/run-test-slice-meson.sh build ${{matrix.nr + 1}} 10
     - name: print test failures
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
       shell: bash
diff --git a/ci/run-test-slice-meson.sh b/ci/run-test-slice-meson.sh
index 961c94fba0..a6df927ba5 100755
--- a/ci/run-test-slice-meson.sh
+++ b/ci/run-test-slice-meson.sh
@@ -9,5 +9,5 @@
 
 group "Run tests" \
 	meson test -C "$1" --no-rebuild --print-errorlogs \
-		--test-args="$GIT_TEST_OPTS" --slice "$((1+$2))/$3" ||
+		--test-args="$GIT_TEST_OPTS" --slice "$(($2))/$3" ||
 handle_failed_tests

-- 
2.53.0.295.g64333814d3.dirty

Re: [PATCH 3/5] ci: make test slicing consistent across Meson/Make

[Justin Tobler]

On 26/02/09 05:56PM, Patrick Steinhardt wrote:
> In the preceding commit we have adjusted test slicing to be one-based
> when using the "ci/run-test-slice.sh" script. But we also have an
> equivalent script for Meson that is still zero-based, which is of course
> inconsistent.
> 
> Adapt the script to be one-based, as well, and adapt the GitHub workflow
> accordingly. Note that GitLab doesn't yet use the script, so it does not
> need to be adapted. This will change in the next commit though.
> 
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  .github/workflows/main.yml | 2 +-
>  ci/run-test-slice-meson.sh | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
> index 2b175dc5c6..1b7a16e1f1 100644
> --- a/.github/workflows/main.yml
> +++ b/.github/workflows/main.yml
> @@ -298,7 +298,7 @@ jobs:
>          path: build
>      - name: Test
>        shell: pwsh
> -      run: ci/run-test-slice-meson.sh build ${{matrix.nr}} 10
> +      run: ci/run-test-slice-meson.sh build ${{matrix.nr + 1}} 10

Due to the changes in the prior patch, GitHub CI passing 0 as the slice
value would cause a failure correct? I wonder if we should combine this
change with the previous patch. Otherwise this patch looks good.

-Justin

Re: [PATCH 3/5] ci: make test slicing consistent across Meson/Make

[Patrick Steinhardt]

On Mon, Feb 09, 2026 at 12:19:57PM -0600, Justin Tobler wrote:
> On 26/02/09 05:56PM, Patrick Steinhardt wrote:
> > diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
> > index 2b175dc5c6..1b7a16e1f1 100644
> > --- a/.github/workflows/main.yml
> > +++ b/.github/workflows/main.yml
> > @@ -298,7 +298,7 @@ jobs:
> >          path: build
> >      - name: Test
> >        shell: pwsh
> > -      run: ci/run-test-slice-meson.sh build ${{matrix.nr}} 10
> > +      run: ci/run-test-slice-meson.sh build ${{matrix.nr + 1}} 10
> 
> Due to the changes in the prior patch, GitHub CI passing 0 as the slice
> value would cause a failure correct? I wonder if we should combine this
> change with the previous patch. Otherwise this patch looks good.

Note that this is the "-meson.sh" variant, so this is a different
script. I'm mostly just touching up this variant so that it behaves the
same as the non-Meson one.

Meson itself would die though in case it's passed an invalid range.

Patrick

Re: [PATCH 3/5] ci: make test slicing consistent across Meson/Make

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> In the preceding commit we have adjusted test slicing to be one-based
> when using the "ci/run-test-slice.sh" script. But we also have an
> equivalent script for Meson that is still zero-based, which is of course
> inconsistent.
>
> Adapt the script to be one-based, as well, and adapt the GitHub workflow
> accordingly. Note that GitLab doesn't yet use the script, so it does not
> need to be adapted. This will change in the next commit though.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  .github/workflows/main.yml | 2 +-
>  ci/run-test-slice-meson.sh | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
> index 2b175dc5c6..1b7a16e1f1 100644
> --- a/.github/workflows/main.yml
> +++ b/.github/workflows/main.yml
> @@ -298,7 +298,7 @@ jobs:
>          path: build
>      - name: Test
>        shell: pwsh
> -      run: ci/run-test-slice-meson.sh build ${{matrix.nr}} 10
> +      run: ci/run-test-slice-meson.sh build ${{matrix.nr + 1}} 10
>      - name: print test failures
>        if: failure() && env.FAILED_TEST_ARTIFACTS != ''
>        shell: bash

Have we successfully run this one?

I am getting

Invalid workflow file: .github/workflows/main.yml#L1
(Line: 153, Col: 12): Unexpected symbol: '+'. Located at position 11
within expression: matrix.nr + 1, (Line: 301, Col: 12): Unexpected
symbol: '+'. Located at position 11 within expression: matrix.nr + 1

https://github.com/orgs/community/discussions/25386 is a 6-year old
discussion so things may have changed quite a lot, but at least back
then the claim was

    Github actions doesn’t support math operations in expressions
    inside ${{ }}. You could add up these two numbers in bash script and
    then use set-env command to give its value to an environment
    variable ...

though.

In the meantime I'll revert the topic out of 'next'.  Sorry for not
catching it while it was in 'seen',.

Re: [PATCH 3/5] ci: make test slicing consistent across Meson/Make

[Jeff King]

On Tue, Feb 10, 2026 at 02:15:13PM -0800, Junio C Hamano wrote:

> > diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
> > index 2b175dc5c6..1b7a16e1f1 100644
> > --- a/.github/workflows/main.yml
> > +++ b/.github/workflows/main.yml
> > @@ -298,7 +298,7 @@ jobs:
> >          path: build
> >      - name: Test
> >        shell: pwsh
> > -      run: ci/run-test-slice-meson.sh build ${{matrix.nr}} 10
> > +      run: ci/run-test-slice-meson.sh build ${{matrix.nr + 1}} 10
> >      - name: print test failures
> >        if: failure() && env.FAILED_TEST_ARTIFACTS != ''
> >        shell: bash
> 
> Have we successfully run this one?
> 
> I am getting
> 
> Invalid workflow file: .github/workflows/main.yml#L1
> (Line: 153, Col: 12): Unexpected symbol: '+'. Located at position 11
> within expression: matrix.nr + 1, (Line: 301, Col: 12): Unexpected
> symbol: '+'. Located at position 11 within expression: matrix.nr + 1
> 
> https://github.com/orgs/community/discussions/25386 is a 6-year old
> discussion so things may have changed quite a lot, but at least back
> then the claim was
> 
>     Github actions doesn’t support math operations in expressions
>     inside ${{ }}. You could add up these two numbers in bash script and
>     then use set-env command to give its value to an environment
>     variable ...
> 
> though.

Right, that's why we used pwsh syntax to do it before, in d3d6493dcf
(ci: use Meson's new `--slice` option, 2025-07-09).

That went away in 17bd1108ea (ci(windows-meson-test): handle options and
output like other test jobs, 2025-11-18), because the "+1" was added
into the script itself there. It looks like the patch under discussion
removes the +1 from the script, so we'd need to go back to the pwsh
syntax.

-Peff

Re: [PATCH 3/5] ci: make test slicing consistent across Meson/Make

[Patrick Steinhardt]

On Tue, Feb 10, 2026 at 05:54:01PM -0500, Jeff King wrote:
> On Tue, Feb 10, 2026 at 02:15:13PM -0800, Junio C Hamano wrote:
> 
> > > diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
> > > index 2b175dc5c6..1b7a16e1f1 100644
> > > --- a/.github/workflows/main.yml
> > > +++ b/.github/workflows/main.yml
> > > @@ -298,7 +298,7 @@ jobs:
> > >          path: build
> > >      - name: Test
> > >        shell: pwsh
> > > -      run: ci/run-test-slice-meson.sh build ${{matrix.nr}} 10
> > > +      run: ci/run-test-slice-meson.sh build ${{matrix.nr + 1}} 10
> > >      - name: print test failures
> > >        if: failure() && env.FAILED_TEST_ARTIFACTS != ''
> > >        shell: bash
> > 
> > Have we successfully run this one?

I haven't kicked off a GitHub workflow for this patch series. Guess I
should've done that. I've created https://github.com/git/git/pull/2195
now to give v2 a test run first.

> > I am getting
> > 
> > Invalid workflow file: .github/workflows/main.yml#L1
> > (Line: 153, Col: 12): Unexpected symbol: '+'. Located at position 11
> > within expression: matrix.nr + 1, (Line: 301, Col: 12): Unexpected
> > symbol: '+'. Located at position 11 within expression: matrix.nr + 1
> > 
> > https://github.com/orgs/community/discussions/25386 is a 6-year old
> > discussion so things may have changed quite a lot, but at least back
> > then the claim was
> > 
> >     Github actions doesn’t support math operations in expressions
> >     inside ${{ }}. You could add up these two numbers in bash script and
> >     then use set-env command to give its value to an environment
> >     variable ...
> > 
> > though.
> 
> Right, that's why we used pwsh syntax to do it before, in d3d6493dcf
> (ci: use Meson's new `--slice` option, 2025-07-09).
> 
> That went away in 17bd1108ea (ci(windows-meson-test): handle options and
> output like other test jobs, 2025-11-18), because the "+1" was added
> into the script itself there. It looks like the patch under discussion
> removes the +1 from the script, so we'd need to go back to the pwsh
> syntax.

Indeed, will fix. Thanks!

Patrick

[PATCH 4/5] gitlab-ci: use "run-test-slice-meson.sh"

[Patrick Steinhardt]

While our GitHub workflow already uses "ci/run-test-slice-meson.sh",
GitLab CI open-codes the parameters. Adapt the latter to also use the
same script so that we always use the same Meson options across both CI
systems.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 .gitlab-ci.yml | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index b419a84e2c..04857b479d 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -183,7 +183,8 @@ test:msvc-meson:
     - job: "build:msvc-meson"
       artifacts: true
   script:
-    - meson test -C build --no-rebuild --print-errorlogs --slice $Env:CI_NODE_INDEX/$Env:CI_NODE_TOTAL
+    - |
+      & "C:/Program Files/Git/usr/bin/bash.exe" -l -c 'ci/run-test-slice-meson.sh build $CI_NODE_INDEX $CI_NODE_TOTAL'
   parallel: 10
   artifacts:
     reports:

-- 
2.53.0.295.g64333814d3.dirty

[PATCH 5/5] gitlab-ci: handle failed tests on MSVC+Meson job

[Patrick Steinhardt]

The MSVC+Meson job does not currently have any logic to print failing
tests, nor does it upload the failed test artifacts. Backfill this logic
to make help debugging efforts in case any of its jobs has failed.

GitHub already knows to do this, so we don't need an equivalent change
over there.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 .gitlab-ci.yml | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 04857b479d..71b8a6e642 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -157,6 +157,8 @@ test:mingw64:
   parallel: 10
 
 .msvc-meson:
+  variables:
+    TEST_OUTPUT_DIRECTORY: "C:/Git-Test"
   tags:
     - saas-windows-medium-amd64
   before_script:
@@ -164,12 +166,13 @@ test:mingw64:
     - choco install -y git meson ninja rust-ms
     - Import-Module $env:ChocolateyInstall\helpers\chocolateyProfile.psm1
     - refreshenv
+    - New-Item -Path $env:TEST_OUTPUT_DIRECTORY -ItemType Directory
 
 build:msvc-meson:
   extends: .msvc-meson
   stage: build
   script:
-    - meson setup build --vsenv -Dperl=disabled -Dbackend_max_links=1 -Dcredential_helpers=wincred
+    - meson setup build --vsenv -Dperl=disabled -Dbackend_max_links=1 -Dcredential_helpers=wincred -Dtest_output_directory="$TEST_OUTPUT_DIRECTORY"
     - meson compile -C build
   artifacts:
     paths:
@@ -185,10 +188,19 @@ test:msvc-meson:
   script:
     - |
       & "C:/Program Files/Git/usr/bin/bash.exe" -l -c 'ci/run-test-slice-meson.sh build $CI_NODE_INDEX $CI_NODE_TOTAL'
+  after_script:
+    - |
+      if ($env:CI_JOB_STATUS -ne "success") {
+        & "C:/Program Files/Git/usr/bin/bash.exe" -l -c 'ci/print-test-failures.sh'
+        Move-Item -Path "$env:TEST_OUTPUT_DIRECTORY/failed-test-artifacts" -Destination t/
+      }
   parallel: 10
   artifacts:
+    paths:
+      - t/failed-test-artifacts
     reports:
       junit: build/meson-logs/testlog.junit.xml
+    when: on_failure
 
 test:fuzz-smoke-tests:
   image: ubuntu:latest

-- 
2.53.0.295.g64333814d3.dirty

Re: [PATCH 5/5] gitlab-ci: handle failed tests on MSVC+Meson job

[Justin Tobler]

On 26/02/09 05:56PM, Patrick Steinhardt wrote:
> The MSVC+Meson job does not currently have any logic to print failing
> tests, nor does it upload the failed test artifacts. Backfill this logic
> to make help debugging efforts in case any of its jobs has failed.
> 
> GitHub already knows to do this, so we don't need an equivalent change
> over there.
> 
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  .gitlab-ci.yml | 14 +++++++++++++-
>  1 file changed, 13 insertions(+), 1 deletion(-)
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 04857b479d..71b8a6e642 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -157,6 +157,8 @@ test:mingw64:
>    parallel: 10
>  
>  .msvc-meson:
> +  variables:
> +    TEST_OUTPUT_DIRECTORY: "C:/Git-Test"
>    tags:
>      - saas-windows-medium-amd64
>    before_script:
> @@ -164,12 +166,13 @@ test:mingw64:
>      - choco install -y git meson ninja rust-ms
>      - Import-Module $env:ChocolateyInstall\helpers\chocolateyProfile.psm1
>      - refreshenv
> +    - New-Item -Path $env:TEST_OUTPUT_DIRECTORY -ItemType Directory

Before the script starts we create the test output directory.

>  build:msvc-meson:
>    extends: .msvc-meson
>    stage: build
>    script:
> -    - meson setup build --vsenv -Dperl=disabled -Dbackend_max_links=1 -Dcredential_helpers=wincred
> +    - meson setup build --vsenv -Dperl=disabled -Dbackend_max_links=1 -Dcredential_helpers=wincred -Dtest_output_directory="$TEST_OUTPUT_DIRECTORY"

Now we set the test output directory build option accordingly.

>      - meson compile -C build
>    artifacts:
>      paths:
> @@ -185,10 +188,19 @@ test:msvc-meson:
>    script:
>      - |
>        & "C:/Program Files/Git/usr/bin/bash.exe" -l -c 'ci/run-test-slice-meson.sh build $CI_NODE_INDEX $CI_NODE_TOTAL'
> +  after_script:
> +    - |
> +      if ($env:CI_JOB_STATUS -ne "success") {
> +        & "C:/Program Files/Git/usr/bin/bash.exe" -l -c 'ci/print-test-failures.sh'
> +        Move-Item -Path "$env:TEST_OUTPUT_DIRECTORY/failed-test-artifacts" -Destination t/
> +      }

Here we print any failures and move them so they are stored as a CI
artifact.

>    parallel: 10
>    artifacts:
> +    paths:
> +      - t/failed-test-artifacts
>      reports:
>        junit: build/meson-logs/testlog.junit.xml
> +    when: on_failure

This patch also looks good.

-Justin