[PATCH 0/5] gitlab-ci: some fixes for failures on Windows

[PATCH 0/5] gitlab-ci: some fixes for failures on Windows

[Patrick Steinhardt]

Hi,

this small patch series fixes some intermittent failures we have
observed with our Windows CI runners. Furthermore, it fixes one
persistent test failure in one of the Windows runners that was
introduced recently via tests for git-last-modified(1).

The pipeline at [1] shows that tests succeed now.

Thanks!

Patrick

[1]: https://gitlab.com/gitlab-org/git/-/merge_requests/426

---
Patrick Steinhardt (5):
      gitlab-ci: dedup instructions to disable realtime monitoring
      gitlab-ci: ignore failures to disable realtime monitoring
      gitlab-ci: drop workaround for Python certificate store on Windows
      gitlab-ci: upload Meson test logs as JUnit reports
      t8020: fix test failure due to indeterministic tag sorting

 .gitlab-ci.yml           | 29 +++++++++++++++++------------
 t/t8020-last-modified.sh | 34 +++++++++++++++++++---------------
 2 files changed, 36 insertions(+), 27 deletions(-)


---
base-commit: 821f583da6d30a84249f75f33501504d597bc16b
change-id: 20251001-pks-gitlab-ci-windows-improvements-16dd3f911688

[PATCH 1/5] gitlab-ci: dedup instructions to disable realtime monitoring

[Patrick Steinhardt]

The instruction to disable realtime monitoring are shared across all of
our Windows-based jobs. Deduplicate it so that we can more readily
iterate on it.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 .gitlab-ci.yml | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index cf122e706f..552c033fb0 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -112,6 +112,9 @@ test:osx:
       - t/failed-test-artifacts
     when: on_failure
 
+.windows_before_script: &windows_before_script
+  - Set-MpPreference -DisableRealtimeMonitoring $true
+
 build:mingw64:
   stage: build
   tags:
@@ -119,7 +122,7 @@ build:mingw64:
   variables:
     NO_PERL: 1
   before_script:
-    - Set-MpPreference -DisableRealtimeMonitoring $true
+    - *windows_before_script
     - ./ci/install-sdk.ps1 -directory "git-sdk"
   script:
     - git-sdk/usr/bin/bash.exe -l -c 'ci/make-test-artifacts.sh artifacts'
@@ -136,7 +139,7 @@ test:mingw64:
     - job: "build:mingw64"
       artifacts: true
   before_script:
-    - Set-MpPreference -DisableRealtimeMonitoring $true
+    - *windows_before_script
     - git-sdk/usr/bin/bash.exe -l -c 'tar xf artifacts/artifacts.tar.gz'
     - New-Item -Path .git/info -ItemType Directory
     - New-Item .git/info/exclude -ItemType File -Value "/git-sdk"
@@ -150,7 +153,7 @@ test:mingw64:
   tags:
     - saas-windows-medium-amd64
   before_script:
-    - Set-MpPreference -DisableRealtimeMonitoring $true
+    - *windows_before_script
     - choco install -y git meson ninja openssl
     - Import-Module $env:ChocolateyInstall\helpers\chocolateyProfile.psm1
     - refreshenv

-- 
2.51.0.700.g236ee7b076.dirty

[PATCH 2/5] gitlab-ci: ignore failures to disable realtime monitoring

[Patrick Steinhardt]

We have recently introduced a change to disable realtime monitoring for
Windows job in GitLab CI. This change led (and still leads) to a quite
significant speedup.

But there's a catch: seemingly, some of the runners we use already have
realtime monitoring disabled. On such a machine, trying to disable the
feature again leads to an error that causes the whole job to fail.

Safeguard against such failures by explicitly ignoring them.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 .gitlab-ci.yml | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 552c033fb0..ed4dc9db94 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -113,7 +113,10 @@ test:osx:
     when: on_failure
 
 .windows_before_script: &windows_before_script
-  - Set-MpPreference -DisableRealtimeMonitoring $true
+  # Disabling realtime monitoring fails on some of the runners, but it
+  # significantly speeds up test execution in the case where it works. We thus
+  # try our luck, but ignore any failures.
+  - Set-MpPreference -DisableRealtimeMonitoring $true; $true
 
 build:mingw64:
   stage: build

-- 
2.51.0.700.g236ee7b076.dirty

[PATCH 3/5] gitlab-ci: drop workaround for Python certificate store on Windows

[Patrick Steinhardt]

On Windows, we have been running into some issues in the past where the
certificate store for Python is broken on the GitLab CI runners using
Windows. The consequence was that we weren't able to establish any SSL
connections via Python, but we need that feature so that we can download
the Meson wraps. The workaround we employed was to import certificates
from the cURL project into the certificate store via OpenSSL.

This is obviously an ugly workaround. But even more importantly, this
workaround fails every time Chocolatey updates its OpenSSL packages. The
problem here is that the old OpenSSL package installer will be removed
immediately once the newer version was published, But the Chocolatey
community repository may not yet have propagated the new version of this
package to all of its caches. The result is that for a couple hours (or
sometimes even one or two days) we always fail to install OpenSSL until
the new version was propagated.

Luckily though, it turns out that the workaround doesn't seem to be
required anymore. Drop it to work around the intermittent failures and
to clean up some now-unneeded legacy cruft.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 .gitlab-ci.yml | 10 +---------
 1 file changed, 1 insertion(+), 9 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index ed4dc9db94..b388154078 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -157,17 +157,9 @@ test:mingw64:
     - saas-windows-medium-amd64
   before_script:
     - *windows_before_script
-    - choco install -y git meson ninja openssl
+    - choco install -y git meson ninja
     - Import-Module $env:ChocolateyInstall\helpers\chocolateyProfile.psm1
     - refreshenv
-    # The certificate store for Python on Windows is broken and fails to fetch
-    # certificates, see https://bugs.python.org/issue36011. This seems to
-    # mostly be an issue with how the GitLab image is set up as it is a
-    # non-issue on GitHub Actions. Work around the issue by importing
-    # cetrificates manually.
-    - Invoke-WebRequest https://curl.haxx.se/ca/cacert.pem -OutFile cacert.pem
-    - openssl pkcs12 -export -nokeys -in cacert.pem -out certs.pfx -passout "pass:"
-    - Import-PfxCertificate -CertStoreLocation Cert:\LocalMachine\Root -FilePath certs.pfx
 
 build:msvc-meson:
   extends: .msvc-meson

-- 
2.51.0.700.g236ee7b076.dirty

Re: [PATCH 3/5] gitlab-ci: drop workaround for Python certificate store on Windows

[Toon Claes]

Patrick Steinhardt <ps@pks.im> writes:

> On Windows, we have been running into some issues in the past where the
> certificate store for Python is broken on the GitLab CI runners using
> Windows. The consequence was that we weren't able to establish any SSL
> connections via Python, but we need that feature so that we can download
> the Meson wraps. The workaround we employed was to import certificates
> from the cURL project into the certificate store via OpenSSL.
>
> This is obviously an ugly workaround. But even more importantly, this
> workaround fails every time Chocolatey updates its OpenSSL packages. The
> problem here is that the old OpenSSL package installer will be removed
> immediately once the newer version was published, But the Chocolatey
> community repository may not yet have propagated the new version of this
> package to all of its caches. The result is that for a couple hours (or
> sometimes even one or two days) we always fail to install OpenSSL until
> the new version was propagated.
>
> Luckily though, it turns out that the workaround doesn't seem to be
> required anymore. Drop it to work around the intermittent failures and
> to clean up some now-unneeded legacy cruft.

I assume this is a result of this change:
https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/gcp/windows-containers/-/merge_requests/72

Makes sense to clean this up.

-- 
Cheers,
Toon

[PATCH 4/5] gitlab-ci: upload Meson test logs as JUnit reports

[Patrick Steinhardt]

When running tests, Meson knows to output both a test log as well as a
JUnit test report that collates results. We don't currently upload these
results in our GitLab CI at all, which makes it hard to see which tests
ran, but also which of our tests may have failed.

Upload these JUnit reports as artifacts to make this information more
accessible. Note that we also do this for some jobs that don't use Meson
and thus don't generate these reports in the first place. GitLab CI
handles missing reports gracefully though, so there is no reason to
special-case those jobs that don't use Meson.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 .gitlab-ci.yml | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index b388154078..85401b34a5 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -70,6 +70,8 @@ test:linux:
   artifacts:
     paths:
       - t/failed-test-artifacts
+    reports:
+      junit: build/meson-logs/testlog.junit.xml
     when: on_failure
 
 test:osx:
@@ -110,6 +112,8 @@ test:osx:
   artifacts:
     paths:
       - t/failed-test-artifacts
+    reports:
+      junit: build/meson-logs/testlog.junit.xml
     when: on_failure
 
 .windows_before_script: &windows_before_script
@@ -181,6 +185,9 @@ test:msvc-meson:
   script:
     - meson test -C build --no-rebuild --print-errorlogs --slice $Env:CI_NODE_INDEX/$Env:CI_NODE_TOTAL
   parallel: 10
+  artifacts:
+    reports:
+      junit: build/meson-logs/testlog.junit.xml
 
 test:fuzz-smoke-tests:
   image: ubuntu:latest

-- 
2.51.0.700.g236ee7b076.dirty

[PATCH 5/5] t8020: fix test failure due to indeterministic tag sorting

[Patrick Steinhardt]

In e6c06e87a2 (last-modified: fix bug when some paths remain unhandled,
2025-09-18), we have fixed a bug where under certain circumstances,
git-last-modified(1) would BUG because there's still some unhandled
paths. The fix claims that the root cause here is criss-cross merges,
and it adds a test case that seemingly exercises this.

Curiously, this test case fails on some systems because the actual
output does not match our expectations:

    diff --git a/expect b/actual
    index 5271607..bdc620e 100644
    --- a/expect
    --- b/actual
    @@ -1,3 +1,3 @@
     km3 a
    -k2 k
    +km2 k
     1 file
    error: last command exited with $?=1
    not ok 15 - last-modified with subdir and criss-cross merge

The output we see is git-name-rev(1) with `--annotate-stdin`. What it
does is to take the output of git-last-modified(1), which contains
object IDs of the blamed commits, and convert those object IDs into the
names of the corresponding tags. Interestingly, we indeed have both "k2"
and "km2" as tags, and even more interestingly both of these tags point
to the same commit. So the output we get isn't _wrong_, as the tags are
ambiguous.

But why do both of these tags point to the same commit? "km2" really is
supposed to be a merge, but due to the way the test is constructed the
merge turns into a fast-forward merge. Which means that the resulting
does not even contain a criss-cross merge in the first place! A quick
test though shows that the test indeed triggers the bug, so the initial
analysis that the behaviour is triggered by such merges must be wrong.

And it is: seemingly, the issue isn't with criss-cross merges, but
rather with a graph where different files in the same directory were
modified on both sides of a merge.

Refactor the test so that we explicitly test for this specific situation
instead of mentioning the "criss-cross merge" red herring. As the test
is very specific to the actual layout of the repository we also adapt it
to use its own standalone repository.

Note that this requires us to drop the `test_when_finished` call in
`check_last_modified` because it's not supported to execute that
function in a subshell.

This refactoring also fixes the original tag ambiguity that caused us to
fail on some platforms.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t8020-last-modified.sh | 34 +++++++++++++++++++---------------
 1 file changed, 19 insertions(+), 15 deletions(-)

diff --git a/t/t8020-last-modified.sh b/t/t8020-last-modified.sh
index e13aad1439..61f00bc15c 100755
--- a/t/t8020-last-modified.sh
+++ b/t/t8020-last-modified.sh
@@ -33,7 +33,6 @@ check_last_modified() {
 	done &&
 
 	cat >expect &&
-	test_when_finished "rm -f tmp.*" &&
 	git ${indir:+-C "$indir"} last-modified "$@" >tmp.1 &&
 	git name-rev --annotate-stdin --name-only --tags \
 		<tmp.1 >tmp.2 &&
@@ -128,20 +127,25 @@ test_expect_success 'only last-modified files in the current tree' '
 	EOF
 '
 
-test_expect_success 'last-modified with subdir and criss-cross merge' '
-	git checkout -b branch-k1 1 &&
-	mkdir -p a k &&
-	test_commit k1 a/file2 &&
-	git checkout -b branch-k2 &&
-	test_commit k2 k/file2 &&
-	git checkout branch-k1 &&
-	test_merge km2 branch-k2 &&
-	test_merge km3 3 &&
-	check_last_modified <<-\EOF
-	km3 a
-	k2 k
-	1 file
-	EOF
+test_expect_success 'subdirectory modified via merge' '
+	test_when_finished rm -rf repo &&
+	git init repo &&
+	(
+		cd repo &&
+		test_commit base &&
+		git switch --create left &&
+		mkdir subdir &&
+		test_commit left subdir/left &&
+		git switch --create right base &&
+		mkdir subdir &&
+		test_commit right subdir/right &&
+		git switch - &&
+		test_merge merge right &&
+		check_last_modified <<-\EOF
+		merge subdir
+		base base.t
+		EOF
+	)
 '
 
 test_expect_success 'cross merge boundaries in blaming' '

-- 
2.51.0.700.g236ee7b076.dirty

Re: [PATCH 5/5] t8020: fix test failure due to indeterministic tag sorting

[Kristoffer Haugsbakk]

On Thu, Oct 2, 2025, at 13:04, Patrick Steinhardt wrote:
> [snip]
>
> But why do both of these tags point to the same commit? "km2" really is
> supposed to be a merge, but due to the way the test is constructed the
> merge turns into a fast-forward merge. Which means that the resulting
> does not even contain a criss-cross merge in the first place! A quick

“the resulting <noun>”

By the way these intro (paragraph) sentences like “But why do” and 
“And it is” really help connect the whole narrative in the message.  
I’ve noticed the same thing on some other recent commits/patches.

> test though shows that the test indeed triggers the bug, so the initial
> analysis that the behaviour is triggered by such merges must be wrong.
>
>[snip]

Re: [PATCH 5/5] t8020: fix test failure due to indeterministic tag sorting

[Patrick Steinhardt]

On Thu, Oct 02, 2025 at 01:10:53PM +0200, Kristoffer Haugsbakk wrote:
> On Thu, Oct 2, 2025, at 13:04, Patrick Steinhardt wrote:
> > [snip]
> >
> > But why do both of these tags point to the same commit? "km2" really is
> > supposed to be a merge, but due to the way the test is constructed the
> > merge turns into a fast-forward merge. Which means that the resulting
> > does not even contain a criss-cross merge in the first place! A quick
> 
> “the resulting <noun>”

Ah, right. What's missing here is "commit graph". Fixed locally, will
send out with the next iteration.

Thanks!

Patrick

Re: [PATCH 5/5] t8020: fix test failure due to indeterministic tag sorting

[Toon Claes]

Patrick Steinhardt <ps@pks.im> writes:

> In e6c06e87a2 (last-modified: fix bug when some paths remain unhandled,
> 2025-09-18), we have fixed a bug where under certain circumstances,
> git-last-modified(1) would BUG because there's still some unhandled
> paths. The fix claims that the root cause here is criss-cross merges,
> and it adds a test case that seemingly exercises this.
>
> Curiously, this test case fails on some systems because the actual
> output does not match our expectations:
>
>     diff --git a/expect b/actual
>     index 5271607..bdc620e 100644
>     --- a/expect
>     --- b/actual
>     @@ -1,3 +1,3 @@
>      km3 a
>     -k2 k
>     +km2 k
>      1 file
>     error: last command exited with $?=1
>     not ok 15 - last-modified with subdir and criss-cross merge
>
> The output we see is git-name-rev(1) with `--annotate-stdin`. What it
> does is to take the output of git-last-modified(1), which contains
> object IDs of the blamed commits, and convert those object IDs into the
> names of the corresponding tags. Interestingly, we indeed have both "k2"
> and "km2" as tags, and even more interestingly both of these tags point
> to the same commit. So the output we get isn't _wrong_, as the tags are
> ambiguous.

Ahha, I see...

> But why do both of these tags point to the same commit? "km2" really is
> supposed to be a merge, but due to the way the test is constructed the
> merge turns into a fast-forward merge.

Ohw :o

> Which means that the resulting
> does not even contain a criss-cross merge in the first place! A quick
> test though shows that the test indeed triggers the bug, so the initial
> analysis that the behaviour is triggered by such merges must be wrong.

I've tried various things, this felt like the simplest case to
reproduce. Apparantly not realizing it was wrong.

> And it is: seemingly, the issue isn't with criss-cross merges, but
> rather with a graph where different files in the same directory were
> modified on both sides of a merge.

Well yes, it felt like it was something like this. But like said, it
_seemed_ I needed a criss-cross merge.

> Refactor the test so that we explicitly test for this specific situation
> instead of mentioning the "criss-cross merge" red herring.

Well, thanks for this cleanup!

> As the test
> is very specific to the actual layout of the repository we also adapt it
> to use its own standalone repository.

I should have done that in the first place. Trying to shoe horn this
test in the existing repo might have guided me to take incorrect
conclusions.
> Note that this requires us to drop the `test_when_finished` call in
> `check_last_modified` because it's not supported to execute that
> function in a subshell.

This is not an issue because these files are always overwritten before
comparing, so this the `rm` wasn't strictly needed.

> This refactoring also fixes the original tag ambiguity that caused us to
> fail on some platforms.

Awesome! 

> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  t/t8020-last-modified.sh | 34 +++++++++++++++++++---------------
>  1 file changed, 19 insertions(+), 15 deletions(-)
>
> diff --git a/t/t8020-last-modified.sh b/t/t8020-last-modified.sh
> index e13aad1439..61f00bc15c 100755
> --- a/t/t8020-last-modified.sh
> +++ b/t/t8020-last-modified.sh
> @@ -33,7 +33,6 @@ check_last_modified() {
>  	done &&
>  
>  	cat >expect &&
> -	test_when_finished "rm -f tmp.*" &&
>  	git ${indir:+-C "$indir"} last-modified "$@" >tmp.1 &&
>  	git name-rev --annotate-stdin --name-only --tags \
>  		<tmp.1 >tmp.2 &&
> @@ -128,20 +127,25 @@ test_expect_success 'only last-modified files in the current tree' '
>  	EOF
>  '
>  
> -test_expect_success 'last-modified with subdir and criss-cross merge' '
> -	git checkout -b branch-k1 1 &&
> -	mkdir -p a k &&
> -	test_commit k1 a/file2 &&
> -	git checkout -b branch-k2 &&
> -	test_commit k2 k/file2 &&
> -	git checkout branch-k1 &&
> -	test_merge km2 branch-k2 &&
> -	test_merge km3 3 &&
> -	check_last_modified <<-\EOF
> -	km3 a
> -	k2 k
> -	1 file
> -	EOF
> +test_expect_success 'subdirectory modified via merge' '
> +	test_when_finished rm -rf repo &&
> +	git init repo &&
> +	(
> +		cd repo &&
> +		test_commit base &&
> +		git switch --create left &&
> +		mkdir subdir &&
> +		test_commit left subdir/left &&
> +		git switch --create right base &&
> +		mkdir subdir &&
> +		test_commit right subdir/right &&
> +		git switch - &&
> +		test_merge merge right &&
> +		check_last_modified <<-\EOF
> +		merge subdir
> +		base base.t
> +		EOF
> +	)
>  '

Looks good to me.

-- 
Cheers,
Toon