[PATCH] gitlab-ci: migrate Windows builds away from Chocolatey

[PATCH] gitlab-ci: migrate Windows builds away from Chocolatey

[Patrick Steinhardt]

The Windows builds in GitLab CI use Chocolatey to install dependencies.
Unfortunately, Chocolatey seems to be very unreliable, which causes the
jobs to fail very regularly. This is a limitation that seems to be
somewhat known [1]:

  As an organization, you want 100% reliability (or at least that
  potential), and you may want full trust and control as well. This is
  something you can get with internally hosted packages, and you are
  unlikely to achieve from use of the Community Package Repository.

So using the Community Package Repository is kind of discouraged in case
one wants reliability. We _do_ want reliability though, and we cannot
easily switch to an enterprise license to fix this issue.

Introduce a new script that downloads and installs dependencies
directly. This has a couple of benefits:

  - We can drop our dependency on Chocolatey completely, thus improving
    reliability.

  - We can easily cache the installers.

  - We get direct control over the exact versions we install.

  - Installing dependencies is sped up from roundabout 3 minutes to 1
    minute.

[1]: https://docs.chocolatey.org/en-us/community-repository/community-packages-disclaimer/#summary

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
Hi

I've been quite annoyed recently because our Windows builds in GitLab CI
are extremely flakey. All of those flakes come from Chocolatey, which is
why this patch moves away from it.

Thanks!

Patrick
---
 .gitlab-ci.yml              | 11 ++++++---
 ci/install-dependencies.ps1 | 55 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 63 insertions(+), 3 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index e0b9a0d82b..87a5343a94 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -161,11 +161,16 @@ test:mingw64:
     TEST_OUTPUT_DIRECTORY: "C:/Git-Test"
   tags:
     - saas-windows-medium-amd64
+  cache:
+    key:
+      files:
+        - ci/install-dependencies.ps1
+    paths:
+      - .dependencies
   before_script:
     - *windows_before_script
-    - choco install -y git meson ninja rust-ms
-    - Import-Module $env:ChocolateyInstall\helpers\chocolateyProfile.psm1
-    - refreshenv
+    - ./ci/install-dependencies.ps1
+    - $env:Path = "C:\Meson;C:\Rust\bin;$env:Path"
     - New-Item -Path $env:TEST_OUTPUT_DIRECTORY -ItemType Directory
 
 build:msvc-meson:
diff --git a/ci/install-dependencies.ps1 b/ci/install-dependencies.ps1
new file mode 100755
index 0000000000..e3b367fa54
--- /dev/null
+++ b/ci/install-dependencies.ps1
@@ -0,0 +1,55 @@
+param(
+    [string]$DownloadDirectory = '.dependencies'
+)
+
+$ErrorActionPreference = 'Stop'
+$ProgressPreference = 'SilentlyContinue'
+
+$GitVersion = '2.54.0.windows.1'
+$MesonVersion = '1.11.0'
+$RustVersion = '1.96.0'
+
+New-Item -Path $DownloadDirectory -ItemType Directory -Force | Out-Null
+New-Item -Path .git/info -ItemType Directory -Force | Out-Null
+New-Item -Path .git/info/exclude -ItemType File -Force | Out-Null
+Add-Content -Path .git/info/exclude -Value "/$DownloadDirectory"
+
+function Get-Installer {
+    param(
+        [Parameter(Mandatory = $true)][string]$Name,
+        [Parameter(Mandatory = $true)][string]$Url
+    )
+
+    $path = Join-Path $DownloadDirectory $Name
+    if (-not (Test-Path $path)) {
+        Write-Host "Downloading $Url"
+        Invoke-WebRequest $Url -OutFile $path -TimeoutSec 300
+    }
+    return $path
+}
+
+function Invoke-Installer {
+    param(
+        [Parameter(Mandatory = $true)][string]$FilePath,
+        [Parameter(Mandatory = $true)][string[]]$ArgumentList
+    )
+
+    Write-Host "Running $FilePath $($ArgumentList -join ' ')"
+    $process = Start-Process -Wait -PassThru -FilePath $FilePath -ArgumentList $ArgumentList
+    if ($process.ExitCode -ne 0) {
+        throw "$FilePath failed with exit code $($process.ExitCode)"
+    }
+}
+
+$gitAssetVersion = $GitVersion -replace '\.windows\.\d+$', ''
+$gitInstaller = Get-Installer "Git-Installer.exe" `
+    "https://github.com/git-for-windows/git/releases/download/v$GitVersion/PortableGit-$gitAssetVersion-64-bit.7z.exe"
+Invoke-Installer $gitInstaller @('-y', '-o"C:\Program Files\Git"')
+
+$mesonMsi = Get-Installer "meson.msi" `
+    "https://github.com/mesonbuild/meson/releases/download/$MesonVersion/meson-$MesonVersion-64.msi"
+Invoke-Installer msiexec.exe @('/i', $mesonMsi, 'INSTALLDIR=C:\Meson', '/quiet', '/norestart')
+
+$rustMsi = Get-Installer "rust.msi" `
+    "https://static.rust-lang.org/dist/rust-$RustVersion-x86_64-pc-windows-msvc.msi"
+Invoke-Installer msiexec.exe @('/i', $rustMsi, 'INSTALLDIR=C:\Rust', 'ADDLOCAL=Rustc,Cargo,Std', '/quiet', '/norestart')

---
base-commit: ea97ad8d017de0c9037451a78008a0fd60abea0c
change-id: 20260615-b4-pks-gitlab-ci-drop-chocolatey-bfe9d4bb1442

Re: [PATCH] gitlab-ci: migrate Windows builds away from Chocolatey

[Justin Tobler]

On 26/06/15 02:21PM, Patrick Steinhardt wrote:
> The Windows builds in GitLab CI use Chocolatey to install dependencies.
> Unfortunately, Chocolatey seems to be very unreliable, which causes the
> jobs to fail very regularly. This is a limitation that seems to be
> somewhat known [1]:
> 
>   As an organization, you want 100% reliability (or at least that
>   potential), and you may want full trust and control as well. This is
>   something you can get with internally hosted packages, and you are
>   unlikely to achieve from use of the Community Package Repository.
> 
> So using the Community Package Repository is kind of discouraged in case
> one wants reliability. We _do_ want reliability though, and we cannot
> easily switch to an enterprise license to fix this issue.

Make sense.

> Introduce a new script that downloads and installs dependencies
> directly. This has a couple of benefits:
> 
>   - We can drop our dependency on Chocolatey completely, thus improving
>     reliability.
> 
>   - We can easily cache the installers.
> 
>   - We get direct control over the exact versions we install.

Naive question: Do we expect to have to update the pinned versions
often?

>   - Installing dependencies is sped up from roundabout 3 minutes to 1
>     minute.

Is fetching the dependencides directly just plain faster? Or is this due
to the caching?

> [1]: https://docs.chocolatey.org/en-us/community-repository/community-packages-disclaimer/#summary
> 
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
> Hi
> 
> I've been quite annoyed recently because our Windows builds in GitLab CI
> are extremely flakey. All of those flakes come from Chocolatey, which is
> why this patch moves away from it.
> 
> Thanks!
> 
> Patrick
> ---
>  .gitlab-ci.yml              | 11 ++++++---
>  ci/install-dependencies.ps1 | 55 +++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 63 insertions(+), 3 deletions(-)
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index e0b9a0d82b..87a5343a94 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -161,11 +161,16 @@ test:mingw64:
>      TEST_OUTPUT_DIRECTORY: "C:/Git-Test"
>    tags:
>      - saas-windows-medium-amd64
> +  cache:
> +    key:
> +      files:
> +        - ci/install-dependencies.ps1
> +    paths:
> +      - .dependencies

Nice that we can cache the installers now.

>    before_script:
>      - *windows_before_script
> -    - choco install -y git meson ninja rust-ms
> -    - Import-Module $env:ChocolateyInstall\helpers\chocolateyProfile.psm1
> -    - refreshenv
> +    - ./ci/install-dependencies.ps1
> +    - $env:Path = "C:\Meson;C:\Rust\bin;$env:Path"

I assume Git is already discoverable on the path?

>      - New-Item -Path $env:TEST_OUTPUT_DIRECTORY -ItemType Directory
>  
>  build:msvc-meson:
> diff --git a/ci/install-dependencies.ps1 b/ci/install-dependencies.ps1
> new file mode 100755
> index 0000000000..e3b367fa54
> --- /dev/null
> +++ b/ci/install-dependencies.ps1
> @@ -0,0 +1,55 @@
> +param(
> +    [string]$DownloadDirectory = '.dependencies'
> +)
> +
> +$ErrorActionPreference = 'Stop'
> +$ProgressPreference = 'SilentlyContinue'
> +
> +$GitVersion = '2.54.0.windows.1'
> +$MesonVersion = '1.11.0'
> +$RustVersion = '1.96.0'
> +
> +New-Item -Path $DownloadDirectory -ItemType Directory -Force | Out-Null
> +New-Item -Path .git/info -ItemType Directory -Force | Out-Null
> +New-Item -Path .git/info/exclude -ItemType File -Force | Out-Null
> +Add-Content -Path .git/info/exclude -Value "/$DownloadDirectory"

Here we create the ".dependencies" directory and add it to
".git/info/exclude" to be ignored.

> +function Get-Installer {
> +    param(
> +        [Parameter(Mandatory = $true)][string]$Name,
> +        [Parameter(Mandatory = $true)][string]$Url
> +    )
> +
> +    $path = Join-Path $DownloadDirectory $Name
> +    if (-not (Test-Path $path)) {
> +        Write-Host "Downloading $Url"
> +        Invoke-WebRequest $Url -OutFile $path -TimeoutSec 300

We only download the installer if it is not already cached. Makes sense.

> +    }
> +    return $path
> +}
> +
> +function Invoke-Installer {
> +    param(
> +        [Parameter(Mandatory = $true)][string]$FilePath,
> +        [Parameter(Mandatory = $true)][string[]]$ArgumentList
> +    )
> +
> +    Write-Host "Running $FilePath $($ArgumentList -join ' ')"
> +    $process = Start-Process -Wait -PassThru -FilePath $FilePath -ArgumentList $ArgumentList
> +    if ($process.ExitCode -ne 0) {
> +        throw "$FilePath failed with exit code $($process.ExitCode)"
> +    }
> +}
> +
> +$gitAssetVersion = $GitVersion -replace '\.windows\.\d+$', ''
> +$gitInstaller = Get-Installer "Git-Installer.exe" `
> +    "https://github.com/git-for-windows/git/releases/download/v$GitVersion/PortableGit-$gitAssetVersion-64-bit.7z.exe"
> +Invoke-Installer $gitInstaller @('-y', '-o"C:\Program Files\Git"')
> +
> +$mesonMsi = Get-Installer "meson.msi" `
> +    "https://github.com/mesonbuild/meson/releases/download/$MesonVersion/meson-$MesonVersion-64.msi"
> +Invoke-Installer msiexec.exe @('/i', $mesonMsi, 'INSTALLDIR=C:\Meson', '/quiet', '/norestart')
> +
> +$rustMsi = Get-Installer "rust.msi" `
> +    "https://static.rust-lang.org/dist/rust-$RustVersion-x86_64-pc-windows-msvc.msi"
> +Invoke-Installer msiexec.exe @('/i', $rustMsi, 'INSTALLDIR=C:\Rust', 'ADDLOCAL=Rustc,Cargo,Std', '/quiet', '/norestart')

Here is actually invoke the helpers to fetch and install the
dependencies. Looks good. I also validated that this job is working on
GitLab CI.

-Justin