qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: "Daniel P. Berrangé" <berrange@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Philippe Mathieu-Daudé" <philmd@linaro.org>,
	"Beraldo Leal" <bleal@redhat.com>,
	"Alex Bennée" <alex.bennee@linaro.org>,
	"Wainer dos Santos Moschetta" <wainersm@redhat.com>,
	"Yonggang Luo" <luoyonggang@gmail.com>,
	"Paolo Bonzini" <pbonzini@redhat.com>,
	"Thomas Huth" <thuth@redhat.com>,
	"Daniel P. Berrangé" <berrange@redhat.com>
Subject: [PATCH 3/8] gitlab: always use updated msys installer
Date: Tue,  1 Aug 2023 14:03:58 +0100	[thread overview]
Message-ID: <20230801130403.164060-4-berrange@redhat.com> (raw)
In-Reply-To: <20230801130403.164060-1-berrange@redhat.com>

We current reference an msys installer binary from mid-2022, which means
after installation, it immediately has to re-download a bunch of newer
content. This wastes precious CI time.

The msys project publishes an installer binary with a fixed URL that
always references the latest content. We cache the downloads in gitlab
though and so once downloaded we would never re-fetch the installer
leading back to the same problem.

To deal with this we also fetch the pgp signature for the installer
on every run, and compare that to the previously cached signature. If
the signature changes, we re-download the full installer.

This ensures we always have the latest installer for msys, while also
maximising use of the gitlab cache.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 .gitlab-ci.d/windows.yml | 30 +++++++++++++++++++++++++++---
 1 file changed, 27 insertions(+), 3 deletions(-)

diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
index 831b080d12..0bc04ad068 100644
--- a/.gitlab-ci.d/windows.yml
+++ b/.gitlab-ci.d/windows.yml
@@ -23,10 +23,34 @@
   - If ( !(Test-Path -Path msys64\var\cache ) ) {
       mkdir msys64\var\cache
     }
-  - If ( !(Test-Path -Path msys64\var\cache\msys2.exe ) ) {
+  - Invoke-WebRequest
+    "https://repo.msys2.org/distrib/msys2-x86_64-latest.sfx.exe.sig"
+    -outfile "msys2.exe.sig"
+  - if ( Test-Path -Path msys64\var\cache\msys2.exe.sig ) {
+      Write-Output "Cached installer sig" ;
+      if ( ((Get-FileHash msys2.exe.sig).Hash -ne (Get-FileHash msys64\var\cache\msys2.exe.sig).Hash) ) {
+        Write-Output "Mis-matched installer sig, new installer download required" ;
+        Remove-Item -Path msys64\var\cache\msys2.exe.sig ;
+        if ( Test-Path -Path msys64\var\cache\msys2.exe ) {
+          Remove-Item -Path msys64\var\cache\msys2.exe
+        }
+      } else {
+        Write-Output "Matched installer sig, cached installer still valid"
+      }
+    } else {
+      Write-Output "No cached installer sig, new installer download required" ;
+      if ( Test-Path -Path msys64\var\cache\msys2.exe ) {
+        Remove-Item -Path msys64\var\cache\msys2.exe
+      }
+    }
+  - if ( !(Test-Path -Path msys64\var\cache\msys2.exe ) ) {
+      Write-Output "Fetching latest installer" ;
       Invoke-WebRequest
-      "https://github.com/msys2/msys2-installer/releases/download/2022-06-03/msys2-base-x86_64-20220603.sfx.exe"
-      -outfile "msys64\var\cache\msys2.exe"
+      "https://repo.msys2.org/distrib/msys2-x86_64-latest.sfx.exe"
+      -outfile "msys64\var\cache\msys2.exe" ;
+      Copy-Item -Path msys2.exe.sig -Destination msys64\var\cache\msys2.exe.sig
+    } else {
+      Write-Output "Using cached installer"
     }
   - Write-Output "Invoking msys2.exe installer at $(Get-Date -Format u)"
   - msys64\var\cache\msys2.exe -y
-- 
2.41.0



  parent reply	other threads:[~2023-08-01 13:04 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-08-01 13:03 [PATCH 0/8] gitlab: speed up msys windows jobs with GCC Daniel P. Berrangé
2023-08-01 13:03 ` [PATCH 1/8] gitlab: remove duplication between msys jobs Daniel P. Berrangé
2023-08-01 18:36   ` Thomas Huth
2023-08-01 13:03 ` [PATCH 2/8] gitlab: print timestamps during windows " Daniel P. Berrangé
2023-08-01 18:38   ` Thomas Huth
2023-08-01 13:03 ` Daniel P. Berrangé [this message]
2023-08-02 15:49   ` [PATCH 3/8] gitlab: always use updated msys installer Thomas Huth
2023-08-03 11:16     ` Daniel P. Berrangé
2023-08-01 13:03 ` [PATCH 4/8] gitlab: drop $CI_PROJECT_DIR from cache path Daniel P. Berrangé
2023-08-02 15:50   ` Thomas Huth
2023-08-01 13:04 ` [PATCH 5/8] gitlab: always populate cache for windows msys jobs Daniel P. Berrangé
2023-08-02 15:50   ` Thomas Huth
2023-08-01 13:04 ` [PATCH 6/8] configure: support passthrough of -Dxxx args to meson Daniel P. Berrangé
2023-08-01 18:42   ` Thomas Huth
2023-08-01 19:07     ` Daniel P. Berrangé
2023-08-01 13:04 ` [PATCH 7/8] gitlab: disable optimization and debug symbols in msys build Daniel P. Berrangé
2023-08-01 18:44   ` Thomas Huth
2023-08-01 19:07     ` Daniel P. Berrangé
2023-08-01 13:04 ` [PATCH 8/8] gitlab: disable FF_SCRIPT_SECTIONS on msys jobs Daniel P. Berrangé
2023-08-01 18:44   ` Thomas Huth
2023-08-01 13:53 ` [PATCH 0/8] gitlab: speed up msys windows jobs with GCC Markus Armbruster
2023-08-01 14:35   ` Daniel P. Berrangé
2023-08-01 18:21     ` Thomas Huth
2023-08-01 18:46 ` Thomas Huth

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230801130403.164060-4-berrange@redhat.com \
    --to=berrange@redhat.com \
    --cc=alex.bennee@linaro.org \
    --cc=bleal@redhat.com \
    --cc=luoyonggang@gmail.com \
    --cc=pbonzini@redhat.com \
    --cc=philmd@linaro.org \
    --cc=qemu-devel@nongnu.org \
    --cc=thuth@redhat.com \
    --cc=wainersm@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).