[PATCH 0/4] Fixes whpx build and improve Windows host CI

qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed

* [PATCH 0/4] Fixes whpx build and improve Windows host CI
@ 2021-01-07  5:38 Yonggang Luo
  2021-01-07  5:38 ` [PATCH 1/4] cirrus/msys2: Exit powershell with $LastExitCode Yonggang Luo
                   ` (3 more replies)
  0 siblings, 4 replies; 8+ messages in thread
From: Yonggang Luo @ 2021-01-07  5:38 UTC (permalink / raw)
  To: qemu-devel; +Cc: Paolo Bonzini, Sunil Muthuswamy, Ed Maste, Yonggang Luo

Exit powershell with $LastExitCode so that the CI
would report the build/testing failure
Fixes include of whp-dispatch.h
Cache msys2 mingw with a proper fingerprint so
that only when msys2 packages are changed need
trigger the re-populate the cache

Yonggang Luo (4):
  cirrus/msys2: Exit powershell with $LastExitCode
  cirrus/msys2: Cache msys2 mingw in a better way.
  whpx: Fixes include of whp-dispatch.h in whpx.h
  maintainers: Add me as Windows Hosted Continuous Integration
    maintainer

 .cirrus.yml           | 119 +++++++++++++++++++++++++-----------------
 MAINTAINERS           |   6 +++
 include/sysemu/whpx.h |   2 +-
 3 files changed, 77 insertions(+), 50 deletions(-)

-- 
2.29.2.windows.3



^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH 1/4] cirrus/msys2: Exit powershell with $LastExitCode
  2021-01-07  5:38 [PATCH 0/4] Fixes whpx build and improve Windows host CI Yonggang Luo
@ 2021-01-07  5:38 ` Yonggang Luo
  2021-01-07  6:58   ` Philippe Mathieu-Daudé
  2021-01-07  5:38 ` [PATCH 2/4] cirrus/msys2: Cache msys2 mingw in a better way Yonggang Luo
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 8+ messages in thread
From: Yonggang Luo @ 2021-01-07  5:38 UTC (permalink / raw)
  To: qemu-devel; +Cc: Paolo Bonzini, Sunil Muthuswamy, Ed Maste, Yonggang Luo

Currently if we don't exit with $LastExitCode manually,
the cirrus would not report the build/testing failure.

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
---
 .cirrus.yml | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/.cirrus.yml b/.cirrus.yml
index 62a9b57530..ff6adabd0d 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -131,5 +131,7 @@ windows_msys2_task:
     - C:\tools\msys64\usr\bin\bash.exe -lc "mkdir build"
     - C:\tools\msys64\usr\bin\bash.exe -lc "cd build && ../configure --python=python3"
     - C:\tools\msys64\usr\bin\bash.exe -lc "cd build && make -j8"
+    - exit $LastExitCode
   test_script:
     - C:\tools\msys64\usr\bin\bash.exe -lc "cd build && make V=1 check"
+    - exit $LastExitCode
-- 
2.29.2.windows.3



^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [PATCH 1/4] cirrus/msys2: Exit powershell with $LastExitCode
  2021-01-07  5:38 ` [PATCH 1/4] cirrus/msys2: Exit powershell with $LastExitCode Yonggang Luo
@ 2021-01-07  6:58   ` Philippe Mathieu-Daudé
  0 siblings, 0 replies; 8+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-01-07  6:58 UTC (permalink / raw)
  To: Yonggang Luo, qemu-devel; +Cc: Paolo Bonzini, Sunil Muthuswamy, Ed Maste

On 1/7/21 6:38 AM, Yonggang Luo wrote:
> Currently if we don't exit with $LastExitCode manually,
> the cirrus would not report the build/testing failure.
> 
> Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
> ---
>  .cirrus.yml | 2 ++
>  1 file changed, 2 insertions(+)

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH 2/4] cirrus/msys2: Cache msys2 mingw in a better way.
  2021-01-07  5:38 [PATCH 0/4] Fixes whpx build and improve Windows host CI Yonggang Luo
  2021-01-07  5:38 ` [PATCH 1/4] cirrus/msys2: Exit powershell with $LastExitCode Yonggang Luo
@ 2021-01-07  5:38 ` Yonggang Luo
  2021-01-07  5:38 ` [PATCH 3/4] whpx: Fixes include of whp-dispatch.h in whpx.h Yonggang Luo
  2021-01-07  5:38 ` [PATCH 4/4] maintainers: Add me as Windows Hosted Continuous Integration maintainer Yonggang Luo
  3 siblings, 0 replies; 8+ messages in thread
From: Yonggang Luo @ 2021-01-07  5:38 UTC (permalink / raw)
  To: qemu-devel; +Cc: Paolo Bonzini, Sunil Muthuswamy, Ed Maste, Yonggang Luo

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
---
 .cirrus.yml | 117 ++++++++++++++++++++++++++++++----------------------
 1 file changed, 68 insertions(+), 49 deletions(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index ff6adabd0d..6f2a958472 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -58,23 +58,61 @@ windows_msys2_task:
     CIRRUS_SHELL: powershell
     MSYS: winsymlinks:nativestrict
     MSYSTEM: MINGW64
+    MSYS2_URL: https://github.com/msys2/msys2-installer/releases/download/2021-01-05/msys2-base-x86_64-20210105.sfx.exe
+    MSYS2_FINGERPRINT: 0
+    MSYS2_PACKAGES: "
+      diffutils git grep make pkg-config sed
+      mingw-w64-x86_64-python
+      mingw-w64-x86_64-python-sphinx
+      mingw-w64-x86_64-toolchain
+      mingw-w64-x86_64-SDL2
+      mingw-w64-x86_64-SDL2_image
+      mingw-w64-x86_64-gtk3
+      mingw-w64-x86_64-glib2
+      mingw-w64-x86_64-ninja
+      mingw-w64-x86_64-jemalloc
+      mingw-w64-x86_64-lzo2
+      mingw-w64-x86_64-zstd
+      mingw-w64-x86_64-libjpeg-turbo
+      mingw-w64-x86_64-pixman
+      mingw-w64-x86_64-libgcrypt
+      mingw-w64-x86_64-libpng
+      mingw-w64-x86_64-libssh
+      mingw-w64-x86_64-libxml2
+      mingw-w64-x86_64-snappy
+      mingw-w64-x86_64-libusb
+      mingw-w64-x86_64-usbredir
+      mingw-w64-x86_64-libtasn1
+      mingw-w64-x86_64-nettle
+      mingw-w64-x86_64-cyrus-sasl
+      mingw-w64-x86_64-curl
+      mingw-w64-x86_64-gnutls
+      mingw-w64-x86_64-libnfs
+    "
     CHERE_INVOKING: 1
-  setup_script:
-    - choco install -y --no-progress 7zip
-    - Write-Output $env:PATH
   msys2_cache:
     folder: C:\tools\archive
     reupload_on_changes: false
-    fingerprint_script: cat .cirrus.yml
+    # These env variables are used to generate fingerprint to trigger the cache procedure
+    # If wanna to force re-populate msys2, increase MSYS2_FINGERPRINT
+    fingerprint_script:
+      - |
+        echo $env:CIRRUS_TASK_NAME
+        echo $env:MSYS2_URL
+        echo $env:MSYS2_FINGERPRINT
+        echo $env:MSYS2_PACKAGES
     populate_script:
       - |
-        md C:\tools
-        md C:\tools\archive
+        md -Force C:\tools\archive\pkg
         $start_time = Get-Date
+        bitsadmin /transfer msys_download /dynamic /download /priority FOREGROUND $env:MSYS2_URL C:\tools\archive\base.exe
+        Write-Output "Download time taken: $((Get-Date).Subtract($start_time))"
         cd C:\tools
-        bitsadmin /transfer msys_download /dynamic /download /priority FOREGROUND https://github.com/msys2/msys2-installer/releases/download/2020-09-03/msys2-base-x86_64-20200903.sfx.exe C:\tools\base.exe
-        Write-Output "Download time taken: $((Get-Date).Subtract($start_time).Seconds) second(s)"
-        C:\tools\base.exe -y
+        C:\tools\archive\base.exe -y
+        del -Force C:\tools\archive\base.exe
+        Write-Output "Base install time taken: $((Get-Date).Subtract($start_time))"
+        $start_time = Get-Date
+
         ((Get-Content -path C:\tools\msys64\etc\\post-install\\07-pacman-key.post -Raw) -replace '--refresh-keys', '--version') | Set-Content -Path C:\tools\msys64\etc\\post-install\\07-pacman-key.post
         C:\tools\msys64\usr\bin\bash.exe -lc "sed -i 's/^CheckSpace/#CheckSpace/g' /etc/pacman.conf"
         C:\tools\msys64\usr\bin\bash.exe -lc "export"
@@ -84,49 +122,30 @@ windows_msys2_task:
         tasklist
         C:\tools\msys64\usr\bin\bash.exe -lc "mv -f /etc/pacman.conf.pacnew /etc/pacman.conf || true"
         C:\tools\msys64\usr\bin\bash.exe -lc "pacman --noconfirm -Suu --overwrite=*"
-        C:\tools\msys64\usr\bin\bash.exe -lc "pacman --noconfirm -S --needed \
-          diffutils git grep make pkg-config sed \
-          mingw-w64-x86_64-python \
-          mingw-w64-x86_64-toolchain \
-          mingw-w64-x86_64-SDL2 \
-          mingw-w64-x86_64-SDL2_image \
-          mingw-w64-x86_64-gtk3 \
-          mingw-w64-x86_64-glib2 \
-          mingw-w64-x86_64-ninja \
-          mingw-w64-x86_64-jemalloc \
-          mingw-w64-x86_64-lzo2 \
-          mingw-w64-x86_64-zstd \
-          mingw-w64-x86_64-libjpeg-turbo \
-          mingw-w64-x86_64-pixman \
-          mingw-w64-x86_64-libgcrypt \
-          mingw-w64-x86_64-libpng \
-          mingw-w64-x86_64-libssh \
-          mingw-w64-x86_64-libxml2 \
-          mingw-w64-x86_64-snappy \
-          mingw-w64-x86_64-libusb \
-          mingw-w64-x86_64-usbredir \
-          mingw-w64-x86_64-libtasn1 \
-          mingw-w64-x86_64-nettle \
-          mingw-w64-x86_64-cyrus-sasl \
-          mingw-w64-x86_64-curl \
-          mingw-w64-x86_64-gnutls \
-          mingw-w64-x86_64-libnfs \
-          "
-        bitsadmin /transfer msys_download /dynamic /download /priority FOREGROUND `
-          https://repo.msys2.org/mingw/x86_64/mingw-w64-x86_64-python-sphinx-2.3.1-1-any.pkg.tar.xz `
-          C:\tools\mingw-w64-x86_64-python-sphinx-2.3.1-1-any.pkg.tar.xz
-        C:\tools\msys64\usr\bin\bash.exe -lc "pacman --noconfirm -U /c/tools/mingw-w64-x86_64-python-sphinx-2.3.1-1-any.pkg.tar.xz"
-        del C:\tools\mingw-w64-x86_64-python-sphinx-2.3.1-1-any.pkg.tar.xz
-        C:\tools\msys64\usr\bin\bash.exe -lc "rm -rf /var/cache/pacman/pkg/*"
-        cd C:\tools\msys64
-        echo "Start archive"
-        cmd /C "7z a -ttar . -so | 7z a -txz -simsys2-x86_64.tar C:\tools\archive\msys2-x86_64.tar.xz"
+        Write-Output "Core install time taken: $((Get-Date).Subtract($start_time))"
+        $start_time = Get-Date
+
+        C:\tools\msys64\usr\bin\bash.exe -lc "pacman --noconfirm -S --needed $env:MSYS2_PACKAGES"
+        Write-Output "Package install time taken: $((Get-Date).Subtract($start_time))"
+        $start_time = Get-Date
+
+        del -Force -ErrorAction SilentlyContinue C:\tools\msys64\etc\mtab
+        del -Force -ErrorAction SilentlyContinue C:\tools\msys64\dev\fd
+        del -Force -ErrorAction SilentlyContinue C:\tools\msys64\dev\stderr
+        del -Force -ErrorAction SilentlyContinue C:\tools\msys64\dev\stdin
+        del -Force -ErrorAction SilentlyContinue C:\tools\msys64\dev\stdout
+        del -Force -Recurse -ErrorAction SilentlyContinue C:\tools\msys64\var\cache\pacman\pkg
+        tar cf C:\tools\archive\msys64.tar -C C:\tools\ msys64
+
+        Write-Output "Package archive time taken: $((Get-Date).Subtract($start_time))"
+        del -Force -Recurse -ErrorAction SilentlyContinue c:\tools\msys64 
   install_script:
     - |
+      $start_time = Get-Date
       cd C:\tools
-      cmd /C "7z x C:\tools\archive\msys2-x86_64.tar.xz -so | 7z x -aoa -simsys2-x86_64.tar -ttar -omsys64"
-      C:\tools\msys64\usr\bin\bash.exe -lc "export"
-
+      ls C:\tools\archive\msys64.tar
+      tar xf C:\tools\archive\msys64.tar
+      Write-Output "Extract msys2 time taken: $((Get-Date).Subtract($start_time))"
   script:
     - C:\tools\msys64\usr\bin\bash.exe -lc "mkdir build"
     - C:\tools\msys64\usr\bin\bash.exe -lc "cd build && ../configure --python=python3"
-- 
2.29.2.windows.3



^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH 3/4] whpx: Fixes include of whp-dispatch.h in whpx.h
  2021-01-07  5:38 [PATCH 0/4] Fixes whpx build and improve Windows host CI Yonggang Luo
  2021-01-07  5:38 ` [PATCH 1/4] cirrus/msys2: Exit powershell with $LastExitCode Yonggang Luo
  2021-01-07  5:38 ` [PATCH 2/4] cirrus/msys2: Cache msys2 mingw in a better way Yonggang Luo
@ 2021-01-07  5:38 ` Yonggang Luo
  2021-01-07  7:22   ` Philippe Mathieu-Daudé
  2021-01-07  5:38 ` [PATCH 4/4] maintainers: Add me as Windows Hosted Continuous Integration maintainer Yonggang Luo
  3 siblings, 1 reply; 8+ messages in thread
From: Yonggang Luo @ 2021-01-07  5:38 UTC (permalink / raw)
  To: qemu-devel; +Cc: Paolo Bonzini, Sunil Muthuswamy, Ed Maste, Yonggang Luo

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
---
 include/sysemu/whpx.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/sysemu/whpx.h b/include/sysemu/whpx.h
index 9346fd92e9..0e6c9faaf6 100644
--- a/include/sysemu/whpx.h
+++ b/include/sysemu/whpx.h
@@ -15,7 +15,7 @@
 
 #ifdef CONFIG_WHPX
 
-#include "whp-dispatch.h"
+#include "target/i386/whpx/whp-dispatch.h"
 
 struct whpx_state {
     uint64_t mem_quota;
-- 
2.29.2.windows.3



^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [PATCH 3/4] whpx: Fixes include of whp-dispatch.h in whpx.h
  2021-01-07  5:38 ` [PATCH 3/4] whpx: Fixes include of whp-dispatch.h in whpx.h Yonggang Luo
@ 2021-01-07  7:22   ` Philippe Mathieu-Daudé
  0 siblings, 0 replies; 8+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-01-07  7:22 UTC (permalink / raw)
  To: Yonggang Luo, qemu-devel; +Cc: Paolo Bonzini, Sunil Muthuswamy, Ed Maste

On 1/7/21 6:38 AM, Yonggang Luo wrote:
> Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
> ---
>  include/sysemu/whpx.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/include/sysemu/whpx.h b/include/sysemu/whpx.h
> index 9346fd92e9..0e6c9faaf6 100644
> --- a/include/sysemu/whpx.h
> +++ b/include/sysemu/whpx.h
> @@ -15,7 +15,7 @@
>  
>  #ifdef CONFIG_WHPX
>  
> -#include "whp-dispatch.h"
> +#include "target/i386/whpx/whp-dispatch.h"

"sysemu" should not include target/...

$ git grep target/ include/sysemu/ | wc -l
0

Maybe what you want is:

-- >8 --
diff --git a/include/sysemu/whpx.h b/include/sysemu/whpx.h
index 9346fd92e93..4f38784d7e4 100644
--- a/include/sysemu/whpx.h
+++ b/include/sysemu/whpx.h
@@ -15,7 +15,7 @@

 #ifdef CONFIG_WHPX

-#include "whp-dispatch.h"
+#include <WinHvPlatformDefs.h>

 struct whpx_state {
     uint64_t mem_quota;
---



^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH 4/4] maintainers: Add me as Windows Hosted Continuous Integration maintainer
  2021-01-07  5:38 [PATCH 0/4] Fixes whpx build and improve Windows host CI Yonggang Luo
                   ` (2 preceding siblings ...)
  2021-01-07  5:38 ` [PATCH 3/4] whpx: Fixes include of whp-dispatch.h in whpx.h Yonggang Luo
@ 2021-01-07  5:38 ` Yonggang Luo
  2021-01-07  7:20   ` Philippe Mathieu-Daudé
  3 siblings, 1 reply; 8+ messages in thread
From: Yonggang Luo @ 2021-01-07  5:38 UTC (permalink / raw)
  To: qemu-devel; +Cc: Paolo Bonzini, Sunil Muthuswamy, Ed Maste, Yonggang Luo

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=UTF8-8, Size: 616 bytes --]

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
---
 MAINTAINERS | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 4be087b88e..4d9df874a1 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3198,6 +3198,12 @@ S: Maintained
 F: .cirrus.yml
 W: https://cirrus-ci.com/github/qemu/qemu
 
+Windows Hosted Continuous Integration
+M: Yonggang Luo <luoyonggang@gmail.com>
+S: Maintained
+F: .cirrus.yml
+W: https://cirrus-ci.com/github/qemu/qemu
+
 GitLab Continuous Integration
 M: Thomas Huth <thuth@redhat.com>
 M: Philippe Mathieu-Daudé <philmd@redhat.com>
-- 
2.29.2.windows.3



^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [PATCH 4/4] maintainers: Add me as Windows Hosted Continuous Integration maintainer
  2021-01-07  5:38 ` [PATCH 4/4] maintainers: Add me as Windows Hosted Continuous Integration maintainer Yonggang Luo
@ 2021-01-07  7:20   ` Philippe Mathieu-Daudé
  0 siblings, 0 replies; 8+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-01-07  7:20 UTC (permalink / raw)
  To: Yonggang Luo, qemu-devel; +Cc: Paolo Bonzini, Sunil Muthuswamy, Ed Maste

On 1/7/21 6:38 AM, Yonggang Luo wrote:
> Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
> ---
>  MAINTAINERS | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 4be087b88e..4d9df874a1 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -3198,6 +3198,12 @@ S: Maintained
>  F: .cirrus.yml
>  W: https://cirrus-ci.com/github/qemu/qemu
>  
> +Windows Hosted Continuous Integration
> +M: Yonggang Luo <luoyonggang@gmail.com>
> +S: Maintained
> +F: .cirrus.yml
> +W: https://cirrus-ci.com/github/qemu/qemu

Thanks for stepping forward!

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>



^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2021-01-07  7:23 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-01-07  5:38 [PATCH 0/4] Fixes whpx build and improve Windows host CI Yonggang Luo
2021-01-07  5:38 ` [PATCH 1/4] cirrus/msys2: Exit powershell with $LastExitCode Yonggang Luo
2021-01-07  6:58   ` Philippe Mathieu-Daudé
2021-01-07  5:38 ` [PATCH 2/4] cirrus/msys2: Cache msys2 mingw in a better way Yonggang Luo
2021-01-07  5:38 ` [PATCH 3/4] whpx: Fixes include of whp-dispatch.h in whpx.h Yonggang Luo
2021-01-07  7:22   ` Philippe Mathieu-Daudé
2021-01-07  5:38 ` [PATCH 4/4] maintainers: Add me as Windows Hosted Continuous Integration maintainer Yonggang Luo
2021-01-07  7:20   ` Philippe Mathieu-Daudé

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).