[PATCH 0/3] testing/next: s390x gitlab updates

[PATCH 0/3] testing/next: s390x gitlab updates

[Alex Bennée]

I was asked to update the custom gitlab runner from the aging 20.04 to
22.04 which has been done. However I needed to update the provisioning
scripts and clean-up some of the cruft. Sadly this doesn't seem to be
passing cleanly as we have:

  - qtest-s390x/migration-test ERROR           98.94s   killed by signal 6 SIGABRT
  - failing TCG tests (on s390x HW)
     - float_convd fails against a generated reference
     - clc returns 1

There are also lots of scary DSO warnings:

  BFD: warning: system-supplied DSO at 0x7fedb6479000 has a corrupt string table index

Can I leave that to the respective maintainers to investigate please
as I have a publishing deadline coming up.

Thanks,

Alex.

Alex Bennée (3):
  build-environment: make some packages optional
  gitlab: migrate the s390x custom machine to 22.04
  gitlab: remove stale s390x-all-linux-static conf hacks

 .gitlab-ci.d/custom-runners.yml               |  2 +-
 ...20.04-s390x.yml => ubuntu-22.04-s390x.yml} | 32 +++++++++----------
 scripts/ci/setup/build-environment.yml        | 16 ++++++++--
 3 files changed, 29 insertions(+), 21 deletions(-)
 rename .gitlab-ci.d/custom-runners/{ubuntu-20.04-s390x.yml => ubuntu-22.04-s390x.yml} (80%)

-- 
2.39.2

[PATCH 1/3] build-environment: make some packages optional

[Alex Bennée]

Upgrading the s390x runner exposed some packages are not available for
it. Add an additional optional stage we only enable for arm64/x86_64
for now.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 scripts/ci/setup/build-environment.yml | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/scripts/ci/setup/build-environment.yml b/scripts/ci/setup/build-environment.yml
index f344d1a850..de0d866a1e 100644
--- a/scripts/ci/setup/build-environment.yml
+++ b/scripts/ci/setup/build-environment.yml
@@ -95,7 +95,6 @@
           - libpam0g-dev
           - libpcre2-dev
           - libpixman-1-dev
-          - libpmem-dev
           - libpng-dev
           - libpulse-dev
           - librbd-dev
@@ -107,7 +106,6 @@
           - libslirp-dev
           - libsnappy-dev
           - libspice-protocol-dev
-          - libspice-server-dev
           - libssh-dev
           - libsystemd-dev
           - libtasn1-6-dev
@@ -119,7 +117,6 @@
           - libvdeplug-dev
           - libvirglrenderer-dev
           - libvte-2.91-dev
-          - libxen-dev
           - libxml2-dev
           - libzstd-dev
           - llvm
@@ -156,6 +153,19 @@
         - ansible_facts['distribution'] == 'Ubuntu'
         - ansible_facts['distribution_version'] == '22.04'
 
+    # not all packages are available for all architectures
+    - name: Install additional packages to build QEMU on Ubuntu 22.04
+      package:
+        name:
+          - libpmem-dev
+          - libspice-server-dev
+          - libxen-dev
+        state: present
+      when:
+        - ansible_facts['distribution'] == 'Ubuntu'
+        - ansible_facts['distribution_version'] == '22.04'
+        - ansible_facts['architecture'] == 'aarch64' or ansible_facts['architecture'] == 'x86_64'
+
     - name: Install armhf cross-compile packages to build QEMU on AArch64 Ubuntu 22.04
       package:
         name:
-- 
2.39.2

[PATCH 2/3] gitlab: migrate the s390x custom machine to 22.04

[Alex Bennée]

20.04 is dead (from QEMU's point of view), long live 22.04!

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 .gitlab-ci.d/custom-runners.yml               |  2 +-
 ...20.04-s390x.yml => ubuntu-22.04-s390x.yml} | 28 +++++++++----------
 2 files changed, 15 insertions(+), 15 deletions(-)
 rename .gitlab-ci.d/custom-runners/{ubuntu-20.04-s390x.yml => ubuntu-22.04-s390x.yml} (88%)

diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
index a0e79acd39..29e52df283 100644
--- a/.gitlab-ci.d/custom-runners.yml
+++ b/.gitlab-ci.d/custom-runners.yml
@@ -29,7 +29,7 @@
       junit: build/meson-logs/testlog.junit.xml
 
 include:
-  - local: '/.gitlab-ci.d/custom-runners/ubuntu-20.04-s390x.yml'
+  - local: '/.gitlab-ci.d/custom-runners/ubuntu-22.04-s390x.yml'
   - local: '/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch64.yml'
   - local: '/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch32.yml'
   - local: '/.gitlab-ci.d/custom-runners/centos-stream-8-x86_64.yml'
diff --git a/.gitlab-ci.d/custom-runners/ubuntu-20.04-s390x.yml b/.gitlab-ci.d/custom-runners/ubuntu-22.04-s390x.yml
similarity index 88%
rename from .gitlab-ci.d/custom-runners/ubuntu-20.04-s390x.yml
rename to .gitlab-ci.d/custom-runners/ubuntu-22.04-s390x.yml
index cdae6c5212..85e2809573 100644
--- a/.gitlab-ci.d/custom-runners/ubuntu-20.04-s390x.yml
+++ b/.gitlab-ci.d/custom-runners/ubuntu-22.04-s390x.yml
@@ -1,13 +1,13 @@
-# All ubuntu-20.04 jobs should run successfully in an environment
+# All ubuntu-22.04 jobs should run successfully in an environment
 # setup by the scripts/ci/setup/build-environment.yml task
-# "Install basic packages to build QEMU on Ubuntu 20.04/20.04"
+# "Install basic packages to build QEMU on Ubuntu 22.04"
 
-ubuntu-20.04-s390x-all-linux-static:
+ubuntu-22.04-s390x-all-linux-static:
  extends: .custom_runner_template
  needs: []
  stage: build
  tags:
- - ubuntu_20.04
+ - ubuntu_22.04
  - s390x
  rules:
  - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
@@ -23,12 +23,12 @@ ubuntu-20.04-s390x-all-linux-static:
  - make --output-sync check-tcg
  - make --output-sync -j`nproc` check
 
-ubuntu-20.04-s390x-all:
+ubuntu-22.04-s390x-all:
  extends: .custom_runner_template
  needs: []
  stage: build
  tags:
- - ubuntu_20.04
+ - ubuntu_22.04
  - s390x
  timeout: 75m
  rules:
@@ -42,12 +42,12 @@ ubuntu-20.04-s390x-all:
  - make --output-sync -j`nproc`
  - make --output-sync -j`nproc` check
 
-ubuntu-20.04-s390x-alldbg:
+ubuntu-22.04-s390x-alldbg:
  extends: .custom_runner_template
  needs: []
  stage: build
  tags:
- - ubuntu_20.04
+ - ubuntu_22.04
  - s390x
  rules:
  - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
@@ -65,12 +65,12 @@ ubuntu-20.04-s390x-alldbg:
  - make --output-sync -j`nproc`
  - make --output-sync -j`nproc` check
 
-ubuntu-20.04-s390x-clang:
+ubuntu-22.04-s390x-clang:
  extends: .custom_runner_template
  needs: []
  stage: build
  tags:
- - ubuntu_20.04
+ - ubuntu_22.04
  - s390x
  rules:
  - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
@@ -87,11 +87,11 @@ ubuntu-20.04-s390x-clang:
  - make --output-sync -j`nproc`
  - make --output-sync -j`nproc` check
 
-ubuntu-20.04-s390x-tci:
+ubuntu-22.04-s390x-tci:
  needs: []
  stage: build
  tags:
- - ubuntu_20.04
+ - ubuntu_22.04
  - s390x
  rules:
  - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
@@ -107,12 +107,12 @@ ubuntu-20.04-s390x-tci:
    || { cat config.log meson-logs/meson-log.txt; exit 1; }
  - make --output-sync -j`nproc`
 
-ubuntu-20.04-s390x-notcg:
+ubuntu-22.04-s390x-notcg:
  extends: .custom_runner_template
  needs: []
  stage: build
  tags:
- - ubuntu_20.04
+ - ubuntu_22.04
  - s390x
  rules:
  - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
-- 
2.39.2

[PATCH 3/3] gitlab: remove stale s390x-all-linux-static conf hacks

[Alex Bennée]

The libssh bug references 18.04 which we are no longer running. We
don't need to disable glusterfs because a linux-user build shouldn't
be trying to link to it anyway.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 .gitlab-ci.d/custom-runners/ubuntu-22.04-s390x.yml | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/.gitlab-ci.d/custom-runners/ubuntu-22.04-s390x.yml b/.gitlab-ci.d/custom-runners/ubuntu-22.04-s390x.yml
index 85e2809573..105981879f 100644
--- a/.gitlab-ci.d/custom-runners/ubuntu-22.04-s390x.yml
+++ b/.gitlab-ci.d/custom-runners/ubuntu-22.04-s390x.yml
@@ -13,11 +13,9 @@ ubuntu-22.04-s390x-all-linux-static:
  - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
  - if: "$S390X_RUNNER_AVAILABLE"
  script:
- # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763
- # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
  - mkdir build
  - cd build
- - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh
+ - ../configure --enable-debug --static --disable-system
    || { cat config.log meson-logs/meson-log.txt; exit 1; }
  - make --output-sync -j`nproc`
  - make --output-sync check-tcg
-- 
2.39.2

Re: [PATCH 1/3] build-environment: make some packages optional

[Thomas Huth]

On 26/04/2024 17.39, Alex Bennée wrote:
> Upgrading the s390x runner exposed some packages are not available for
> it. Add an additional optional stage we only enable for arm64/x86_64
> for now.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>   scripts/ci/setup/build-environment.yml | 16 +++++++++++++---
>   1 file changed, 13 insertions(+), 3 deletions(-)
> 
> diff --git a/scripts/ci/setup/build-environment.yml b/scripts/ci/setup/build-environment.yml
> index f344d1a850..de0d866a1e 100644
> --- a/scripts/ci/setup/build-environment.yml
> +++ b/scripts/ci/setup/build-environment.yml
> @@ -95,7 +95,6 @@
>             - libpam0g-dev
>             - libpcre2-dev
>             - libpixman-1-dev
> -          - libpmem-dev
>             - libpng-dev
>             - libpulse-dev
>             - librbd-dev
> @@ -107,7 +106,6 @@
>             - libslirp-dev
>             - libsnappy-dev
>             - libspice-protocol-dev
> -          - libspice-server-dev
>             - libssh-dev
>             - libsystemd-dev
>             - libtasn1-6-dev
> @@ -119,7 +117,6 @@
>             - libvdeplug-dev
>             - libvirglrenderer-dev
>             - libvte-2.91-dev
> -          - libxen-dev
>             - libxml2-dev
>             - libzstd-dev
>             - llvm
> @@ -156,6 +153,19 @@
>           - ansible_facts['distribution'] == 'Ubuntu'
>           - ansible_facts['distribution_version'] == '22.04'
>   
> +    # not all packages are available for all architectures
> +    - name: Install additional packages to build QEMU on Ubuntu 22.04
> +      package:
> +        name:
> +          - libpmem-dev
> +          - libspice-server-dev
> +          - libxen-dev
> +        state: present
> +      when:
> +        - ansible_facts['distribution'] == 'Ubuntu'
> +        - ansible_facts['distribution_version'] == '22.04'
> +        - ansible_facts['architecture'] == 'aarch64' or ansible_facts['architecture'] == 'x86_64'
> +
>       - name: Install armhf cross-compile packages to build QEMU on AArch64 Ubuntu 22.04
>         package:
>           name:

Reviewed-by: Thomas Huth <thuth@redhat.com>

Re: [PATCH 2/3] gitlab: migrate the s390x custom machine to 22.04

[Thomas Huth]

On 26/04/2024 17.39, Alex Bennée wrote:
> 20.04 is dead (from QEMU's point of view), long live 22.04!
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>   .gitlab-ci.d/custom-runners.yml               |  2 +-
>   ...20.04-s390x.yml => ubuntu-22.04-s390x.yml} | 28 +++++++++----------
>   2 files changed, 15 insertions(+), 15 deletions(-)
>   rename .gitlab-ci.d/custom-runners/{ubuntu-20.04-s390x.yml => ubuntu-22.04-s390x.yml} (88%)

Reviewed-by: Thomas Huth <thuth@redhat.com>

Re: [PATCH 3/3] gitlab: remove stale s390x-all-linux-static conf hacks

[Thomas Huth]

On 26/04/2024 17.39, Alex Bennée wrote:
> The libssh bug references 18.04 which we are no longer running. We
> don't need to disable glusterfs because a linux-user build shouldn't
> be trying to link to it anyway.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>   .gitlab-ci.d/custom-runners/ubuntu-22.04-s390x.yml | 4 +---
>   1 file changed, 1 insertion(+), 3 deletions(-)

Reviewed-by: Thomas Huth <thuth@redhat.com>

Re: [PATCH 0/3] testing/next: s390x gitlab updates

[Richard Henderson]

On 4/26/24 08:39, Alex Bennée wrote:
> I was asked to update the custom gitlab runner from the aging 20.04 to
> 22.04 which has been done. However I needed to update the provisioning
> scripts and clean-up some of the cruft. Sadly this doesn't seem to be
> passing cleanly as we have:
> 
>    - qtest-s390x/migration-test ERROR           98.94s   killed by signal 6 SIGABRT
>    - failing TCG tests (on s390x HW)
>       - float_convd fails against a generated reference
>       - clc returns 1

I've had a look at the clc failure.

It fails because of bad address space layout, where the NULL page isn't unmapped, so the 
expected SIGSEGV does *not* happen.

This is unfortunate and we could do better.

However, with the upgrade, --static --enable-pie no longer works.
 From config.log,

cc -m64 -Werror -fPIE -DPIE -o config-temp/qemu-conf.exe config-temp/qemu-conf.c -static-pie
/usr/bin/ld: cannot find rcrt1.o: No such file or directory

which suggests a missing package.

Alternately, we could drop --static, as it's not really relevant to this testing.  With 
that, we get PIE dynamically linked executables, which do not trigger the bad layout.

But at some point we should make sure that the NULL page(s) are reserved PROT_NONE for the 
guest, which ensure this stays fixed.


r~