[PULL 00/32] testing updates for 10.0 (tuxrun, libvirt, dockerfiles, misc)

[PULL 00/32] testing updates for 10.0 (tuxrun, libvirt, dockerfiles, misc)

[Alex Bennée]

The following changes since commit bc6afa1c711da5b4f37c9685a812c77b114d84cb:

  Merge tag 'pull-xenfv-20250109-1' of https://gitlab.com/dwmw2/qemu into staging (2025-01-09 08:39:32 -0500)

are available in the Git repository at:

  https://gitlab.com/stsquad/qemu.git tags/pull-10.0-testing-updates-100125-1

for you to fetch changes up to 70b7c929ba39f15a94fbc63d9be88e8da907cc46:

  MAINTAINERS: Remove myself from reviewers (2025-01-10 11:18:38 +0000)

----------------------------------------------------------------
Testing updates for 10.0

  - update the tuxrun images to the latest baseline
  - add the m68k tuxrun test
  - ensure qtest checks the result of clock_step operations
  - introduce new ztsd helper to functional tests
  - ensure aarch64_virt test exits early when no TCG
  - add new test to exercise virtio-vulkan
  - bump libvirt-ci to latest version
  - move riscv64 cross container from sid to trixie
  - remove workaround from mips containers now upstream updated
  - fix VM tests to use correct path for local QEMU binary
  - add ability to get a root debug shell on VM images
  - add keymap dependency to vnc tests
  - remove retiring maintainers from avocado and dockerfiles

----------------------------------------------------------------
Alex Bennée (28):
      tests/functional: update the arm tuxrun tests
      tests/functional: update the i386 tuxrun tests
      tests/functional: add a m68k tuxrun tests
      tests/functional: update the mips32 tuxrun tests
      tests/functional: update the mips32el tuxrun tests
      tests/functional: update the mips64 tuxrun tests
      tests/functional: update the mips64el tuxrun tests
      tests/functional: update the ppc32 tuxrun tests
      tests/functional: update the ppc64 tuxrun tests
      tests/functional: update the riscv32 tuxrun tests
      tests/functional: update the riscv64 tuxrun tests
      tests/functional: update the s390x tuxrun tests
      tests/functional: update the sparc64 tuxrun tests
      tests/functional: update the x86_64 tuxrun tests
      tests/qtest: remove clock_steps from virtio tests
      system/qtest: properly feedback results of clock_[step|set]
      tests/functional: remove hacky sleep from the tests
      tests/functional: add zstd support to uncompress utility
      tests/functional: update tuxruntest to use uncompress utility
      tests/functional: remove unused kernel_command_line
      tests/functional: bail aarch64_virt tests early if missing TCG
      tests/functional: extend test_aarch64_virt with vulkan test
      tests/lcitool: bump to latest version of libvirt-ci
      tests/docker: move riscv64 cross container from sid to trixie
      tests/vm: fix build_path based path
      tests/vm: partially un-tabify help output
      tests/vm: allow interactive login as root
      pc-bios: ensure keymaps dependencies set vnc tests

Daniel P. Berrangé (1):
      tests/lcitool: remove temp workaround for debian mips64el

Philippe Mathieu-Daudé (1):
      dockerfiles: Remove 'MAINTAINER' entry in debian-tricore-cross.docker

Pierrick Bouvier (1):
      tests/functional/aarch64: add tests for FEAT_RME

Wainer dos Santos Moschetta (1):
      MAINTAINERS: Remove myself from reviewers

 MAINTAINERS                                        |   3 +-
 system/qtest.c                                     |  23 ++--
 tests/qtest/libqos/virtio.c                        |   4 -
 .gitlab-ci.d/cirrus/freebsd-14.vars                |   2 +-
 pc-bios/keymaps/meson.build                        |  17 ++-
 .../dockerfiles/debian-mips64el-cross.docker       |   9 ++
 .../docker/dockerfiles/debian-riscv64-cross.docker |   4 +-
 .../docker/dockerfiles/debian-tricore-cross.docker |   2 -
 tests/functional/meson.build                       |   5 +
 tests/functional/qemu_test/tuxruntest.py           |  12 +--
 tests/functional/qemu_test/uncompress.py           |  24 +++++
 tests/functional/test_aarch64_rme_sbsaref.py       |  69 ++++++++++++
 tests/functional/test_aarch64_rme_virt.py          |  98 +++++++++++++++++
 tests/functional/test_aarch64_virt.py              | 117 +++++++++++++++++----
 tests/functional/test_arm_tuxrun.py                |  28 ++---
 tests/functional/test_i386_tuxrun.py               |   8 +-
 tests/functional/test_m68k_tuxrun.py               |  34 ++++++
 tests/functional/test_mips64_tuxrun.py             |   8 +-
 tests/functional/test_mips64el_tuxrun.py           |   8 +-
 tests/functional/test_mips_tuxrun.py               |   8 +-
 tests/functional/test_mipsel_tuxrun.py             |   8 +-
 tests/functional/test_ppc64_tuxrun.py              |  16 +--
 tests/functional/test_ppc_tuxrun.py                |   8 +-
 tests/functional/test_riscv32_tuxrun.py            |   8 +-
 tests/functional/test_riscv64_tuxrun.py            |  16 +--
 tests/functional/test_s390x_tuxrun.py              |   8 +-
 tests/functional/test_sparc64_tuxrun.py            |   8 +-
 tests/functional/test_x86_64_tuxrun.py             |   8 +-
 tests/lcitool/libvirt-ci                           |   2 +-
 tests/lcitool/mappings.yml                         |  29 -----
 tests/lcitool/refresh                              |   4 +-
 tests/qtest/meson.build                            |   2 +-
 tests/vm/Makefile.include                          |  29 ++---
 tests/vm/basevm.py                                 |  12 ++-
 tests/vm/generated/freebsd.json                    |   2 +-
 35 files changed, 461 insertions(+), 182 deletions(-)
 create mode 100755 tests/functional/test_aarch64_rme_sbsaref.py
 create mode 100755 tests/functional/test_aarch64_rme_virt.py
 create mode 100755 tests/functional/test_m68k_tuxrun.py

-- 
2.39.5

[PULL 01/32] tests/functional: update the arm tuxrun tests

[Alex Bennée]

Now there are new up to date images available we should update to them.

Cc: Anders Roxell <anders.roxell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-2-alex.bennee@linaro.org>

diff --git a/tests/functional/test_arm_tuxrun.py b/tests/functional/test_arm_tuxrun.py
index 944f0756e1..4ac85f48ac 100755
--- a/tests/functional/test_arm_tuxrun.py
+++ b/tests/functional/test_arm_tuxrun.py
@@ -17,14 +17,14 @@
 class TuxRunArmTest(TuxRunBaselineTest):
 
     ASSET_ARMV5_KERNEL = Asset(
-        'https://storage.tuxboot.com/20230331/armv5/zImage',
-        'c95af2f27647c12265d75e9df44c22ff5228c59855f54aaa70f41ec2842e3a4d')
+        'https://storage.tuxboot.com/buildroot/20241119/armv5/zImage',
+        '3931a3908dbcf0ec0fe292d035ffc4dfed95f797dedd4a59ccfcf7a46e6f92d4')
     ASSET_ARMV5_ROOTFS = Asset(
-        'https://storage.tuxboot.com/20230331/armv5/rootfs.ext4.zst',
-        '17177afa74e7294da0642861f08c88ca3c836764299a54bf6d1ce276cb9712a5')
+        'https://storage.tuxboot.com/buildroot/20241119/armv5/rootfs.ext4.zst',
+        '60ff78b68c7021df378e4fc2d66d3b016484d1acc7e07fb8920c1d8e30f4571f')
     ASSET_ARMV5_DTB = Asset(
-        'https://storage.tuxboot.com/20230331/armv5/versatile-pb.dtb',
-        '0bc0c0b0858cefd3c32b385c0d66d97142ded29472a496f4f490e42fc7615b25')
+        'https://storage.tuxboot.com/buildroot/20241119/armv5/versatile-pb.dtb',
+        '50988e69ef3f3b08bfb9146e8fe414129990029e8dfbed444953b7e14809530a')
 
     def test_armv5(self):
         self.set_machine('versatilepb')
@@ -37,11 +37,11 @@ def test_armv5(self):
                            drive="virtio-blk-pci")
 
     ASSET_ARMV7_KERNEL = Asset(
-        'https://storage.tuxboot.com/20230331/armv7/zImage',
-        '4c7a22e9f15875bec06bd2a29d822496571eb297d4f22694099ffcdb19077572')
+        'https://storage.tuxboot.com/buildroot/20241119/armv7/zImage',
+        '1377bc3d90de5ce57ab17cd67429fe8b15c2e9964248c775c682b67e6299b991')
     ASSET_ARMV7_ROOTFS = Asset(
-        'https://storage.tuxboot.com/20230331/armv7/rootfs.ext4.zst',
-        'ab1fbbeaddda1ffdd45c9405a28cd5370c20f23a7cbc809cc90dc9f243a8eb5a')
+        'https://storage.tuxboot.com/buildroot/20241119/armv7/rootfs.ext4.zst',
+        'ed2cbc69bd6b3fbd5cafb5ee961393c7cfbe726446f14301c67d6b1f28bfdb51')
 
     def test_armv7(self):
         self.set_machine('virt')
@@ -52,11 +52,11 @@ def test_armv7(self):
                            rootfs_asset=self.ASSET_ARMV7_ROOTFS)
 
     ASSET_ARMV7BE_KERNEL = Asset(
-        'https://storage.tuxboot.com/20230331/armv7be/zImage',
-        '7facc62082b57af12015b08f7fdbaf2f123ba07a478367853ae12b219afc9f2f')
+        'https://storage.tuxboot.com/buildroot/20241119/armv7be/zImage',
+        'a244e6da99f1bbd254827ec7681bd4aac9eb1aa05aaebc6b15e5d289ebb683f3')
     ASSET_ARMV7BE_ROOTFS = Asset(
-        'https://storage.tuxboot.com/20230331/armv7be/rootfs.ext4.zst',
-        '42ed46dd2d59986206c5b1f6cf35eab58fe3fd20c96b41aaa16b32f3f90a9835')
+        'https://storage.tuxboot.com/buildroot/20241119/armv7be/rootfs.ext4.zst',
+        'd4f9c57860a512163f30ecc69b2174d1a1bdeb853a43dc49a09cfcfe84e428ea')
 
     def test_armv7be(self):
         self.set_machine('virt')
-- 
2.39.5

[PULL 02/32] tests/functional: update the i386 tuxrun tests

[Alex Bennée]

Now there are new up to date images available we should update to them.

Cc: Anders Roxell <anders.roxell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-3-alex.bennee@linaro.org>

diff --git a/tests/functional/test_i386_tuxrun.py b/tests/functional/test_i386_tuxrun.py
index c593ffbe8c..f3ccf11ae8 100755
--- a/tests/functional/test_i386_tuxrun.py
+++ b/tests/functional/test_i386_tuxrun.py
@@ -17,11 +17,11 @@
 class TuxRunI386Test(TuxRunBaselineTest):
 
     ASSET_I386_KERNEL = Asset(
-        'https://storage.tuxboot.com/20230331/i386/bzImage',
-        'a3e5b32a354729e65910f5a1ffcda7c14a6c12a55e8213fb86e277f1b76ed956')
+        'https://storage.tuxboot.com/buildroot/20241119/i386/bzImage',
+        '47fb44e38e34101eb0f71a2a01742b959d40ed5fd67cefb5608a39be11d3b74e')
     ASSET_I386_ROOTFS = Asset(
-        'https://storage.tuxboot.com/20230331/i386/rootfs.ext4.zst',
-        'f15e66b2bf673a210ec2a4b2e744a80530b36289e04f5388aab812b97f69754a')
+        'https://storage.tuxboot.com/buildroot/20241119/i386/rootfs.ext4.zst',
+        'a1a3b3b4c9dccd6475b58db95c107b468b736b700f6620985a8ed050a73d51c8')
 
     def test_i386(self):
         self.set_machine('q35')
-- 
2.39.5

[PULL 03/32] tests/functional: add a m68k tuxrun tests

[Alex Bennée]

We didn't have this before and as it exercises the m68k virt platform
it seems worth adding. We don't wait for the shutdown because QEMU
will auto-exit on the shutdown.

Cc: Laurent Vivier <laurent@vivier.eu>
Cc: Anders Roxell <anders.roxell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-4-alex.bennee@linaro.org>

diff --git a/MAINTAINERS b/MAINTAINERS
index 2101b51217..cab9018d9f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1288,6 +1288,7 @@ F: include/hw/intc/goldfish_pic.h
 F: include/hw/intc/m68k_irqc.h
 F: include/hw/misc/virt_ctrl.h
 F: docs/specs/virt-ctlr.rst
+F: tests/functional/test_m68k_tuxrun.py
 
 MicroBlaze Machines
 -------------------
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index a5087fcb34..7890dcb86d 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -116,6 +116,7 @@ tests_m68k_system_thorough = [
   'm68k_mcf5208evb',
   'm68k_nextcube',
   'm68k_q800',
+  'm68k_tuxrun',
 ]
 
 tests_microblaze_system_thorough = [
diff --git a/tests/functional/test_m68k_tuxrun.py b/tests/functional/test_m68k_tuxrun.py
new file mode 100755
index 0000000000..7eacba135f
--- /dev/null
+++ b/tests/functional/test_m68k_tuxrun.py
@@ -0,0 +1,34 @@
+#!/usr/bin/env python3
+#
+# Functional test that boots known good tuxboot images the same way
+# that tuxrun (www.tuxrun.org) does. This tool is used by things like
+# the LKFT project to run regression tests on kernels.
+#
+# Copyright (c) 2024 Linaro Ltd.
+#
+# Author:
+#  Alex Bennée <alex.bennee@linaro.org>
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+from qemu_test import Asset
+from qemu_test.tuxruntest import TuxRunBaselineTest
+
+class TuxRunM68KTest(TuxRunBaselineTest):
+
+    ASSET_M68K_KERNEL = Asset(
+        'https://storage.tuxboot.com/buildroot/20241119/m68k/vmlinux',
+        '7754e1d5cec753ccf1dc6894729a7f54c1a4965631ebf56df8e4ce1163ad19d8')
+    ASSET_M68K_ROOTFS = Asset(
+        'https://storage.tuxboot.com/buildroot/20241119/m68k/rootfs.ext4.zst',
+        '557962ffff265607912e82232cf21adbe0e4e5a88e1e1d411ce848c37f0213e9')
+
+    def test_m68k(self):
+        self.set_machine('virt')
+        self.cpu="m68040"
+        self.common_tuxrun(kernel_asset=self.ASSET_M68K_KERNEL,
+                           rootfs_asset=self.ASSET_M68K_ROOTFS,
+                           drive="virtio-blk-device")
+
+if __name__ == '__main__':
+    TuxRunBaselineTest.main()
-- 
2.39.5

[PULL 04/32] tests/functional: update the mips32 tuxrun tests

[Alex Bennée]

Now there are new up to date images available we should update to them.

Cc: Anders Roxell <anders.roxell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-5-alex.bennee@linaro.org>

diff --git a/tests/functional/test_mips_tuxrun.py b/tests/functional/test_mips_tuxrun.py
index 6fec44c2bf..6771dbd57e 100755
--- a/tests/functional/test_mips_tuxrun.py
+++ b/tests/functional/test_mips_tuxrun.py
@@ -17,11 +17,11 @@
 class TuxRunMipsTest(TuxRunBaselineTest):
 
     ASSET_MIPS_KERNEL = Asset(
-        'https://storage.tuxboot.com/20230331/mips32/vmlinux',
-        'bfd2172f8b17fb32970ca0c8c58f59c5a4ca38aa5855d920be3a69b5d16e52f0')
+        'https://storage.tuxboot.com/buildroot/20241119/mips32/vmlinux',
+        'b6f97fc698ae8c96456ad8c996c7454228074df0d7520dedd0a15e2913700a19')
     ASSET_MIPS_ROOTFS = Asset(
-        'https://storage.tuxboot.com/20230331/mips32/rootfs.ext4.zst',
-        'fc3da0b4c2f38d74c6d705123bb0f633c76ed953128f9d0859378c328a6d11a0')
+        'https://storage.tuxboot.com/buildroot/20241119/mips32/rootfs.ext4.zst',
+        '87055cf3cbde3fd134e5039e7b87feb03231d8c4b21ee712b8ba3308dfa72f50')
 
     def test_mips32(self):
         self.set_machine('malta')
-- 
2.39.5

[PULL 05/32] tests/functional: update the mips32el tuxrun tests

[Alex Bennée]

Now there are new up to date images available we should update to them.

Cc: Anders Roxell <anders.roxell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-6-alex.bennee@linaro.org>

diff --git a/tests/functional/test_mipsel_tuxrun.py b/tests/functional/test_mipsel_tuxrun.py
index 2965bbd913..d4b39baab5 100755
--- a/tests/functional/test_mipsel_tuxrun.py
+++ b/tests/functional/test_mipsel_tuxrun.py
@@ -17,11 +17,11 @@
 class TuxRunMipsELTest(TuxRunBaselineTest):
 
     ASSET_MIPSEL_KERNEL = Asset(
-        'https://storage.tuxboot.com/20230331/mips32el/vmlinux',
-        '8573867c68a8443db8de6d08bb33fb291c189ca2ca671471d3973a3e712096a3')
+        'https://storage.tuxboot.com/buildroot/20241119/mips32el/vmlinux',
+        '660dd8c7a6ca7a32d37b4e6348865532ab0edb66802e8cc07869338444cf4929')
     ASSET_MIPSEL_ROOTFS = Asset(
-        'https://storage.tuxboot.com/20230331/mips32el/rootfs.ext4.zst',
-        'e799768e289fd69209c21f4dacffa11baea7543d5db101e8ce27e3bc2c41d90e')
+        'https://storage.tuxboot.com/buildroot/20241119/mips32el/rootfs.ext4.zst',
+        'c5d69542bcaed54a4f34671671eb4be5c608ee02671d4d0436544367816a73b1')
 
     def test_mips32el(self):
         self.set_machine('malta')
-- 
2.39.5

[PULL 06/32] tests/functional: update the mips64 tuxrun tests

[Alex Bennée]

Now there are new up to date images available we should update to them.

Cc: Anders Roxell <anders.roxell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-7-alex.bennee@linaro.org>

diff --git a/tests/functional/test_mips64_tuxrun.py b/tests/functional/test_mips64_tuxrun.py
index 54af1ae794..0e4c65961d 100755
--- a/tests/functional/test_mips64_tuxrun.py
+++ b/tests/functional/test_mips64_tuxrun.py
@@ -17,11 +17,11 @@
 class TuxRunMips64Test(TuxRunBaselineTest):
 
     ASSET_MIPS64_KERNEL = Asset(
-        'https://storage.tuxboot.com/20230331/mips64/vmlinux',
-        '09010e51e4b8bcbbd2494786ffb48eca78f228e96e5c5438344b0eac4029dc61')
+        'https://storage.tuxboot.com/buildroot/20241119/mips64/vmlinux',
+        'fe2882d216898ba2c56b49ba59f46ad392f36871f7fe325373cd926848b9dbdc')
     ASSET_MIPS64_ROOTFS = Asset(
-        'https://storage.tuxboot.com/20230331/mips64/rootfs.ext4.zst',
-        '69d91eeb04df3d8d172922c6993bb37d4deeb6496def75d8580f6f9de3e431da')
+        'https://storage.tuxboot.com/buildroot/20241119/mips64/rootfs.ext4.zst',
+        'b8c98400216b6d4fb3b3ff05e9929aa015948b596cf0b82234813c84a4f7f4d5')
 
     def test_mips64(self):
         self.set_machine('malta')
-- 
2.39.5

[PULL 07/32] tests/functional: update the mips64el tuxrun tests

[Alex Bennée]

Now there are new up to date images available we should update to them.

Cc: Anders Roxell <anders.roxell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-8-alex.bennee@linaro.org>

diff --git a/tests/functional/test_mips64el_tuxrun.py b/tests/functional/test_mips64el_tuxrun.py
index 819549a27b..0a24757c51 100755
--- a/tests/functional/test_mips64el_tuxrun.py
+++ b/tests/functional/test_mips64el_tuxrun.py
@@ -17,11 +17,11 @@
 class TuxRunMips64ELTest(TuxRunBaselineTest):
 
     ASSET_MIPS64EL_KERNEL = Asset(
-        'https://storage.tuxboot.com/20230331/mips64el/vmlinux',
-        'd4e08965e2155c4cccce7c5f34d18fe34c636cda2f2c9844387d614950155266')
+        'https://storage.tuxboot.com/buildroot/20241119/mips64el/vmlinux',
+        '0d2829a96f005229839c4cd586d4d8a136ea4b488d29821611c8e97f2266bfa9')
     ASSET_MIPS64EL_ROOTFS = Asset(
-        'https://storage.tuxboot.com/20230331/mips64el/rootfs.ext4.zst',
-        'fba585368f5915b1498ed081863474b2d7ec4e97cdd46d21bdcb2f9698f83de4')
+        'https://storage.tuxboot.com/buildroot/20241119/mips64el/rootfs.ext4.zst',
+        '69c8b69a4f1582ce4c6f01a994968f5d73bffb2fc99cbeeeb26c8b5a28eaeb84')
 
     def test_mips64el(self):
         self.set_machine('malta')
-- 
2.39.5

[PULL 08/32] tests/functional: update the ppc32 tuxrun tests

[Alex Bennée]

Now there are new up to date images available we should update to them.

Cc: Anders Roxell <anders.roxell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-9-alex.bennee@linaro.org>

diff --git a/tests/functional/test_ppc_tuxrun.py b/tests/functional/test_ppc_tuxrun.py
index 50b76946c4..5458a7fb71 100755
--- a/tests/functional/test_ppc_tuxrun.py
+++ b/tests/functional/test_ppc_tuxrun.py
@@ -17,11 +17,11 @@
 class TuxRunPPC32Test(TuxRunBaselineTest):
 
     ASSET_PPC32_KERNEL = Asset(
-        'https://storage.tuxboot.com/20230331/ppc32/uImage',
-        '1a68f74b860fda022fb12e03c5efece8c2b8b590d96cca37a8481a3ae0b3f81f')
+        'https://storage.tuxboot.com/buildroot/20241119/ppc32/uImage',
+        'aa5d81deabdb255a318c4bc5ffd6fdd2b5da1ef39f1955dcc35b671d258b68e9')
     ASSET_PPC32_ROOTFS = Asset(
-        'https://storage.tuxboot.com/20230331/ppc32/rootfs.ext4.zst',
-        '8885b9d999cc24d679542a02e9b6aaf48f718f2050ece6b8347074b6ee41dd09')
+        'https://storage.tuxboot.com/buildroot/20241119/ppc32/rootfs.ext4.zst',
+        '67554f830269d6bf53b67c7dd206bcc821e463993d526b1644066fea8117019b')
 
     def test_ppc32(self):
         self.set_machine('ppce500')
-- 
2.39.5

[PULL 09/32] tests/functional: update the ppc64 tuxrun tests

[Alex Bennée]

Now there are new up to date images available we should update to them.

Cc: Anders Roxell <anders.roxell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-10-alex.bennee@linaro.org>

diff --git a/tests/functional/test_ppc64_tuxrun.py b/tests/functional/test_ppc64_tuxrun.py
index 8a98d18ab3..05c6162b5e 100755
--- a/tests/functional/test_ppc64_tuxrun.py
+++ b/tests/functional/test_ppc64_tuxrun.py
@@ -85,11 +85,11 @@ def ppc64_common_tuxrun(self, kernel_asset, rootfs_asset, prefix):
                                drive="scsi-hd")
 
     ASSET_PPC64_KERNEL = Asset(
-        'https://storage.tuxboot.com/20230331/ppc64/vmlinux',
-        'f22a9b9e924174a4c199f4c7e5d91a2339fcfe51c6eafd0907dc3e09b64ab728')
+        'https://storage.tuxboot.com/buildroot/20241119/ppc64/vmlinux',
+        '8219d5cb26e7654ad7826fe8aee6290f7c01eef44f2cd6d26c15fe8f99e1c17c')
     ASSET_PPC64_ROOTFS = Asset(
-        'https://storage.tuxboot.com/20230331/ppc64/rootfs.ext4.zst',
-        '1d953e81a4379e537fc8e41e05a0a59d9b453eef97aa03d47866c6c45b00bdff')
+        'https://storage.tuxboot.com/buildroot/20241119/ppc64/rootfs.ext4.zst',
+        'b68e12314303c5dd0fef37ae98021299a206085ae591893e73557af99a02d373')
 
     def test_ppc64(self):
         self.ppc64_common_tuxrun(kernel_asset=self.ASSET_PPC64_KERNEL,
@@ -97,11 +97,11 @@ def test_ppc64(self):
                                  prefix='tuxrun_ppc64_')
 
     ASSET_PPC64LE_KERNEL = Asset(
-        'https://storage.tuxboot.com/20230331/ppc64le/vmlinux',
-        '979eb61b445a010fb13e2b927126991f8ceef9c590fa2be0996c00e293e80cf2')
+        'https://storage.tuxboot.com/buildroot/20241119/ppc64le/vmlinux',
+        '21aea1fbc18bf6fa7d8ca4ea48d4940b2c8363c077acd564eb47d769b7495279')
     ASSET_PPC64LE_ROOTFS = Asset(
-        'https://storage.tuxboot.com/20230331/ppc64le/rootfs.ext4.zst',
-        'b442678c93fb8abe1f7d3bfa20556488de6b475c22c8fed363f42cf81a0a3906')
+        'https://storage.tuxboot.com/buildroot/20241119/ppc64le/rootfs.ext4.zst',
+        '67d36a3f9597b738e8b7359bdf04500f4d9bb82fc35eaa65aa439d888b2392f4')
 
     def test_ppc64le(self):
         self.ppc64_common_tuxrun(kernel_asset=self.ASSET_PPC64LE_KERNEL,
-- 
2.39.5

[PULL 10/32] tests/functional: update the riscv32 tuxrun tests

[Alex Bennée]

Now there are new up to date images available we should update to them.

Cc: Anders Roxell <anders.roxell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-11-alex.bennee@linaro.org>

diff --git a/tests/functional/test_riscv32_tuxrun.py b/tests/functional/test_riscv32_tuxrun.py
index 49b57cd428..3c570208d0 100755
--- a/tests/functional/test_riscv32_tuxrun.py
+++ b/tests/functional/test_riscv32_tuxrun.py
@@ -17,11 +17,11 @@
 class TuxRunRiscV32Test(TuxRunBaselineTest):
 
     ASSET_RISCV32_KERNEL = Asset(
-        'https://storage.tuxboot.com/20230331/riscv32/Image',
-        '89599407d7334de629a40e7ad6503c73670359eb5f5ae9d686353a3d6deccbd5')
+        'https://storage.tuxboot.com/buildroot/20241119/riscv32/Image',
+        '872bc8f8e0d4661825d5f47f7bec64988e9d0a8bd5db8917d57e16f66d83b329')
     ASSET_RISCV32_ROOTFS = Asset(
-        'https://storage.tuxboot.com/20230331/riscv32/rootfs.ext4.zst',
-        '7168d296d0283238ea73cd5a775b3dd608e55e04c7b92b76ecce31bb13108cba')
+        'https://storage.tuxboot.com/buildroot/20241119/riscv32/rootfs.ext4.zst',
+        '511ad34e63222db08d6c1da16fad224970de36517a784110956ba6a24a0ee5f6')
 
     def test_riscv32(self):
         self.set_machine('virt')
-- 
2.39.5

[PULL 11/32] tests/functional: update the riscv64 tuxrun tests

[Alex Bennée]

Now there are new up to date images available we should update to them.
Note we re-use the riscv32 kernel and rootfs for test_riscv64_rv32.

Cc: Anders Roxell <anders.roxell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-12-alex.bennee@linaro.org>

diff --git a/tests/functional/test_riscv64_tuxrun.py b/tests/functional/test_riscv64_tuxrun.py
index 4e2449539c..0d8de36204 100755
--- a/tests/functional/test_riscv64_tuxrun.py
+++ b/tests/functional/test_riscv64_tuxrun.py
@@ -17,18 +17,18 @@
 class TuxRunRiscV64Test(TuxRunBaselineTest):
 
     ASSET_RISCV64_KERNEL = Asset(
-        'https://storage.tuxboot.com/20230331/riscv64/Image',
-        'cd634badc65e52fb63465ec99e309c0de0369f0841b7d9486f9729e119bac25e')
+        'https://storage.tuxboot.com/buildroot/20241119/riscv64/Image',
+        '2bd8132a3bf21570290042324fff48c987f42f2a00c08de979f43f0662ebadba')
     ASSET_RISCV64_ROOTFS = Asset(
-        'https://storage.tuxboot.com/20230331/riscv64/rootfs.ext4.zst',
-        'b18e3a3bdf27be03da0b285e84cb71bf09eca071c3a087b42884b6982ed679eb')
+        'https://storage.tuxboot.com/buildroot/20241119/riscv64/rootfs.ext4.zst',
+        'aa4736a9872651dfc0d95e709465eedf1134fd19d42b8cb305bfd776f9801004')
 
     ASSET_RISCV32_KERNEL = Asset(
-        'https://storage.tuxboot.com/20230331/riscv32/Image',
-        '89599407d7334de629a40e7ad6503c73670359eb5f5ae9d686353a3d6deccbd5')
+        'https://storage.tuxboot.com/buildroot/20241119/riscv32/Image',
+        '872bc8f8e0d4661825d5f47f7bec64988e9d0a8bd5db8917d57e16f66d83b329')
     ASSET_RISCV32_ROOTFS = Asset(
-        'https://storage.tuxboot.com/20230331/riscv32/rootfs.ext4.zst',
-        '7168d296d0283238ea73cd5a775b3dd608e55e04c7b92b76ecce31bb13108cba')
+        'https://storage.tuxboot.com/buildroot/20241119/riscv32/rootfs.ext4.zst',
+        '511ad34e63222db08d6c1da16fad224970de36517a784110956ba6a24a0ee5f6')
 
     def test_riscv64(self):
         self.set_machine('virt')
-- 
2.39.5

[PULL 12/32] tests/functional: update the s390x tuxrun tests

[Alex Bennée]

Now there are new up to date images available we should update to them.

Cc: Anders Roxell <anders.roxell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-13-alex.bennee@linaro.org>

diff --git a/tests/functional/test_s390x_tuxrun.py b/tests/functional/test_s390x_tuxrun.py
index dcab17c68b..a7db4bfd84 100755
--- a/tests/functional/test_s390x_tuxrun.py
+++ b/tests/functional/test_s390x_tuxrun.py
@@ -17,11 +17,11 @@
 class TuxRunS390xTest(TuxRunBaselineTest):
 
     ASSET_S390X_KERNEL = Asset(
-        'https://storage.tuxboot.com/20230331/s390/bzImage',
-        '0414e98dd1c3dafff8496c9cd9c28a5f8d04553bb5ba37e906a812b48d442ef0')
+        'https://storage.tuxboot.com/buildroot/20241119/s390/bzImage',
+        'ee67e91db52a2aed104a7c72b2a08987c678f8179c029626789c35d6dd0fedf1')
     ASSET_S390X_ROOTFS = Asset(
-        'https://storage.tuxboot.com/20230331/s390/rootfs.ext4.zst',
-        '88c37c32276677f873a25ab9ec6247895b8e3e6f8259134de2a616080b8ab3fc')
+        'https://storage.tuxboot.com/buildroot/20241119/s390/rootfs.ext4.zst',
+        'bff7971fc2fef56372d98afe4557b82fd0a785a241e44c29b058e577ad1bbb44')
 
     def test_s390(self):
         self.wait_for_shutdown=False
-- 
2.39.5

[PULL 13/32] tests/functional: update the sparc64 tuxrun tests

[Alex Bennée]

Now there are new up to date images available we should update to them.

Cc: Anders Roxell <anders.roxell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-14-alex.bennee@linaro.org>

diff --git a/tests/functional/test_sparc64_tuxrun.py b/tests/functional/test_sparc64_tuxrun.py
index 1c2c005630..3be08d6102 100755
--- a/tests/functional/test_sparc64_tuxrun.py
+++ b/tests/functional/test_sparc64_tuxrun.py
@@ -17,11 +17,11 @@
 class TuxRunSparc64Test(TuxRunBaselineTest):
 
     ASSET_SPARC64_KERNEL = Asset(
-        'https://storage.tuxboot.com/20230331/sparc64/vmlinux',
-        'e34313e4325ff21deaa3d38a502aa09a373ef62b9bd4d7f8f29388b688225c55')
+        'https://storage.tuxboot.com/buildroot/20241119/sparc64/vmlinux',
+        'a04cfb2e70a264051d161fdd93aabf4b2a9472f2e435c14ed18c5848c5fed261')
     ASSET_SPARC64_ROOTFS = Asset(
-        'https://storage.tuxboot.com/20230331/sparc64/rootfs.ext4.zst',
-        'ad2f1dc436ab51583543d25d2c210cab478645d47078d30d129a66ab0e281d76')
+        'https://storage.tuxboot.com/buildroot/20241119/sparc64/rootfs.ext4.zst',
+        '479c3dc104c82b68be55e2c0c5c38cd473d0b37ad4badccde4775bb88ce34611')
 
     def test_sparc64(self):
         self.root='sda'
-- 
2.39.5

[PULL 14/32] tests/functional: update the x86_64 tuxrun tests

[Alex Bennée]

Now there are new up to date images available we should update to them.

Cc: Anders Roxell <anders.roxell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-15-alex.bennee@linaro.org>

diff --git a/tests/functional/test_x86_64_tuxrun.py b/tests/functional/test_x86_64_tuxrun.py
index 4f96139871..fcbc62b1b0 100755
--- a/tests/functional/test_x86_64_tuxrun.py
+++ b/tests/functional/test_x86_64_tuxrun.py
@@ -17,11 +17,11 @@
 class TuxRunX86Test(TuxRunBaselineTest):
 
     ASSET_X86_64_KERNEL = Asset(
-        'https://storage.tuxboot.com/20230331/x86_64/bzImage',
-        '2bc7480a669ee9b6b82500a236aba0c54233debe98cb968268fa230f52f03461')
+        'https://storage.tuxboot.com/buildroot/20241119/x86_64/bzImage',
+        'f57bfc6553bcd6e0a54aab86095bf642b33b5571d14e3af1731b18c87ed5aef8')
     ASSET_X86_64_ROOTFS = Asset(
-        'https://storage.tuxboot.com/20230331/x86_64/rootfs.ext4.zst',
-        'b72ac729769b8f51c6dffb221113c9a063c774dbe1d66af30eb593c4e9999b4b')
+        'https://storage.tuxboot.com/buildroot/20241119/x86_64/rootfs.ext4.zst',
+        '4b8b2a99117519c5290e1202cb36eb6c7aaba92b357b5160f5970cf5fb78a751')
 
     def test_x86_64(self):
         self.set_machine('q35')
-- 
2.39.5

[PULL 15/32] tests/functional/aarch64: add tests for FEAT_RME

[Alex Bennée]

From: Pierrick Bouvier <pierrick.bouvier@linaro.org>

This boot an OP-TEE environment, and launch a nested guest VM inside it
using the Realms feature. We do it for virt and sbsa-ref platforms.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Message-Id: <20241220165212.3653495-1-pierrick.bouvier@linaro.org>
[AJB: tweak ordering of setup, strip changelog from commit]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20250108121054.1126164-16-alex.bennee@linaro.org>

diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 7890dcb86d..bd3d903cfc 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -13,6 +13,8 @@ endif
 test_timeouts = {
   'aarch64_aspeed' : 600,
   'aarch64_raspi4' : 480,
+  'aarch64_rme_virt' : 1200,
+  'aarch64_rme_sbsaref' : 1200,
   'aarch64_sbsaref_alpine' : 720,
   'aarch64_sbsaref_freebsd' : 720,
   'aarch64_tuxrun' : 240,
@@ -60,6 +62,8 @@ tests_aarch64_system_thorough = [
   'aarch64_aspeed',
   'aarch64_raspi3',
   'aarch64_raspi4',
+  'aarch64_rme_virt',
+  'aarch64_rme_sbsaref',
   'aarch64_sbsaref',
   'aarch64_sbsaref_alpine',
   'aarch64_sbsaref_freebsd',
diff --git a/tests/functional/test_aarch64_rme_sbsaref.py b/tests/functional/test_aarch64_rme_sbsaref.py
new file mode 100755
index 0000000000..93bb528338
--- /dev/null
+++ b/tests/functional/test_aarch64_rme_sbsaref.py
@@ -0,0 +1,69 @@
+#!/usr/bin/env python3
+#
+# Functional test that boots a Realms environment on sbsa-ref machine and a
+# nested guest VM using it.
+#
+# Copyright (c) 2024 Linaro Ltd.
+#
+# Author: Pierrick Bouvier <pierrick.bouvier@linaro.org>
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+import time
+import os
+import logging
+
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import exec_command, wait_for_console_pattern
+from qemu_test import exec_command_and_wait_for_pattern
+from test_aarch64_rme_virt import test_realms_guest
+
+class Aarch64RMESbsaRefMachine(QemuSystemTest):
+
+    # Stack is built with OP-TEE build environment from those instructions:
+    # https://linaro.atlassian.net/wiki/spaces/QEMU/pages/29051027459/
+    # https://github.com/pbo-linaro/qemu-rme-stack
+    ASSET_RME_STACK_SBSA = Asset(
+        ('https://fileserver.linaro.org/s/KJyeBxL82mz2r7F/'
+         'download/rme-stack-op-tee-4.2.0-cca-v4-sbsa.tar.gz'),
+         'dd9ab28ec869bdf3b5376116cb3689103b43433fd5c4bca0f4a8d8b3c104999e')
+
+    # This tests the FEAT_RME cpu implementation, by booting a VM supporting it,
+    # and launching a nested VM using it.
+    def test_aarch64_rme_sbsaref(self):
+        self.set_machine('sbsa-ref')
+        self.require_accelerator('tcg')
+
+        self.vm.set_console()
+
+        stack_path_tar_gz = self.ASSET_RME_STACK_SBSA.fetch()
+        self.archive_extract(stack_path_tar_gz, format="tar")
+
+        rme_stack = self.scratch_file('rme-stack-op-tee-4.2.0-cca-v4-sbsa')
+        pflash0 = os.path.join(rme_stack, 'images', 'SBSA_FLASH0.fd')
+        pflash1 = os.path.join(rme_stack, 'images', 'SBSA_FLASH1.fd')
+        virtual = os.path.join(rme_stack, 'images', 'disks', 'virtual')
+        drive = os.path.join(rme_stack, 'out-br', 'images', 'rootfs.ext4')
+
+        self.vm.add_args('-cpu', 'max,x-rme=on,pauth-impdef=on')
+        self.vm.add_args('-m', '2G')
+        self.vm.add_args('-M', 'sbsa-ref')
+        self.vm.add_args('-drive', f'file={pflash0},format=raw,if=pflash')
+        self.vm.add_args('-drive', f'file={pflash1},format=raw,if=pflash')
+        self.vm.add_args('-drive', f'file=fat:rw:{virtual},format=raw')
+        self.vm.add_args('-drive', f'format=raw,if=none,file={drive},id=hd0')
+        self.vm.add_args('-device', 'virtio-blk-pci,drive=hd0')
+        self.vm.add_args('-device', 'virtio-9p-pci,fsdev=shr0,mount_tag=shr0')
+        self.vm.add_args('-fsdev', f'local,security_model=none,path={rme_stack},id=shr0')
+        self.vm.add_args('-device', 'virtio-net-pci,netdev=net0')
+        self.vm.add_args('-netdev', 'user,id=net0')
+
+        self.vm.launch()
+        # Wait for host VM boot to complete.
+        wait_for_console_pattern(self, 'Welcome to Buildroot')
+        exec_command_and_wait_for_pattern(self, 'root', '#')
+
+        test_realms_guest(self)
+
+if __name__ == '__main__':
+    QemuSystemTest.main()
diff --git a/tests/functional/test_aarch64_rme_virt.py b/tests/functional/test_aarch64_rme_virt.py
new file mode 100755
index 0000000000..42b9229b4c
--- /dev/null
+++ b/tests/functional/test_aarch64_rme_virt.py
@@ -0,0 +1,98 @@
+#!/usr/bin/env python3
+#
+# Functional test that boots a Realms environment on virt machine and a nested
+# guest VM using it.
+#
+# Copyright (c) 2024 Linaro Ltd.
+#
+# Author: Pierrick Bouvier <pierrick.bouvier@linaro.org>
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+import time
+import os
+import logging
+
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import exec_command, wait_for_console_pattern
+from qemu_test import exec_command_and_wait_for_pattern
+
+def test_realms_guest(test_rme_instance):
+
+    # Boot the (nested) guest VM
+    exec_command(test_rme_instance,
+                 'qemu-system-aarch64 -M virt,gic-version=3 '
+                 '-cpu host -enable-kvm -m 512M '
+                 '-M confidential-guest-support=rme0 '
+                 '-object rme-guest,id=rme0 '
+                 '-device virtio-net-pci,netdev=net0,romfile= '
+                 '-netdev user,id=net0 '
+                 '-kernel /mnt/out/bin/Image '
+                 '-initrd /mnt/out-br/images/rootfs.cpio '
+                 '-serial stdio')
+    # Detect Realm activation during (nested) guest boot.
+    wait_for_console_pattern(test_rme_instance,
+                             'SMC_RMI_REALM_ACTIVATE')
+    # Wait for (nested) guest boot to complete.
+    wait_for_console_pattern(test_rme_instance,
+                             'Welcome to Buildroot')
+    exec_command_and_wait_for_pattern(test_rme_instance, 'root', '#')
+    # query (nested) guest cca report
+    exec_command(test_rme_instance, 'cca-workload-attestation report')
+    wait_for_console_pattern(test_rme_instance,
+                             '"cca-platform-hash-algo-id": "sha-256"')
+    wait_for_console_pattern(test_rme_instance,
+                             '"cca-realm-hash-algo-id": "sha-512"')
+    wait_for_console_pattern(test_rme_instance,
+                             '"cca-realm-public-key-hash-algo-id": "sha-256"')
+
+class Aarch64RMEVirtMachine(QemuSystemTest):
+
+    # Stack is built with OP-TEE build environment from those instructions:
+    # https://linaro.atlassian.net/wiki/spaces/QEMU/pages/29051027459/
+    # https://github.com/pbo-linaro/qemu-rme-stack
+    ASSET_RME_STACK_VIRT = Asset(
+        ('https://fileserver.linaro.org/s/iaRsNDJp2CXHMSJ/'
+         'download/rme-stack-op-tee-4.2.0-cca-v4-qemu_v8.tar.gz'),
+         '1851adc232b094384d8b879b9a2cfff07ef3d6205032b85e9b3a4a9ae6b0b7ad')
+
+    # This tests the FEAT_RME cpu implementation, by booting a VM supporting it,
+    # and launching a nested VM using it.
+    def test_aarch64_rme_virt(self):
+        self.set_machine('virt')
+        self.vm.set_console()
+        self.require_accelerator('tcg')
+
+        stack_path_tar_gz = self.ASSET_RME_STACK_VIRT.fetch()
+        self.archive_extract(stack_path_tar_gz, format="tar")
+
+        rme_stack = self.scratch_file('rme-stack-op-tee-4.2.0-cca-v4-qemu_v8')
+        kernel = os.path.join(rme_stack, 'out', 'bin', 'Image')
+        bios = os.path.join(rme_stack, 'out', 'bin', 'flash.bin')
+        drive = os.path.join(rme_stack, 'out-br', 'images', 'rootfs.ext4')
+
+        self.vm.add_args('-cpu', 'max,x-rme=on,pauth-impdef=on')
+        self.vm.add_args('-m', '2G')
+        self.vm.add_args('-M', 'virt,acpi=off,'
+                         'virtualization=on,'
+                         'secure=on,'
+                         'gic-version=3')
+        self.vm.add_args('-bios', bios)
+        self.vm.add_args('-kernel', kernel)
+        self.vm.add_args('-drive', f'format=raw,if=none,file={drive},id=hd0')
+        self.vm.add_args('-device', 'virtio-blk-pci,drive=hd0')
+        self.vm.add_args('-device', 'virtio-9p-device,fsdev=shr0,mount_tag=shr0')
+        self.vm.add_args('-fsdev', f'local,security_model=none,path={rme_stack},id=shr0')
+        self.vm.add_args('-device', 'virtio-net-pci,netdev=net0')
+        self.vm.add_args('-netdev', 'user,id=net0')
+        self.vm.add_args('-append', 'root=/dev/vda')
+
+        self.vm.launch()
+        # Wait for host VM boot to complete.
+        wait_for_console_pattern(self, 'Welcome to Buildroot')
+        exec_command_and_wait_for_pattern(self, 'root', '#')
+
+        test_realms_guest(self)
+
+if __name__ == '__main__':
+    QemuSystemTest.main()
-- 
2.39.5

[PULL 16/32] tests/qtest: remove clock_steps from virtio tests

[Alex Bennée]

In the qtest environment time will not step forward if the system is
paused (timers disabled) or we have no timer events to fire. As a
result VirtIO events are responded to directly and we don't need to
step time forward.

We still do timeout processing to handle the fact the target QEMU may
not be ready to respond right away. This will usually be due to a slow
CI system or if QEMU is running under something like rr.

Future qtest patches will assert that time actually changes when a
step is requested.

Reviewed-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-17-alex.bennee@linaro.org>

diff --git a/tests/qtest/libqos/virtio.c b/tests/qtest/libqos/virtio.c
index a21b6eee9c..2e7979652f 100644
--- a/tests/qtest/libqos/virtio.c
+++ b/tests/qtest/libqos/virtio.c
@@ -170,7 +170,6 @@ void qvirtio_wait_queue_isr(QTestState *qts, QVirtioDevice *d,
     gint64 start_time = g_get_monotonic_time();
 
     for (;;) {
-        qtest_clock_step(qts, 100);
         if (d->bus->get_queue_isr_status(d, vq)) {
             return;
         }
@@ -192,7 +191,6 @@ uint8_t qvirtio_wait_status_byte_no_isr(QTestState *qts, QVirtioDevice *d,
     uint8_t val;
 
     while ((val = qtest_readb(qts, addr)) == 0xff) {
-        qtest_clock_step(qts, 100);
         g_assert(!d->bus->get_queue_isr_status(d, vq));
         g_assert(g_get_monotonic_time() - start_time <= timeout_us);
     }
@@ -219,14 +217,12 @@ void qvirtio_wait_used_elem(QTestState *qts, QVirtioDevice *d,
     for (;;) {
         uint32_t got_desc_idx;
 
-        qtest_clock_step(qts, 100);
 
         if (d->bus->get_queue_isr_status(d, vq) &&
             qvirtqueue_get_buf(qts, vq, &got_desc_idx, len)) {
             g_assert_cmpint(got_desc_idx, ==, desc_idx);
             return;
         }
-
         g_assert(g_get_monotonic_time() - start_time <= timeout_us);
     }
 }
-- 
2.39.5

[PULL 17/32] system/qtest: properly feedback results of clock_[step|set]

[Alex Bennée]

Time will not advance if the system is paused or there are no timer
events set for the future. In absence of pending timer events
advancing time would make no difference the system state. Attempting
to do so would be a bug and the test or device under test would need
fixing.

Tighten up the result reporting to `FAIL` if time was not advanced.

Reviewed-by: Fabiano Rosas <farosas@suse.de>
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2687
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-18-alex.bennee@linaro.org>

diff --git a/system/qtest.c b/system/qtest.c
index 99ef2042f6..e68ed0f2a8 100644
--- a/system/qtest.c
+++ b/system/qtest.c
@@ -78,6 +78,11 @@ static void *qtest_server_send_opaque;
  * let you adjust the value of the clock (monotonically).  All the commands
  * return the current value of the clock in nanoseconds.
  *
+ * If the commands FAIL then time wasn't advanced which is likely
+ * because the machine was in a paused state or no timer events exist
+ * in the future. This will cause qtest to abort and the test will
+ * need to check its assumptions.
+ *
  * .. code-block:: none
  *
  *  > clock_step
@@ -710,7 +715,8 @@ static void qtest_process_command(CharBackend *chr, gchar **words)
             qtest_sendf(chr, "OK little\n");
         }
     } else if (qtest_enabled() && strcmp(words[0], "clock_step") == 0) {
-        int64_t ns;
+        int64_t old_ns = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
+        int64_t ns, new_ns;
 
         if (words[1]) {
             int ret = qemu_strtoi64(words[1], NULL, 0, &ns);
@@ -719,11 +725,10 @@ static void qtest_process_command(CharBackend *chr, gchar **words)
             ns = qemu_clock_deadline_ns_all(QEMU_CLOCK_VIRTUAL,
                                             QEMU_TIMER_ATTR_ALL);
         }
-        qemu_clock_advance_virtual_time(
-            qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) + ns);
+        new_ns = qemu_clock_advance_virtual_time(old_ns + ns);
         qtest_send_prefix(chr);
-        qtest_sendf(chr, "OK %"PRIi64"\n",
-                    (int64_t)qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));
+        qtest_sendf(chr, "%s %"PRIi64"\n",
+                    new_ns > old_ns ? "OK" : "FAIL", new_ns);
     } else if (strcmp(words[0], "module_load") == 0) {
         Error *local_err = NULL;
         int rv;
@@ -740,16 +745,16 @@ static void qtest_process_command(CharBackend *chr, gchar **words)
             qtest_sendf(chr, "FAIL\n");
         }
     } else if (qtest_enabled() && strcmp(words[0], "clock_set") == 0) {
-        int64_t ns;
+        int64_t ns, new_ns;
         int ret;
 
         g_assert(words[1]);
         ret = qemu_strtoi64(words[1], NULL, 0, &ns);
         g_assert(ret == 0);
-        qemu_clock_advance_virtual_time(ns);
+        new_ns = qemu_clock_advance_virtual_time(ns);
         qtest_send_prefix(chr);
-        qtest_sendf(chr, "OK %"PRIi64"\n",
-                    (int64_t)qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));
+        qtest_sendf(chr, "%s %"PRIi64"\n",
+                    new_ns == ns ? "OK" : "FAIL", new_ns);
     } else if (process_command_cb && process_command_cb(chr, words)) {
         /* Command got consumed by the callback handler */
     } else {
-- 
2.39.5

[PULL 18/32] tests/functional: remove hacky sleep from the tests

[Alex Bennée]

We have proper detection of prompts now so we don't need to guess with
sleep() sprinkled through the test. The extra step of calling halt is
just to flush the final bits of the log (although the last line is
still missed).

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-19-alex.bennee@linaro.org>

diff --git a/tests/functional/test_aarch64_virt.py b/tests/functional/test_aarch64_virt.py
index 08576b0694..2d9995a95d 100755
--- a/tests/functional/test_aarch64_virt.py
+++ b/tests/functional/test_aarch64_virt.py
@@ -10,12 +10,12 @@
 #
 # SPDX-License-Identifier: GPL-2.0-or-later
 
-import time
 import logging
 from subprocess import check_call, DEVNULL
 
 from qemu_test import QemuSystemTest, Asset
-from qemu_test import exec_command, wait_for_console_pattern
+from qemu_test import exec_command_and_wait_for_pattern
+from qemu_test import wait_for_console_pattern
 from qemu_test import get_qemu_img
 
 
@@ -107,18 +107,22 @@ def common_aarch64_virt(self, machine):
                          'virtio-blk-device,drive=scratch')
 
         self.vm.launch()
-        self.wait_for_console_pattern('Welcome to Buildroot')
-        time.sleep(0.1)
-        exec_command(self, 'root')
-        time.sleep(0.1)
-        exec_command(self, 'dd if=/dev/hwrng of=/dev/vda bs=512 count=4')
-        time.sleep(0.1)
-        exec_command(self, 'md5sum /dev/vda')
-        time.sleep(0.1)
-        exec_command(self, 'cat /proc/interrupts')
-        time.sleep(0.1)
-        exec_command(self, 'cat /proc/self/maps')
-        time.sleep(0.1)
+
+        ps1='#'
+        self.wait_for_console_pattern('login:')
+
+        commands = [
+            ('root', ps1),
+            ('cat /proc/interrupts', ps1),
+            ('cat /proc/self/maps', ps1),
+            ('uname -a', ps1),
+            ('dd if=/dev/hwrng of=/dev/vda bs=512 count=4', ps1),
+            ('md5sum /dev/vda', ps1),
+            ('halt -n', 'reboot: System halted')
+        ]
+
+        for cmd, pattern in commands:
+            exec_command_and_wait_for_pattern(self, cmd, pattern)
 
     def test_aarch64_virt_gicv3(self):
         self.common_aarch64_virt("virt,gic_version=3")
-- 
2.39.5

[PULL 19/32] tests/functional: add zstd support to uncompress utility

[Alex Bennée]

Rather than using the python library (which has a different API
anyway) lets just call the binary. zstdtools is already in out
qemu.yml so all test containers should have it around. Tests should
still use @skipIfMissingCommands('zstd') to gracefully handle when
only minimal dependencies have been installed.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-20-alex.bennee@linaro.org>

diff --git a/tests/functional/qemu_test/uncompress.py b/tests/functional/qemu_test/uncompress.py
index 6d02ded066..76dcf22385 100644
--- a/tests/functional/qemu_test/uncompress.py
+++ b/tests/functional/qemu_test/uncompress.py
@@ -10,8 +10,10 @@
 import gzip
 import lzma
 import os
+import stat
 import shutil
 from urllib.parse import urlparse
+from subprocess import check_call, CalledProcessError
 
 from .asset import Asset
 
@@ -38,6 +40,24 @@ def lzma_uncompress(xz_path, output_path):
             os.remove(output_path)
             raise
 
+
+def zstd_uncompress(zstd_path, output_path):
+    if os.path.exists(output_path):
+        return
+
+    try:
+        check_call(['zstd', "-f", "-d", zstd_path,
+                    "-o", output_path])
+    except CalledProcessError as e:
+        os.remove(output_path)
+        raise Exception(
+            f"Unable to decompress zstd file {zstd_path} with {e}") from e
+
+    # zstd copies source archive permissions for the output
+    # file, so must make this writable for QEMU
+    os.chmod(output_path, stat.S_IRUSR | stat.S_IWUSR)
+
+
 '''
 @params compressed: filename, Asset, or file-like object to uncompress
 @params uncompressed: filename to uncompress into
@@ -59,6 +79,8 @@ def uncompress(compressed, uncompressed, format=None):
         lzma_uncompress(str(compressed), uncompressed)
     elif format == "gz":
         gzip_uncompress(str(compressed), uncompressed)
+    elif format == "zstd":
+        zstd_uncompress(str(compressed), uncompressed)
     else:
         raise Exception(f"Unknown compression format {format}")
 
@@ -79,5 +101,7 @@ def guess_uncompress_format(compressed):
         return "xz"
     elif ext == ".gz":
         return "gz"
+    elif ext in [".zstd", ".zst"]:
+        return 'zstd'
     else:
         raise Exception(f"Unknown compression format for {compressed}")
-- 
2.39.5

[PULL 20/32] tests/functional: update tuxruntest to use uncompress utility

[Alex Bennée]

Use the utility functions to reduce code duplication.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-21-alex.bennee@linaro.org>

diff --git a/tests/functional/qemu_test/tuxruntest.py b/tests/functional/qemu_test/tuxruntest.py
index 7227a83757..41a4945a14 100644
--- a/tests/functional/qemu_test/tuxruntest.py
+++ b/tests/functional/qemu_test/tuxruntest.py
@@ -73,17 +73,7 @@ def fetch_tuxrun_assets(self, kernel_asset, rootfs_asset, dtb_asset=None):
         Fetch the TuxBoot assets.
         """
         kernel_image =  kernel_asset.fetch()
-        disk_image_zst = rootfs_asset.fetch()
-
-        disk_image = self.scratch_file("rootfs.ext4")
-
-        check_call(['zstd', "-f", "-d", disk_image_zst,
-                    "-o", disk_image],
-                   stdout=DEVNULL, stderr=DEVNULL)
-        # zstd copies source archive permissions for the output
-        # file, so must make this writable for QEMU
-        os.chmod(disk_image, stat.S_IRUSR | stat.S_IWUSR)
-
+        disk_image = self.uncompress(rootfs_asset)
         dtb = dtb_asset.fetch() if dtb_asset is not None else None
 
         return (kernel_image, disk_image, dtb)
-- 
2.39.5

[PULL 21/32] tests/functional: remove unused kernel_command_line

[Alex Bennée]

The Alpine test boots from the CDROM so we don't --append a command
line. Drop the unused code.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-22-alex.bennee@linaro.org>

diff --git a/tests/functional/test_aarch64_virt.py b/tests/functional/test_aarch64_virt.py
index 2d9995a95d..b3d3b0ee51 100755
--- a/tests/functional/test_aarch64_virt.py
+++ b/tests/functional/test_aarch64_virt.py
@@ -41,8 +41,6 @@ def test_alpine_virt_tcg_gic_max(self):
 
         self.set_machine('virt')
         self.vm.set_console()
-        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
-                               'console=ttyAMA0')
         self.require_accelerator("tcg")
 
         self.vm.add_args("-accel", "tcg")
-- 
2.39.5

[PULL 22/32] tests/functional: bail aarch64_virt tests early if missing TCG

[Alex Bennée]

The set_machine and require_accelerator steps can bail early so move
those to the front of the test functions. While we are at it also
clean up some long lines when adding the vm arguments.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-23-alex.bennee@linaro.org>

diff --git a/tests/functional/test_aarch64_virt.py b/tests/functional/test_aarch64_virt.py
index b3d3b0ee51..201c5ed023 100755
--- a/tests/functional/test_aarch64_virt.py
+++ b/tests/functional/test_aarch64_virt.py
@@ -40,9 +40,9 @@ def test_alpine_virt_tcg_gic_max(self):
         iso_path = self.ASSET_ALPINE_ISO.fetch()
 
         self.set_machine('virt')
-        self.vm.set_console()
         self.require_accelerator("tcg")
 
+        self.vm.set_console()
         self.vm.add_args("-accel", "tcg")
         self.vm.add_args("-cpu", "max,pauth-impdef=on")
         self.vm.add_args("-machine",
@@ -71,15 +71,16 @@ def common_aarch64_virt(self, machine):
         Common code to launch basic virt machine with kernel+initrd
         and a scratch disk.
         """
+        self.set_machine('virt')
+        self.require_accelerator("tcg")
+
         logger = logging.getLogger('aarch64_virt')
 
         kernel_path = self.ASSET_KERNEL.fetch()
 
-        self.set_machine('virt')
         self.vm.set_console()
         kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
                                'console=ttyAMA0')
-        self.require_accelerator("tcg")
         self.vm.add_args('-cpu', 'max,pauth-impdef=on',
                          '-machine', machine,
                          '-accel', 'tcg',
@@ -100,7 +101,9 @@ def common_aarch64_virt(self, machine):
 
         # Add the device
         self.vm.add_args('-blockdev',
-                         f"driver=qcow2,file.driver=file,file.filename={image_path},node-name=scratch")
+                         "driver=qcow2,"
+                         "file.driver=file,"
+                         f"file.filename={image_path},node-name=scratch")
         self.vm.add_args('-device',
                          'virtio-blk-device,drive=scratch')
 
-- 
2.39.5

[PULL 23/32] tests/functional: extend test_aarch64_virt with vulkan test

[Alex Bennée]

Now that we have virtio-gpu Vulkan support, let's add a test for it.
Currently this is using images build by buildroot:

  https://lists.buildroot.org/pipermail/buildroot/2024-December/768196.html

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-24-alex.bennee@linaro.org>

diff --git a/tests/functional/test_aarch64_virt.py b/tests/functional/test_aarch64_virt.py
index 201c5ed023..6b2336a28d 100755
--- a/tests/functional/test_aarch64_virt.py
+++ b/tests/functional/test_aarch64_virt.py
@@ -13,10 +13,12 @@
 import logging
 from subprocess import check_call, DEVNULL
 
+from qemu.machine.machine import VMLaunchFailure
+
 from qemu_test import QemuSystemTest, Asset
-from qemu_test import exec_command_and_wait_for_pattern
+from qemu_test import exec_command, exec_command_and_wait_for_pattern
 from qemu_test import wait_for_console_pattern
-from qemu_test import get_qemu_img
+from qemu_test import skipIfMissingCommands, get_qemu_img
 
 
 class Aarch64VirtMachine(QemuSystemTest):
@@ -132,5 +134,73 @@ def test_aarch64_virt_gicv2(self):
         self.common_aarch64_virt("virt,gic-version=2")
 
 
+    ASSET_VIRT_GPU_KERNEL = Asset(
+        'https://fileserver.linaro.org/s/ce5jXBFinPxtEdx/'
+        'download?path=%2F&files='
+        'Image',
+        '89e5099d26166204cc5ca4bb6d1a11b92c217e1f82ec67e3ba363d09157462f6')
+
+    ASSET_VIRT_GPU_ROOTFS = Asset(
+        'https://fileserver.linaro.org/s/ce5jXBFinPxtEdx/'
+        'download?path=%2F&files='
+        'rootfs.ext4.zstd',
+        '792da7573f5dc2913ddb7c638151d4a6b2d028a4cb2afb38add513c1924bdad4')
+
+    @skipIfMissingCommands('zstd')
+    def test_aarch64_virt_with_gpu(self):
+        # This tests boots with a buildroot test image that contains
+        # vkmark and other GPU exercising tools. We run a headless
+        # weston that nevertheless still exercises the virtio-gpu
+        # backend.
+
+        self.set_machine('virt')
+        self.require_accelerator("tcg")
+
+        kernel_path = self.ASSET_VIRT_GPU_KERNEL.fetch()
+        image_path = self.uncompress(self.ASSET_VIRT_GPU_ROOTFS, format="zstd")
+
+        self.vm.set_console()
+        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
+                               'console=ttyAMA0 root=/dev/vda')
+
+        self.vm.add_args("-accel", "tcg")
+        self.vm.add_args("-cpu", "neoverse-v1,pauth-impdef=on")
+        self.vm.add_args("-machine", "virt,gic-version=max",
+                         '-kernel', kernel_path,
+                         '-append', kernel_command_line)
+        self.vm.add_args("-smp", "2", "-m", "2048")
+        self.vm.add_args("-device",
+                         "virtio-gpu-gl-pci,hostmem=4G,blob=on,venus=on")
+        self.vm.add_args("-display", "egl-headless")
+        self.vm.add_args("-display", "dbus,gl=on")
+        self.vm.add_args("-device", "virtio-blk-device,drive=hd0")
+        self.vm.add_args("-blockdev",
+                         "driver=raw,file.driver=file,"
+                         "node-name=hd0,read-only=on,"
+                         f"file.filename={image_path}")
+        self.vm.add_args("-snapshot")
+
+        try:
+            self.vm.launch()
+        except VMLaunchFailure as excp:
+            if "old virglrenderer, blob resources unsupported" in excp.output:
+                self.skipTest("No blob support for virtio-gpu")
+            elif "old virglrenderer, venus unsupported" in excp.output:
+                self.skipTest("No venus support for virtio-gpu")
+            else:
+                self.log.info("unhandled launch failure: {excp.output}")
+                raise excp
+
+        self.wait_for_console_pattern('buildroot login:')
+        exec_command(self, 'root')
+        exec_command(self, 'export XDG_RUNTIME_DIR=/tmp')
+        exec_command_and_wait_for_pattern(self,
+                                          "weston -B headless "
+                                          "--renderer gl "
+                                          "--shell kiosk "
+                                          "-- vkmark -b:duration=1.0",
+                                          "vkmark Score")
+
+
 if __name__ == '__main__':
     QemuSystemTest.main()
-- 
2.39.5

[PULL 24/32] tests/lcitool: bump to latest version of libvirt-ci

[Alex Bennée]

We will shortly need this to build our riscv64 cross container.
However to keep the delta down just do the bump first. As ccache4 is
now preferred for FreeBSD to get the latest version there is a little
update in the FreeBSD metadata.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-25-alex.bennee@linaro.org>

diff --git a/.gitlab-ci.d/cirrus/freebsd-14.vars b/.gitlab-ci.d/cirrus/freebsd-14.vars
index 0a7ac5e0e1..0997c47af5 100644
--- a/.gitlab-ci.d/cirrus/freebsd-14.vars
+++ b/.gitlab-ci.d/cirrus/freebsd-14.vars
@@ -11,6 +11,6 @@ MAKE='/usr/local/bin/gmake'
 NINJA='/usr/local/bin/ninja'
 PACKAGING_COMMAND='pkg'
 PIP3='/usr/local/bin/pip'
-PKGS='alsa-lib bash bison bzip2 ca_root_nss capstone4 ccache cmocka ctags curl cyrus-sasl dbus diffutils dtc flex fusefs-libs3 gettext git glib gmake gnutls gsed gtk-vnc gtk3 json-c libepoxy libffi libgcrypt libjpeg-turbo libnfs libslirp libspice-server libssh libtasn1 llvm lzo2 meson mtools ncurses nettle ninja opencv pixman pkgconf png py311-numpy py311-pillow py311-pip py311-pyyaml py311-sphinx py311-sphinx_rtd_theme py311-tomli python3 rpm2cpio rust rust-bindgen-cli sdl2 sdl2_image snappy sndio socat spice-protocol tesseract usbredir virglrenderer vte3 xorriso zstd'
+PKGS='alsa-lib bash bison bzip2 ca_root_nss capstone4 ccache4 cmocka ctags curl cyrus-sasl dbus diffutils dtc flex fusefs-libs3 gettext git glib gmake gnutls gsed gtk-vnc gtk3 json-c libepoxy libffi libgcrypt libjpeg-turbo libnfs libslirp libspice-server libssh libtasn1 llvm lzo2 meson mtools ncurses nettle ninja opencv pixman pkgconf png py311-numpy py311-pillow py311-pip py311-pyyaml py311-sphinx py311-sphinx_rtd_theme py311-tomli python3 rpm2cpio rust rust-bindgen-cli sdl2 sdl2_image snappy sndio socat spice-protocol tesseract usbredir virglrenderer vte3 xorriso zstd'
 PYPI_PKGS=''
 PYTHON='/usr/local/bin/python3'
diff --git a/tests/lcitool/libvirt-ci b/tests/lcitool/libvirt-ci
index 9ad3f70bde..b6a65806bc 160000
--- a/tests/lcitool/libvirt-ci
+++ b/tests/lcitool/libvirt-ci
@@ -1 +1 @@
-Subproject commit 9ad3f70bde9865d5ad18f36d256d472e72b5cbf3
+Subproject commit b6a65806bc9b2b56985f5e97c936b77c7e7a99fc
diff --git a/tests/vm/generated/freebsd.json b/tests/vm/generated/freebsd.json
index 3cb7fb7060..81fc38d798 100644
--- a/tests/vm/generated/freebsd.json
+++ b/tests/vm/generated/freebsd.json
@@ -13,7 +13,7 @@
     "bzip2",
     "ca_root_nss",
     "capstone4",
-    "ccache",
+    "ccache4",
     "cmocka",
     "ctags",
     "curl",
-- 
2.39.5

[PULL 25/32] tests/docker: move riscv64 cross container from sid to trixie

[Alex Bennée]

Although riscv64 isn't going to be a release architecture for trixie
the packages are still built while it is testing. Moving from sid will
also avoid some of the volatility we get from tracking the bleeding
edge.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-26-alex.bennee@linaro.org>

diff --git a/tests/docker/dockerfiles/debian-riscv64-cross.docker b/tests/docker/dockerfiles/debian-riscv64-cross.docker
index 4d8ca83cb3..b0386cd3a1 100644
--- a/tests/docker/dockerfiles/debian-riscv64-cross.docker
+++ b/tests/docker/dockerfiles/debian-riscv64-cross.docker
@@ -1,10 +1,10 @@
 # THIS FILE WAS AUTO-GENERATED
 #
-#  $ lcitool dockerfile --layers all --cross-arch riscv64 debian-sid qemu-minimal
+#  $ lcitool dockerfile --layers all --cross-arch riscv64 debian-13 qemu-minimal
 #
 # https://gitlab.com/libvirt/libvirt-ci
 
-FROM docker.io/library/debian:sid-slim
+FROM docker.io/library/debian:trixie-slim
 
 RUN export DEBIAN_FRONTEND=noninteractive && \
     apt-get update && \
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index 6720516b94..53f8d2585f 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -220,7 +220,9 @@ try:
                         trailer=cross_build("powerpc64le-linux-gnu-",
                                             "ppc64-softmmu,ppc64-linux-user"))
 
-    generate_dockerfile("debian-riscv64-cross", "debian-sid",
+    # while not yet a release architecture the packages are still
+    # build while part of testing
+    generate_dockerfile("debian-riscv64-cross", "debian-13",
                         project="qemu-minimal",
                         cross="riscv64",
                         trailer=cross_build("riscv64-linux-gnu-",
-- 
2.39.5

[PULL 26/32] tests/lcitool: remove temp workaround for debian mips64el

[Alex Bennée]

From: Daniel P. Berrangé <berrange@redhat.com>

The workaround applied in

  commit c60473d29254b79d9437eface8b342e84663ba66
  Author: Alex Bennée <alex.bennee@linaro.org>
  Date:   Wed Oct 2 10:03:33 2024 +0200

    testing: bump mips64el cross to bookworm and fix package list

Is no longer required since the affected builds are now fixed.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20241217133525.3836570-1-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-27-alex.bennee@linaro.org>

diff --git a/tests/docker/dockerfiles/debian-mips64el-cross.docker b/tests/docker/dockerfiles/debian-mips64el-cross.docker
index c09a8da890..9f6c4763c5 100644
--- a/tests/docker/dockerfiles/debian-mips64el-cross.docker
+++ b/tests/docker/dockerfiles/debian-mips64el-cross.docker
@@ -93,13 +93,18 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
                       libcmocka-dev:mips64el \
                       libcurl4-gnutls-dev:mips64el \
                       libdaxctl-dev:mips64el \
+                      libdrm-dev:mips64el \
+                      libepoxy-dev:mips64el \
                       libfdt-dev:mips64el \
                       libffi-dev:mips64el \
                       libfuse3-dev:mips64el \
+                      libgbm-dev:mips64el \
                       libgcrypt20-dev:mips64el \
                       libglib2.0-dev:mips64el \
                       libglusterfs-dev:mips64el \
                       libgnutls28-dev:mips64el \
+                      libgtk-3-dev:mips64el \
+                      libgtk-vnc-2.0-dev:mips64el \
                       libibverbs-dev:mips64el \
                       libiscsi-dev:mips64el \
                       libjemalloc-dev:mips64el \
@@ -119,6 +124,8 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
                       librbd-dev:mips64el \
                       librdmacm-dev:mips64el \
                       libsasl2-dev:mips64el \
+                      libsdl2-dev:mips64el \
+                      libsdl2-image-dev:mips64el \
                       libseccomp-dev:mips64el \
                       libselinux1-dev:mips64el \
                       libslirp-dev:mips64el \
@@ -134,6 +141,8 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
                       libusb-1.0-0-dev:mips64el \
                       libusbredirhost-dev:mips64el \
                       libvdeplug-dev:mips64el \
+                      libvirglrenderer-dev:mips64el \
+                      libvte-2.91-dev:mips64el \
                       libxdp-dev:mips64el \
                       libzstd-dev:mips64el \
                       nettle-dev:mips64el \
diff --git a/tests/lcitool/mappings.yml b/tests/lcitool/mappings.yml
index f8186b0e69..74eb13d62b 100644
--- a/tests/lcitool/mappings.yml
+++ b/tests/lcitool/mappings.yml
@@ -6,23 +6,6 @@ mappings:
   flake8:
     OpenSUSELeap15:
 
-  # Due to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1081535 we
-  # have to disable all packages that depend on libgl1-mesa-dri:mips64el
-  gtk3:
-    mips64el-deb:
-
-  libdrm:
-    mips64el-deb:
-
-  libepoxy:
-    mips64el-deb:
-
-  gtk-vnc:
-    mips64el-deb:
-
-  mesa-libgbm:
-    mips64el-deb:
-
   meson:
     OpenSUSELeap15:
 
@@ -81,18 +64,6 @@ mappings:
   python3-wheel:
     OpenSUSELeap15: python311-pip
 
-  sdl2:
-    mips64el-deb:
-
-  sdl2-image:
-    mips64el-deb:
-
-  virglrenderer:
-    mips64el-deb:
-
-  vte:
-    mips64el-deb:
-
 pypi_mappings:
   # Request more recent version
   meson:
-- 
2.39.5

[PULL 27/32] tests/vm: fix build_path based path

[Alex Bennée]

We no longer need to go into the per-arch build directories to find
the build directories binary. Lets call it directly.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-28-alex.bennee@linaro.org>

diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 4a1af04b9a..6f3f2e76df 100644
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -520,8 +520,7 @@ def get_qemu_path(arch, build_path=None):
     if "QEMU" in os.environ:
         qemu_path = os.environ["QEMU"]
     elif build_path:
-        qemu_path = os.path.join(build_path, arch + "-softmmu")
-        qemu_path = os.path.join(qemu_path, "qemu-system-" + arch)
+        qemu_path = os.path.join(build_path, "qemu-system-" + arch)
     else:
         # Default is to use system path for qemu.
         qemu_path = "qemu-system-" + arch
-- 
2.39.5

[PULL 28/32] tests/vm: partially un-tabify help output

[Alex Bennée]

While the make syntax itself uses tabs having a mixture of tabs and
spaces in the vm-help output make no sense and confuses things lining
up between terminal and editor. Fix that.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-29-alex.bennee@linaro.org>

diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
index 13ed80f72d..d80ca79a28 100644
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -64,23 +64,23 @@ endif
 	@echo "  vm-boot-ssh-<guest>             - Boot guest and login via ssh"
 	@echo
 	@echo "Special variables:"
-	@echo "    BUILD_TARGET=foo		 - Override the build target"
-	@echo "    DEBUG=1              	 - Enable verbose output on host and interactive debugging"
-	@echo '    EXTRA_CONFIGURE_OPTS="..."   - Pass to configure step'
-	@echo "    J=[0..9]*            	 - Override the -jN parameter for make commands"
-	@echo "    LOG_CONSOLE=1        	 - Log console to file in: ~/.cache/qemu-vm "
-	@echo "    USE_TCG=1        	         - Use TCG for cross-arch images"
-	@echo "    QEMU=/path/to/qemu		 - Change path to QEMU binary"
+	@echo "    BUILD_TARGET=foo              - Override the build target"
+	@echo "    DEBUG=1                       - Enable verbose output on host and interactive debugging"
+	@echo '    EXTRA_CONFIGURE_OPTS="..."    - Pass to configure step'
+	@echo "    J=[0..9]*                     - Override the -jN parameter for make commands"
+	@echo "    LOG_CONSOLE=1                 - Log console to file in: ~/.cache/qemu-vm "
+	@echo "    USE_TCG=1                     - Use TCG for cross-arch images"
+	@echo "    QEMU=/path/to/qemu            - Change path to QEMU binary"
 ifeq ($(HAVE_PYTHON_YAML),yes)
-	@echo "    QEMU_CONFIG=/path/conf.yml   - Change path to VM configuration .yml file."
+	@echo "    QEMU_CONFIG=/path/conf.yml    - Change path to VM configuration .yml file."
 else
 	@echo "    (install python3-yaml to enable support for yaml file to configure a VM.)"
 endif
-	@echo "                                   See conf_example_*.yml for file format details."
-	@echo "    QEMU_IMG=/path/to/qemu-img	 - Change path to qemu-img tool"
-	@echo "    QEMU_LOCAL=1                 - Use QEMU binary local to this build."
-	@echo "    TARGET_LIST=a,b,c    	 - Override target list in builds"
-	@echo "    V=1				 - Enable verbose output on host and guest commands"
+	@echo "                                    See conf_example_*.yml for file format details."
+	@echo "    QEMU_IMG=/path/to/qemu-img    - Change path to qemu-img tool"
+	@echo "    QEMU_LOCAL=1                  - Use QEMU binary local to this build."
+	@echo "    TARGET_LIST=a,b,c             - Override target list in builds"
+	@echo "    V=1                           - Enable verbose output on host and guest commands"
 
 vm-build-all: $(addprefix vm-build-, $(IMAGES))
 
-- 
2.39.5

[PULL 29/32] tests/vm: allow interactive login as root

[Alex Bennée]

This is useful when debugging and you want to add packages to an
image.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-30-alex.bennee@linaro.org>

diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
index d80ca79a28..14188bba1c 100644
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -66,6 +66,7 @@ endif
 	@echo "Special variables:"
 	@echo "    BUILD_TARGET=foo              - Override the build target"
 	@echo "    DEBUG=1                       - Enable verbose output on host and interactive debugging"
+	@echo "    ROOT_USER=1                   - Login as root user for interactive shell"
 	@echo '    EXTRA_CONFIGURE_OPTS="..."    - Pass to configure step'
 	@echo "    J=[0..9]*                     - Override the -jN parameter for make commands"
 	@echo "    LOG_CONSOLE=1                 - Log console to file in: ~/.cache/qemu-vm "
@@ -141,6 +142,6 @@ vm-boot-ssh-%: $(IMAGES_DIR)/%.img $(VM_VENV)
 		$(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \
 		$(if $(LOG_CONSOLE),--log-console) \
 		--image "$<" \
-		--interactive \
+		$(if $(ROOT_USER),--interactive-root,-interactive) \
 		false, \
 		"  VM-BOOT-SSH $*") || true
diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 6f3f2e76df..6d41ac7574 100644
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -612,8 +612,11 @@ def get_default_jobs():
     parser.add_argument("--source-path", default=None,
                         help="Path of source directory, "\
                         "for finding additional files. ")
-    parser.add_argument("--interactive", "-I", action="store_true",
-                        help="Interactively run command")
+    int_ops = parser.add_mutually_exclusive_group()
+    int_ops.add_argument("--interactive", "-I", action="store_true",
+                         help="Interactively run command")
+    int_ops.add_argument("--interactive-root", action="store_true",
+                         help="Interactively run command as root")
     parser.add_argument("--snapshot", "-s", action="store_true",
                         help="run tests with a snapshot")
     parser.add_argument("--genisoimage", default="genisoimage",
@@ -675,6 +678,8 @@ def main(vmcls, config=None):
         exitcode = 3
     if args.interactive:
         vm.ssh()
+    elif args.interactive_root:
+        vm.ssh_root()
 
     if not args.snapshot:
         vm.graceful_shutdown()
-- 
2.39.5

[PULL 30/32] pc-bios: ensure keymaps dependencies set vnc tests

[Alex Bennée]

I was seeing failures on vnc-display-test on FreeBSD:

  make vm-build-freebsd V=1 TARGET_LIST=aarch64-softmmu BUILD_TARGET=check-qtest QEMU_LOCAL=1 DEBUG=1

Leads to:

  qemu-system-aarch64: -vnc none: could not read keymap file: 'en-us'
  Broken pipe
  ../src/tests/qtest/libqtest.c:196: kill_qemu() tried to terminate QEMU process but encountered exit status 1 (expected 0)

which was as far as I could tell because we don't populate the
$BLD/pc-bios/keymaps (although scripts/symlink-install-tree.py
attempts to symlink qemu-bundle/usr/local/share/qemu/keymaps/ to that
dir).

Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-31-alex.bennee@linaro.org>

diff --git a/pc-bios/keymaps/meson.build b/pc-bios/keymaps/meson.build
index 0bd8ce0077..a79a09b276 100644
--- a/pc-bios/keymaps/meson.build
+++ b/pc-bios/keymaps/meson.build
@@ -39,19 +39,18 @@ else
   native_qemu_keymap = qemu_keymap
 endif
 
+keymap_targets = []
 if native_qemu_keymap.found()
-  t = []
   foreach km, args: keymaps
     # generate with qemu-kvm
-    t += custom_target(km,
-                       build_by_default: true,
-                       output: km,
-                       command: [native_qemu_keymap, '-f', '@OUTPUT@', args.split()],
-                       install: have_system,
-                       install_dir: qemu_datadir / 'keymaps')
+    keymap_targets += custom_target(km,
+                                    build_by_default: true,
+                                    output: km,
+                                    command: [native_qemu_keymap, '-f', '@OUTPUT@', args.split()],
+                                    install: have_system,
+                                    install_dir: qemu_datadir / 'keymaps')
   endforeach
-
-  alias_target('update-keymaps', t)
+  alias_target('update-keymaps', keymap_targets)
 else
   install_data(keymaps.keys(), install_dir: qemu_datadir / 'keymaps')
 endif
diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build
index c5a70021c5..f75c1057a4 100644
--- a/tests/qtest/meson.build
+++ b/tests/qtest/meson.build
@@ -383,7 +383,7 @@ qtests = {
 if vnc.found()
   gvnc = dependency('gvnc-1.0', method: 'pkg-config', required: false)
   if gvnc.found()
-    qtests += {'vnc-display-test': [gvnc]}
+    qtests += {'vnc-display-test': [gvnc, keymap_targets]}
     qtests_generic += [ 'vnc-display-test' ]
   endif
 endif
-- 
2.39.5

[PULL 31/32] dockerfiles: Remove 'MAINTAINER' entry in debian-tricore-cross.docker

[Alex Bennée]

From: Philippe Mathieu-Daudé <f4bug@amsat.org>

AMSAT closed its email service [*] so my personal email
address is now defunct. Remove it to avoid bouncing emails.

[*] https://web.archive.org/web/20240617194936/https://forum.amsat-dl.org/index.php?thread/4581-amsat-mail-alias-service-to-end-august-1-2024/

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20250102152513.61065-1-philmd@linaro.org>
[AJB: update URL to web.archive.org]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-32-alex.bennee@linaro.org>

diff --git a/tests/docker/dockerfiles/debian-tricore-cross.docker b/tests/docker/dockerfiles/debian-tricore-cross.docker
index 479b4d6eba..7e00e870ce 100644
--- a/tests/docker/dockerfiles/debian-tricore-cross.docker
+++ b/tests/docker/dockerfiles/debian-tricore-cross.docker
@@ -11,8 +11,6 @@
 #
 FROM docker.io/library/debian:11-slim
 
-MAINTAINER Philippe Mathieu-Daudé <f4bug@amsat.org>
-
 RUN apt update && \
     DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
     DEBIAN_FRONTEND=noninteractive eatmydata apt install -yy \
-- 
2.39.5

[PULL 32/32] MAINTAINERS: Remove myself from reviewers

[Alex Bennée]

From: Wainer dos Santos Moschetta <wainersm@redhat.com>

The time I spent contributing to QEMU was great, but I've not been active
for a long time now. So removing myself from the reviewers list of
"Integration Testing with the Avocado framework" and
"Build and test automation" subsystems.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20250103194450.360789-1-wainersm@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-33-alex.bennee@linaro.org>

diff --git a/MAINTAINERS b/MAINTAINERS
index cab9018d9f..c1d954c9de 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -4152,7 +4152,6 @@ M: Alex Bennée <alex.bennee@linaro.org>
 T: git https://gitlab.com/stsquad/qemu testing/next
 M: Philippe Mathieu-Daudé <philmd@linaro.org>
 M: Thomas Huth <thuth@redhat.com>
-R: Wainer dos Santos Moschetta <wainersm@redhat.com>
 S: Maintained
 F: .github/workflows/lockdown.yml
 F: .gitlab-ci.yml
@@ -4198,7 +4197,6 @@ Integration Testing with the Avocado framework
 W: https://trello.com/b/6Qi1pxVn/avocado-qemu
 R: Cleber Rosa <crosa@redhat.com>
 R: Philippe Mathieu-Daudé <philmd@linaro.org>
-R: Wainer dos Santos Moschetta <wainersm@redhat.com>
 S: Odd Fixes
 F: tests/avocado/
 
-- 
2.39.5

Re: [PULL 00/32] testing updates for 10.0 (tuxrun, libvirt, dockerfiles, misc)

[Stefan Hajnoczi]

On Fri, 10 Jan 2025 at 08:22, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> The following changes since commit bc6afa1c711da5b4f37c9685a812c77b114d84cb:
>
>   Merge tag 'pull-xenfv-20250109-1' of https://gitlab.com/dwmw2/qemu into staging (2025-01-09 08:39:32 -0500)
>
> are available in the Git repository at:
>
>   https://gitlab.com/stsquad/qemu.git tags/pull-10.0-testing-updates-100125-1
>
> for you to fetch changes up to 70b7c929ba39f15a94fbc63d9be88e8da907cc46:
>
>   MAINTAINERS: Remove myself from reviewers (2025-01-10 11:18:38 +0000)
>
> ----------------------------------------------------------------
> Testing updates for 10.0
>
>   - update the tuxrun images to the latest baseline
>   - add the m68k tuxrun test
>   - ensure qtest checks the result of clock_step operations
>   - introduce new ztsd helper to functional tests
>   - ensure aarch64_virt test exits early when no TCG
>   - add new test to exercise virtio-vulkan

Hi Alex,
Please take a look at the following CI job failure and send a new
revision of this pull request if something needs to be fixed:
https://gitlab.com/qemu-project/qemu/-/jobs/8822037880#L593

Thanks,
Stefan

>   - bump libvirt-ci to latest version
>   - move riscv64 cross container from sid to trixie
>   - remove workaround from mips containers now upstream updated
>   - fix VM tests to use correct path for local QEMU binary
>   - add ability to get a root debug shell on VM images
>   - add keymap dependency to vnc tests
>   - remove retiring maintainers from avocado and dockerfiles
>
> ----------------------------------------------------------------
> Alex Bennée (28):
>       tests/functional: update the arm tuxrun tests
>       tests/functional: update the i386 tuxrun tests
>       tests/functional: add a m68k tuxrun tests
>       tests/functional: update the mips32 tuxrun tests
>       tests/functional: update the mips32el tuxrun tests
>       tests/functional: update the mips64 tuxrun tests
>       tests/functional: update the mips64el tuxrun tests
>       tests/functional: update the ppc32 tuxrun tests
>       tests/functional: update the ppc64 tuxrun tests
>       tests/functional: update the riscv32 tuxrun tests
>       tests/functional: update the riscv64 tuxrun tests
>       tests/functional: update the s390x tuxrun tests
>       tests/functional: update the sparc64 tuxrun tests
>       tests/functional: update the x86_64 tuxrun tests
>       tests/qtest: remove clock_steps from virtio tests
>       system/qtest: properly feedback results of clock_[step|set]
>       tests/functional: remove hacky sleep from the tests
>       tests/functional: add zstd support to uncompress utility
>       tests/functional: update tuxruntest to use uncompress utility
>       tests/functional: remove unused kernel_command_line
>       tests/functional: bail aarch64_virt tests early if missing TCG
>       tests/functional: extend test_aarch64_virt with vulkan test
>       tests/lcitool: bump to latest version of libvirt-ci
>       tests/docker: move riscv64 cross container from sid to trixie
>       tests/vm: fix build_path based path
>       tests/vm: partially un-tabify help output
>       tests/vm: allow interactive login as root
>       pc-bios: ensure keymaps dependencies set vnc tests
>
> Daniel P. Berrangé (1):
>       tests/lcitool: remove temp workaround for debian mips64el
>
> Philippe Mathieu-Daudé (1):
>       dockerfiles: Remove 'MAINTAINER' entry in debian-tricore-cross.docker
>
> Pierrick Bouvier (1):
>       tests/functional/aarch64: add tests for FEAT_RME
>
> Wainer dos Santos Moschetta (1):
>       MAINTAINERS: Remove myself from reviewers
>
>  MAINTAINERS                                        |   3 +-
>  system/qtest.c                                     |  23 ++--
>  tests/qtest/libqos/virtio.c                        |   4 -
>  .gitlab-ci.d/cirrus/freebsd-14.vars                |   2 +-
>  pc-bios/keymaps/meson.build                        |  17 ++-
>  .../dockerfiles/debian-mips64el-cross.docker       |   9 ++
>  .../docker/dockerfiles/debian-riscv64-cross.docker |   4 +-
>  .../docker/dockerfiles/debian-tricore-cross.docker |   2 -
>  tests/functional/meson.build                       |   5 +
>  tests/functional/qemu_test/tuxruntest.py           |  12 +--
>  tests/functional/qemu_test/uncompress.py           |  24 +++++
>  tests/functional/test_aarch64_rme_sbsaref.py       |  69 ++++++++++++
>  tests/functional/test_aarch64_rme_virt.py          |  98 +++++++++++++++++
>  tests/functional/test_aarch64_virt.py              | 117 +++++++++++++++++----
>  tests/functional/test_arm_tuxrun.py                |  28 ++---
>  tests/functional/test_i386_tuxrun.py               |   8 +-
>  tests/functional/test_m68k_tuxrun.py               |  34 ++++++
>  tests/functional/test_mips64_tuxrun.py             |   8 +-
>  tests/functional/test_mips64el_tuxrun.py           |   8 +-
>  tests/functional/test_mips_tuxrun.py               |   8 +-
>  tests/functional/test_mipsel_tuxrun.py             |   8 +-
>  tests/functional/test_ppc64_tuxrun.py              |  16 +--
>  tests/functional/test_ppc_tuxrun.py                |   8 +-
>  tests/functional/test_riscv32_tuxrun.py            |   8 +-
>  tests/functional/test_riscv64_tuxrun.py            |  16 +--
>  tests/functional/test_s390x_tuxrun.py              |   8 +-
>  tests/functional/test_sparc64_tuxrun.py            |   8 +-
>  tests/functional/test_x86_64_tuxrun.py             |   8 +-
>  tests/lcitool/libvirt-ci                           |   2 +-
>  tests/lcitool/mappings.yml                         |  29 -----
>  tests/lcitool/refresh                              |   4 +-
>  tests/qtest/meson.build                            |   2 +-
>  tests/vm/Makefile.include                          |  29 ++---
>  tests/vm/basevm.py                                 |  12 ++-
>  tests/vm/generated/freebsd.json                    |   2 +-
>  35 files changed, 461 insertions(+), 182 deletions(-)
>  create mode 100755 tests/functional/test_aarch64_rme_sbsaref.py
>  create mode 100755 tests/functional/test_aarch64_rme_virt.py
>  create mode 100755 tests/functional/test_m68k_tuxrun.py
>
> --
> 2.39.5
>
>

Re: [PULL 00/32] testing updates for 10.0 (tuxrun, libvirt, dockerfiles, misc)

[Alex Bennée]

Stefan Hajnoczi <stefanha@gmail.com> writes:

> On Fri, 10 Jan 2025 at 08:22, Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>> The following changes since commit bc6afa1c711da5b4f37c9685a812c77b114d84cb:
>>
>>   Merge tag 'pull-xenfv-20250109-1' of https://gitlab.com/dwmw2/qemu into staging (2025-01-09 08:39:32 -0500)
>>
>> are available in the Git repository at:
>>
>>   https://gitlab.com/stsquad/qemu.git tags/pull-10.0-testing-updates-100125-1
>>
>> for you to fetch changes up to 70b7c929ba39f15a94fbc63d9be88e8da907cc46:
>>
>>   MAINTAINERS: Remove myself from reviewers (2025-01-10 11:18:38 +0000)
>>
>> ----------------------------------------------------------------
>> Testing updates for 10.0
>>
>>   - update the tuxrun images to the latest baseline
>>   - add the m68k tuxrun test
>>   - ensure qtest checks the result of clock_step operations
>>   - introduce new ztsd helper to functional tests
>>   - ensure aarch64_virt test exits early when no TCG
>>   - add new test to exercise virtio-vulkan
>
> Hi Alex,
> Please take a look at the following CI job failure and send a new
> revision of this pull request if something needs to be fixed:
> https://gitlab.com/qemu-project/qemu/-/jobs/8822037880#L593

Ahh the acceptance tests don't run automatically on my CI run. The
problem was the opensuse image doesn't see the DRI node. I've tweaked
the failure leg:

        try:
            self.vm.launch()
        except VMLaunchFailure as excp:
            if "old virglrenderer, blob resources unsupported" in excp.output:
                self.skipTest("No blob support for virtio-gpu")
            elif "old virglrenderer, venus unsupported" in excp.output:
                self.skipTest("No venus support for virtio-gpu")
            elif "egl: no drm render node available" in excp.output:
                self.skipTest("Can't access host DRM render node")
            else:
                self.log.info(f"unhandled launch failure: {excp.output}")
                raise excp

so it now catches that (and also fixes a small bug with the unhandled
case). I've posted v2 of the PR.

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [PULL 15/32] tests/functional/aarch64: add tests for FEAT_RME

[Thomas Huth]

On 10/01/2025 14.17, Alex Bennée wrote:
> From: Pierrick Bouvier <pierrick.bouvier@linaro.org>
> 
> This boot an OP-TEE environment, and launch a nested guest VM inside it
> using the Realms feature. We do it for virt and sbsa-ref platforms.

  Hi,

FWIW, I just saw this test_aarch64_rme_virt test failing once. Looking at 
the console.log, it seems like the guest crashed somewhere at the end of the 
firmware or very early in the kernel:

EFI stub: Booting Linux Kernel...
2025-01-24 13:25:28,226:
2025-01-24 13:25:28,226:
2025-01-24 13:25:28,227: Synchronous Exception at 0x00000000BF526498
2025-01-24 13:25:28,227:
2025-01-24 13:25:28,227:
2025-01-24 13:25:28,228: Synchronous Exception at 0x00000000BF526498

I wasn't able to reproduce it afterwards anymore, but in case somebody runs 
into this later again and wonders if it is a later regression: It happened 
for me with commit cf86770c7aa31ebd.

  Thomas

Re: [PULL 15/32] tests/functional/aarch64: add tests for FEAT_RME

[Pierrick Bouvier]

Hi Thomas,

On 1/24/25 05:29, Thomas Huth wrote:
> On 10/01/2025 14.17, Alex Bennée wrote:
>> From: Pierrick Bouvier <pierrick.bouvier@linaro.org>
>>
>> This boot an OP-TEE environment, and launch a nested guest VM inside it
>> using the Realms feature. We do it for virt and sbsa-ref platforms.
> 
>    Hi,
> 
> FWIW, I just saw this test_aarch64_rme_virt test failing once. Looking at
> the console.log, it seems like the guest crashed somewhere at the end of the
> firmware or very early in the kernel:
> 
> EFI stub: Booting Linux Kernel...
> 2025-01-24 13:25:28,226:
> 2025-01-24 13:25:28,226:
> 2025-01-24 13:25:28,227: Synchronous Exception at 0x00000000BF526498
> 2025-01-24 13:25:28,227:
> 2025-01-24 13:25:28,227:
> 2025-01-24 13:25:28,228: Synchronous Exception at 0x00000000BF526498
> 
> I wasn't able to reproduce it afterwards anymore, but in case somebody runs
> into this later again and wonders if it is a later regression: It happened
> for me with commit cf86770c7aa31ebd.
> 
>    Thomas
> 

I met this kind of error before (not on rme), and if I remember well, 
it's coming from EDK2.

We might have a race condition here.

Re: [PULL 15/32] tests/functional/aarch64: add tests for FEAT_RME

[Alex Bennée]

Pierrick Bouvier <pierrick.bouvier@linaro.org> writes:

> Hi Thomas,
>
> On 1/24/25 05:29, Thomas Huth wrote:
>> On 10/01/2025 14.17, Alex Bennée wrote:
>>> From: Pierrick Bouvier <pierrick.bouvier@linaro.org>
>>>
>>> This boot an OP-TEE environment, and launch a nested guest VM inside it
>>> using the Realms feature. We do it for virt and sbsa-ref platforms.
>>    Hi,
>> FWIW, I just saw this test_aarch64_rme_virt test failing once.
>> Looking at
>> the console.log, it seems like the guest crashed somewhere at the end of the
>> firmware or very early in the kernel:
>> EFI stub: Booting Linux Kernel...
>> 2025-01-24 13:25:28,226:
>> 2025-01-24 13:25:28,226:
>> 2025-01-24 13:25:28,227: Synchronous Exception at 0x00000000BF526498
>> 2025-01-24 13:25:28,227:
>> 2025-01-24 13:25:28,227:
>> 2025-01-24 13:25:28,228: Synchronous Exception at 0x00000000BF526498
>> I wasn't able to reproduce it afterwards anymore, but in case
>> somebody runs
>> into this later again and wonders if it is a later regression: It happened
>> for me with commit cf86770c7aa31ebd.
>>    Thomas
>> 
>
> I met this kind of error before (not on rme), and if I remember well,
> it's coming from EDK2.
>
> We might have a race condition here.

I'm measuring a 1-3% hit rates:

  retry.py -n 400 -c -- ./pyvenv/bin/meson test --setup thorough --suite func-thorough func-aarch64-aarch64_rme_virt func-aarch64-aarch64_rme_sbsaref -v

Gave:

  Results summary:
  0: 388 times (97.00%), avg time 196.924 (7.99 varience/2.83 deviation)
  1: 12 times (3.00%), avg time 1205.900 (0.00 varience/0.01 deviation)
  Ran command 400 times, 388 passes

It might be worth trying it under rr --chaos mode and seeing if that
shakes anything out.

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [PULL 23/32] tests/functional: extend test_aarch64_virt with vulkan test

[Peter Maydell]

On Fri, 10 Jan 2025 at 13:23, Alex Bennée <alex.bennee@linaro.org> wrote:
> Now that we have virtio-gpu Vulkan support, let's add a test for it.
> Currently this is using images build by buildroot:
>
>   https://lists.buildroot.org/pipermail/buildroot/2024-December/768196.html
>
> Reviewed-by: Thomas Huth <thuth@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20250108121054.1126164-24-alex.bennee@linaro.org>

Hi; this test currently fails for me with a clang sanitizer
build (ubuntu 24.04 host). It seems to run weston in the guest,
which fails with:

2025-02-17 16:11:10,218: [16:11:10.672] Command line: weston -B
headless --renderer gl --shell kiosk -- vkmark -b:duration=1.0
2025-02-17 16:11:10,224: [16:11:10.675] OS: Linux, 6.11.10, #2 SMP Thu
Dec  5 16:27:12 GMT 2024, aarch64
2025-02-17 16:11:10,225: [16:11:10.680] Flight recorder: enabled
2025-02-17 16:11:10,226: [16:11:10.681] warning: XDG_RUNTIME_DIR
"/tmp" is not configured
2025-02-17 16:11:10,226: correctly.  Unix access mode must be 0700
(current mode is 0777),
2025-02-17 16:11:10,226: and must be owned by the user UID 0 (current
owner is UID 0).
2025-02-17 16:11:10,227: Refer to your distribution on how to get it, or
2025-02-17 16:11:10,227:
http://www.freedesktop.org/wiki/Specifications/basedir-spec
2025-02-17 16:11:10,228: on how to implement it.
2025-02-17 16:11:10,240: [16:11:10.695] Starting with no config file.
2025-02-17 16:11:10,253: [16:11:10.707] Output repaint window is 7 ms maximum.
2025-02-17 16:11:10,262: [16:11:10.716] Loading module
'/usr/lib/libweston-14/headless-backend.so'
2025-02-17 16:11:10,313: [16:11:10.768] Loading module
'/usr/lib/libweston-14/gl-renderer.so'
2025-02-17 16:11:21,858: libEGL warning: egl: failed to create dri2 screen
2025-02-17 16:11:21,959: libEGL warning: egl: failed to create dri2 screen
2025-02-17 16:11:22,023: libEGL warning: egl: failed to create dri2 screen
2025-02-17 16:11:22,032: [16:11:22.486] failed to initialize display
2025-02-17 16:11:22,033: [16:11:22.488] EGL error state:
EGL_NOT_INITIALIZED (0x3001)
2025-02-17 16:11:22,036: [16:11:22.490] fatal: failed to create
compositor backend

Then eventually the test framework times it ou and sends it
a SIGTERM, and QEMU SEGVs inside libEGL trying to run an
exit handler:

qemu-system-aarch64: terminating on signal 15 from pid 242824
(/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/pyvenv/bin/python3)
UndefinedBehaviorSanitizer:DEADLYSIGNAL
==243045==ERROR: UndefinedBehaviorSanitizer: SEGV on unknown address
0x73fbfefe6a31 (pc 0x73fbba9788e9 bp 0x73fbbbe0af80 sp 0x7ffd676fbfe0
T243045)
==243045==The signal is caused by a READ memory access.
    #0 0x73fbba9788e9
(/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x15788e9)
(BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
    #1 0x73fbbaafc178
(/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x16fc178)
(BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
    #2 0x73fbba62564f
(/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x122564f)
(BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
    #3 0x73fbbab067d7
(/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x17067d7)
(BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
    #4 0x73fbba63b786
(/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x123b786)
(BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
    #5 0x73fbba96290a
(/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x156290a)
(BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
    #6 0x73fbba941c5c
(/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x1541c5c)
(BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
    #7 0x73fbc2041f20
(/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0x41f20) (BuildId:
6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
    #8 0x73fbc2041f68
(/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0x41f68) (BuildId:
6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
    #9 0x73fbc2034ca9
(/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0x34ca9) (BuildId:
6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
    #10 0x73fbc203ae90
(/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0x3ae90) (BuildId:
6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
    #11 0x73fbc203aeda
(/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0x3aeda) (BuildId:
6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
    #12 0x73fbc20a45f5
(/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0xa45f5) (BuildId:
6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
    #13 0x73fbc20a2bfc
(/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0xa2bfc) (BuildId:
6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
    #14 0x73fbd3047a75 in __run_exit_handlers stdlib/exit.c:108:8
    #15 0x73fbd3047bbd in exit stdlib/exit.c:138:3
    #16 0x5a5bab5e3fdb in qemu_default_main
/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/../../system/main.c:52:5
    #17 0x5a5bab5e3f9e in main
/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/../../system/main.c:76:9
    #18 0x73fbd302a1c9 in __libc_start_call_main
csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #19 0x73fbd302a28a in __libc_start_main csu/../csu/libc-start.c:360:3
    #20 0x5a5ba9c5b554 in _start
(/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/qemu-system-aarch64+0x15dc554)
(BuildId: 8efda3601b42aa2644dde35d1d63f7b22b649a33)

UndefinedBehaviorSanitizer can not provide additional info.
SUMMARY: UndefinedBehaviorSanitizer: SEGV
(/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x15788e9)
(BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
==243045==ABORTING

thanks
-- PMM

Re: [PULL 23/32] tests/functional: extend test_aarch64_virt with vulkan test

[Alex Bennée]

Peter Maydell <peter.maydell@linaro.org> writes:

> On Fri, 10 Jan 2025 at 13:23, Alex Bennée <alex.bennee@linaro.org> wrote:
>> Now that we have virtio-gpu Vulkan support, let's add a test for it.
>> Currently this is using images build by buildroot:
>>
>>   https://lists.buildroot.org/pipermail/buildroot/2024-December/768196.html
>>
>> Reviewed-by: Thomas Huth <thuth@redhat.com>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> Message-Id: <20250108121054.1126164-24-alex.bennee@linaro.org>
>
> Hi; this test currently fails for me with a clang sanitizer
> build (ubuntu 24.04 host). It seems to run weston in the guest,
> which fails with:
>
> 2025-02-17 16:11:10,218: [16:11:10.672] Command line: weston -B
> headless --renderer gl --shell kiosk -- vkmark -b:duration=1.0
> 2025-02-17 16:11:10,224: [16:11:10.675] OS: Linux, 6.11.10, #2 SMP Thu
> Dec  5 16:27:12 GMT 2024, aarch64
> 2025-02-17 16:11:10,225: [16:11:10.680] Flight recorder: enabled
> 2025-02-17 16:11:10,226: [16:11:10.681] warning: XDG_RUNTIME_DIR
> "/tmp" is not configured
> 2025-02-17 16:11:10,226: correctly.  Unix access mode must be 0700
> (current mode is 0777),
> 2025-02-17 16:11:10,226: and must be owned by the user UID 0 (current
> owner is UID 0).
> 2025-02-17 16:11:10,227: Refer to your distribution on how to get it, or
> 2025-02-17 16:11:10,227:
> http://www.freedesktop.org/wiki/Specifications/basedir-spec
> 2025-02-17 16:11:10,228: on how to implement it.
> 2025-02-17 16:11:10,240: [16:11:10.695] Starting with no config file.
> 2025-02-17 16:11:10,253: [16:11:10.707] Output repaint window is 7 ms maximum.
> 2025-02-17 16:11:10,262: [16:11:10.716] Loading module
> '/usr/lib/libweston-14/headless-backend.so'
> 2025-02-17 16:11:10,313: [16:11:10.768] Loading module
> '/usr/lib/libweston-14/gl-renderer.so'
> 2025-02-17 16:11:21,858: libEGL warning: egl: failed to create dri2 screen
> 2025-02-17 16:11:21,959: libEGL warning: egl: failed to create dri2 screen
> 2025-02-17 16:11:22,023: libEGL warning: egl: failed to create dri2 screen
> 2025-02-17 16:11:22,032: [16:11:22.486] failed to initialize display
> 2025-02-17 16:11:22,033: [16:11:22.488] EGL error state:
> EGL_NOT_INITIALIZED (0x3001)
> 2025-02-17 16:11:22,036: [16:11:22.490] fatal: failed to create
> compositor backend

I guess we can catch that in the test case and fail the test.

>
> Then eventually the test framework times it ou and sends it
> a SIGTERM, and QEMU SEGVs inside libEGL trying to run an
> exit handler:
>
> qemu-system-aarch64: terminating on signal 15 from pid 242824
> (/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/pyvenv/bin/python3)
> UndefinedBehaviorSanitizer:DEADLYSIGNAL
> ==243045==ERROR: UndefinedBehaviorSanitizer: SEGV on unknown address
> 0x73fbfefe6a31 (pc 0x73fbba9788e9 bp 0x73fbbbe0af80 sp 0x7ffd676fbfe0
> T243045)
> ==243045==The signal is caused by a READ memory access.
>     #0 0x73fbba9788e9
> (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x15788e9)
> (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
>     #1 0x73fbbaafc178
> (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x16fc178)
> (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
>     #2 0x73fbba62564f
> (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x122564f)
> (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
>     #3 0x73fbbab067d7
> (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x17067d7)
> (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
>     #4 0x73fbba63b786
> (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x123b786)
> (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
>     #5 0x73fbba96290a
> (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x156290a)
> (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
>     #6 0x73fbba941c5c
> (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x1541c5c)
> (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
>     #7 0x73fbc2041f20
> (/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0x41f20) (BuildId:
> 6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
>     #8 0x73fbc2041f68
> (/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0x41f68) (BuildId:
> 6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
>     #9 0x73fbc2034ca9
> (/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0x34ca9) (BuildId:
> 6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
>     #10 0x73fbc203ae90
> (/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0x3ae90) (BuildId:
> 6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
>     #11 0x73fbc203aeda
> (/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0x3aeda) (BuildId:
> 6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
>     #12 0x73fbc20a45f5
> (/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0xa45f5) (BuildId:
> 6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
>     #13 0x73fbc20a2bfc
> (/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0xa2bfc) (BuildId:
> 6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
>     #14 0x73fbd3047a75 in __run_exit_handlers stdlib/exit.c:108:8

Ok - interesting that we can corrupt the nVidia state so much it dies on
cleanup. I'll try and replicate with my intel setup.

>     #15 0x73fbd3047bbd in exit stdlib/exit.c:138:3
>     #16 0x5a5bab5e3fdb in qemu_default_main
> /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/../../system/main.c:52:5
>     #17 0x5a5bab5e3f9e in main
> /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/../../system/main.c:76:9
>     #18 0x73fbd302a1c9 in __libc_start_call_main
> csu/../sysdeps/nptl/libc_start_call_main.h:58:16
>     #19 0x73fbd302a28a in __libc_start_main csu/../csu/libc-start.c:360:3
>     #20 0x5a5ba9c5b554 in _start
> (/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/qemu-system-aarch64+0x15dc554)
> (BuildId: 8efda3601b42aa2644dde35d1d63f7b22b649a33)
>
> UndefinedBehaviorSanitizer can not provide additional info.
> SUMMARY: UndefinedBehaviorSanitizer: SEGV
> (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x15788e9)
> (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
> ==243045==ABORTING
>
> thanks
> -- PMM

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [PULL 23/32] tests/functional: extend test_aarch64_virt with vulkan test

[Thomas Huth]

On 17/02/2025 17.30, Peter Maydell wrote:
> On Fri, 10 Jan 2025 at 13:23, Alex Bennée <alex.bennee@linaro.org> wrote:
>> Now that we have virtio-gpu Vulkan support, let's add a test for it.
>> Currently this is using images build by buildroot:
>>
>>    https://lists.buildroot.org/pipermail/buildroot/2024-December/768196.html
>>
>> Reviewed-by: Thomas Huth <thuth@redhat.com>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> Message-Id: <20250108121054.1126164-24-alex.bennee@linaro.org>
> 
> Hi; this test currently fails for me with a clang sanitizer
> build (ubuntu 24.04 host). It seems to run weston in the guest,
> which fails with:
> 
> 2025-02-17 16:11:10,218: [16:11:10.672] Command line: weston -B
> headless --renderer gl --shell kiosk -- vkmark -b:duration=1.0
> 2025-02-17 16:11:10,224: [16:11:10.675] OS: Linux, 6.11.10, #2 SMP Thu
> Dec  5 16:27:12 GMT 2024, aarch64
> 2025-02-17 16:11:10,225: [16:11:10.680] Flight recorder: enabled
> 2025-02-17 16:11:10,226: [16:11:10.681] warning: XDG_RUNTIME_DIR
> "/tmp" is not configured
> 2025-02-17 16:11:10,226: correctly.  Unix access mode must be 0700
> (current mode is 0777),
> 2025-02-17 16:11:10,226: and must be owned by the user UID 0 (current
> owner is UID 0).
> 2025-02-17 16:11:10,227: Refer to your distribution on how to get it, or
> 2025-02-17 16:11:10,227:
> http://www.freedesktop.org/wiki/Specifications/basedir-spec
> 2025-02-17 16:11:10,228: on how to implement it.
> 2025-02-17 16:11:10,240: [16:11:10.695] Starting with no config file.
> 2025-02-17 16:11:10,253: [16:11:10.707] Output repaint window is 7 ms maximum.
> 2025-02-17 16:11:10,262: [16:11:10.716] Loading module
> '/usr/lib/libweston-14/headless-backend.so'
> 2025-02-17 16:11:10,313: [16:11:10.768] Loading module
> '/usr/lib/libweston-14/gl-renderer.so'
> 2025-02-17 16:11:21,858: libEGL warning: egl: failed to create dri2 screen
> 2025-02-17 16:11:21,959: libEGL warning: egl: failed to create dri2 screen
> 2025-02-17 16:11:22,023: libEGL warning: egl: failed to create dri2 screen
> 2025-02-17 16:11:22,032: [16:11:22.486] failed to initialize display
> 2025-02-17 16:11:22,033: [16:11:22.488] EGL error state:
> EGL_NOT_INITIALIZED (0x3001)
> 2025-02-17 16:11:22,036: [16:11:22.490] fatal: failed to create
> compositor backend
> 
> Then eventually the test framework times it ou and sends it
> a SIGTERM, and QEMU SEGVs inside libEGL trying to run an
> exit handler:
> 
> qemu-system-aarch64: terminating on signal 15 from pid 242824
> (/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/pyvenv/bin/python3)
> UndefinedBehaviorSanitizer:DEADLYSIGNAL
> ==243045==ERROR: UndefinedBehaviorSanitizer: SEGV on unknown address
> 0x73fbfefe6a31 (pc 0x73fbba9788e9 bp 0x73fbbbe0af80 sp 0x7ffd676fbfe0
> T243045)
> ==243045==The signal is caused by a READ memory access.
>      #0 0x73fbba9788e9
> (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x15788e9)
> (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
>      #1 0x73fbbaafc178
> (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x16fc178)
> (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
>      #2 0x73fbba62564f
> (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x122564f)
> (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
>      #3 0x73fbbab067d7
> (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x17067d7)
> (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
>      #4 0x73fbba63b786
> (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x123b786)
> (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
>      #5 0x73fbba96290a
> (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x156290a)
> (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
>      #6 0x73fbba941c5c
> (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x1541c5c)
> (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
>      #7 0x73fbc2041f20
> (/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0x41f20) (BuildId:
> 6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
>      #8 0x73fbc2041f68
> (/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0x41f68) (BuildId:
> 6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
>      #9 0x73fbc2034ca9
> (/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0x34ca9) (BuildId:
> 6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
>      #10 0x73fbc203ae90
> (/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0x3ae90) (BuildId:
> 6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
>      #11 0x73fbc203aeda
> (/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0x3aeda) (BuildId:
> 6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
>      #12 0x73fbc20a45f5
> (/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0xa45f5) (BuildId:
> 6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
>      #13 0x73fbc20a2bfc
> (/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0xa2bfc) (BuildId:
> 6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
>      #14 0x73fbd3047a75 in __run_exit_handlers stdlib/exit.c:108:8
>      #15 0x73fbd3047bbd in exit stdlib/exit.c:138:3
>      #16 0x5a5bab5e3fdb in qemu_default_main
> /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/../../system/main.c:52:5
>      #17 0x5a5bab5e3f9e in main
> /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/../../system/main.c:76:9
>      #18 0x73fbd302a1c9 in __libc_start_call_main
> csu/../sysdeps/nptl/libc_start_call_main.h:58:16
>      #19 0x73fbd302a28a in __libc_start_main csu/../csu/libc-start.c:360:3
>      #20 0x5a5ba9c5b554 in _start
> (/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/qemu-system-aarch64+0x15dc554)
> (BuildId: 8efda3601b42aa2644dde35d1d63f7b22b649a33)
> 
> UndefinedBehaviorSanitizer can not provide additional info.
> SUMMARY: UndefinedBehaviorSanitizer: SEGV
> (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x15788e9)
> (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
> ==243045==ABORTING

FWIW, I just saw this test also failing in a normal clang build (without 
sanitizers enabled). In the console log:

2025-02-18 07:08:47,497: [06:08:47.588] Loading module 
'/usr/lib/weston/kiosk-shell.so'
2025-02-18 07:08:47,914: =======================================================
2025-02-18 07:08:47,915: vkmark 2017.08
2025-02-18 07:08:47,915: =======================================================
2025-02-18 07:08:47,915: Vendor ID:      0x8086
2025-02-18 07:08:47,915: Device ID:      0x9A60
2025-02-18 07:08:47,916: Device Name:    Virtio-GPU Venus (Intel(R) UHD 
Graphics (TGL GT1))
2025-02-18 07:08:47,916: Driver Version: 100675584
2025-02-18 07:08:47,916: Device UUID:    c5930b2b12677aad53343f8a072209af
2025-02-18 07:08:47,916: =======================================================
2025-02-18 07:08:52,277: [vertex] device-local=true:MESA-VIRTIO: debug: 
stuck in fence wait with iter at 1024
2025-02-18 07:09:03,142: MESA-VIRTIO: debug: stuck in fence wait with iter 
at 2048
2025-02-18 07:09:24,640: MESA-VIRTIO: debug: stuck in fence wait with iter 
at 3072
2025-02-18 07:09:46,192: MESA-VIRTIO: debug: stuck in fence wait with iter 
at 4096
2025-02-18 07:10:28,665: MESA-VIRTIO: debug: stuck in fence wait with iter 
at 5120
2025-02-18 07:11:11,067: MESA-VIRTIO: debug: stuck in fence wait with iter 
at 6144
2025-02-18 07:11:53,619: MESA-VIRTIO: debug: stuck in fence wait with iter 
at 7168
2025-02-18 07:12:36,397: MESA-VIRTIO: debug: stuck in fence wait with iter 
at 8192
2025-02-18 07:14:01,431: MESA-VIRTIO: debug: stuck in fence wait with iter 
at 9216
2025-02-18 07:15:26,387: MESA-VIRTIO: debug: stuck in fence wait with iter 
at 10240
2025-02-18 07:16:51,349: MESA-VIRTIO: debug: stuck in fence wait with iter 
at 11264
2025-02-18 07:18:16,409: MESA-VIRTIO: debug: stuck in fence wait with iter 
at 12288
2025-02-18 07:19:41,439: MESA-VIRTIO: debug: stuck in fence wait with iter 
at 13312

Should we maybe mark it as flaky for the time being?

  Thomas

Re: [PULL 23/32] tests/functional: extend test_aarch64_virt with vulkan test

[Alex Bennée]

Thomas Huth <thuth@redhat.com> writes:

> On 17/02/2025 17.30, Peter Maydell wrote:
>> On Fri, 10 Jan 2025 at 13:23, Alex Bennée <alex.bennee@linaro.org> wrote:
>>> Now that we have virtio-gpu Vulkan support, let's add a test for it.
>>> Currently this is using images build by buildroot:
>>>
>>>    https://lists.buildroot.org/pipermail/buildroot/2024-December/768196.html
>>>
>>> Reviewed-by: Thomas Huth <thuth@redhat.com>
>>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>>> Message-Id: <20250108121054.1126164-24-alex.bennee@linaro.org>
>> Hi; this test currently fails for me with a clang sanitizer
>> build (ubuntu 24.04 host). It seems to run weston in the guest,
>> which fails with:
>> 2025-02-17 16:11:10,218: [16:11:10.672] Command line: weston -B
>> headless --renderer gl --shell kiosk -- vkmark -b:duration=1.0
>> 2025-02-17 16:11:10,224: [16:11:10.675] OS: Linux, 6.11.10, #2 SMP Thu
>> Dec  5 16:27:12 GMT 2024, aarch64
>> 2025-02-17 16:11:10,225: [16:11:10.680] Flight recorder: enabled
>> 2025-02-17 16:11:10,226: [16:11:10.681] warning: XDG_RUNTIME_DIR
>> "/tmp" is not configured
>> 2025-02-17 16:11:10,226: correctly.  Unix access mode must be 0700
>> (current mode is 0777),
>> 2025-02-17 16:11:10,226: and must be owned by the user UID 0 (current
>> owner is UID 0).
>> 2025-02-17 16:11:10,227: Refer to your distribution on how to get it, or
>> 2025-02-17 16:11:10,227:
>> http://www.freedesktop.org/wiki/Specifications/basedir-spec
>> 2025-02-17 16:11:10,228: on how to implement it.
>> 2025-02-17 16:11:10,240: [16:11:10.695] Starting with no config file.
>> 2025-02-17 16:11:10,253: [16:11:10.707] Output repaint window is 7 ms maximum.
>> 2025-02-17 16:11:10,262: [16:11:10.716] Loading module
>> '/usr/lib/libweston-14/headless-backend.so'
>> 2025-02-17 16:11:10,313: [16:11:10.768] Loading module
>> '/usr/lib/libweston-14/gl-renderer.so'
>> 2025-02-17 16:11:21,858: libEGL warning: egl: failed to create dri2 screen
>> 2025-02-17 16:11:21,959: libEGL warning: egl: failed to create dri2 screen
>> 2025-02-17 16:11:22,023: libEGL warning: egl: failed to create dri2 screen
>> 2025-02-17 16:11:22,032: [16:11:22.486] failed to initialize display
>> 2025-02-17 16:11:22,033: [16:11:22.488] EGL error state:
>> EGL_NOT_INITIALIZED (0x3001)
>> 2025-02-17 16:11:22,036: [16:11:22.490] fatal: failed to create
>> compositor backend
>> Then eventually the test framework times it ou and sends it
>> a SIGTERM, and QEMU SEGVs inside libEGL trying to run an
>> exit handler:
>> qemu-system-aarch64: terminating on signal 15 from pid 242824
>> (/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/pyvenv/bin/python3)
>> UndefinedBehaviorSanitizer:DEADLYSIGNAL
>> ==243045==ERROR: UndefinedBehaviorSanitizer: SEGV on unknown address
>> 0x73fbfefe6a31 (pc 0x73fbba9788e9 bp 0x73fbbbe0af80 sp 0x7ffd676fbfe0
>> T243045)
>> ==243045==The signal is caused by a READ memory access.
>>      #0 0x73fbba9788e9
>> (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x15788e9)
>> (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
>>      #1 0x73fbbaafc178
>> (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x16fc178)
>> (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
>>      #2 0x73fbba62564f
>> (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x122564f)
>> (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
>>      #3 0x73fbbab067d7
>> (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x17067d7)
>> (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
>>      #4 0x73fbba63b786
>> (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x123b786)
>> (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
>>      #5 0x73fbba96290a
>> (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x156290a)
>> (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
>>      #6 0x73fbba941c5c
>> (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x1541c5c)
>> (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
>>      #7 0x73fbc2041f20
>> (/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0x41f20) (BuildId:
>> 6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
>>      #8 0x73fbc2041f68
>> (/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0x41f68) (BuildId:
>> 6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
>>      #9 0x73fbc2034ca9
>> (/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0x34ca9) (BuildId:
>> 6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
>>      #10 0x73fbc203ae90
>> (/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0x3ae90) (BuildId:
>> 6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
>>      #11 0x73fbc203aeda
>> (/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0x3aeda) (BuildId:
>> 6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
>>      #12 0x73fbc20a45f5
>> (/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0xa45f5) (BuildId:
>> 6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
>>      #13 0x73fbc20a2bfc
>> (/lib/x86_64-linux-gnu/libEGL_nvidia.so.0+0xa2bfc) (BuildId:
>> 6cd9e3e571aa104d4fa5512a5c7196617fea6b51)
>>      #14 0x73fbd3047a75 in __run_exit_handlers stdlib/exit.c:108:8
>>      #15 0x73fbd3047bbd in exit stdlib/exit.c:138:3
>>      #16 0x5a5bab5e3fdb in qemu_default_main
>> /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/../../system/main.c:52:5
>>      #17 0x5a5bab5e3f9e in main
>> /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/../../system/main.c:76:9
>>      #18 0x73fbd302a1c9 in __libc_start_call_main
>> csu/../sysdeps/nptl/libc_start_call_main.h:58:16
>>      #19 0x73fbd302a28a in __libc_start_main csu/../csu/libc-start.c:360:3
>>      #20 0x5a5ba9c5b554 in _start
>> (/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/qemu-system-aarch64+0x15dc554)
>> (BuildId: 8efda3601b42aa2644dde35d1d63f7b22b649a33)
>> UndefinedBehaviorSanitizer can not provide additional info.
>> SUMMARY: UndefinedBehaviorSanitizer: SEGV
>> (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x15788e9)
>> (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
>> ==243045==ABORTING
>
> FWIW, I just saw this test also failing in a normal clang build
> (without sanitizers enabled). In the console log:
>
> 2025-02-18 07:08:47,497: [06:08:47.588] Loading module
> '/usr/lib/weston/kiosk-shell.so'
> 2025-02-18 07:08:47,914: =======================================================
> 2025-02-18 07:08:47,915: vkmark 2017.08
> 2025-02-18 07:08:47,915: =======================================================
> 2025-02-18 07:08:47,915: Vendor ID:      0x8086
> 2025-02-18 07:08:47,915: Device ID:      0x9A60
> 2025-02-18 07:08:47,916: Device Name:    Virtio-GPU Venus (Intel(R)
> UHD Graphics (TGL GT1))
> 2025-02-18 07:08:47,916: Driver Version: 100675584
> 2025-02-18 07:08:47,916: Device UUID:    c5930b2b12677aad53343f8a072209af
> 2025-02-18 07:08:47,916: =======================================================
> 2025-02-18 07:08:52,277: [vertex] device-local=true:MESA-VIRTIO:
> debug: stuck in fence wait with iter at 1024
> 2025-02-18 07:09:03,142: MESA-VIRTIO: debug: stuck in fence wait with
> iter at 2048
> 2025-02-18 07:09:24,640: MESA-VIRTIO: debug: stuck in fence wait with
> iter at 3072
> 2025-02-18 07:09:46,192: MESA-VIRTIO: debug: stuck in fence wait with
> iter at 4096
> 2025-02-18 07:10:28,665: MESA-VIRTIO: debug: stuck in fence wait with
> iter at 5120
> 2025-02-18 07:11:11,067: MESA-VIRTIO: debug: stuck in fence wait with
> iter at 6144
> 2025-02-18 07:11:53,619: MESA-VIRTIO: debug: stuck in fence wait with
> iter at 7168
> 2025-02-18 07:12:36,397: MESA-VIRTIO: debug: stuck in fence wait with
> iter at 8192
> 2025-02-18 07:14:01,431: MESA-VIRTIO: debug: stuck in fence wait with
> iter at 9216
> 2025-02-18 07:15:26,387: MESA-VIRTIO: debug: stuck in fence wait with
> iter at 10240
> 2025-02-18 07:16:51,349: MESA-VIRTIO: debug: stuck in fence wait with
> iter at 11264
> 2025-02-18 07:18:16,409: MESA-VIRTIO: debug: stuck in fence wait with
> iter at 12288
> 2025-02-18 07:19:41,439: MESA-VIRTIO: debug: stuck in fence wait with
> iter at 13312
>
> Should we maybe mark it as flaky for the time being?

I think the tests are too sensitive to host conditions
(mesa/virglrenderer builds/flags and underlying graphics hardware). I'd
rather detect the known flakiness and report it as a skip so they still
run on known good setups.

Hopefully this will work itself out as distros update and we can narrow
down requirements in configure.

>
>  Thomas

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [PULL 23/32] tests/functional: extend test_aarch64_virt with vulkan test

[Philippe Mathieu-Daudé]

(+Markus for CLI)

On 10/1/25 14:17, Alex Bennée wrote:
> Now that we have virtio-gpu Vulkan support, let's add a test for it.
> Currently this is using images build by buildroot:
> 
>    https://lists.buildroot.org/pipermail/buildroot/2024-December/768196.html
> 
> Reviewed-by: Thomas Huth <thuth@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20250108121054.1126164-24-alex.bennee@linaro.org>
> 
> diff --git a/tests/functional/test_aarch64_virt.py b/tests/functional/test_aarch64_virt.py
> index 201c5ed023..6b2336a28d 100755
> --- a/tests/functional/test_aarch64_virt.py
> +++ b/tests/functional/test_aarch64_virt.py
> @@ -13,10 +13,12 @@
>   import logging
>   from subprocess import check_call, DEVNULL
>   
> +from qemu.machine.machine import VMLaunchFailure
> +
>   from qemu_test import QemuSystemTest, Asset
> -from qemu_test import exec_command_and_wait_for_pattern
> +from qemu_test import exec_command, exec_command_and_wait_for_pattern
>   from qemu_test import wait_for_console_pattern
> -from qemu_test import get_qemu_img
> +from qemu_test import skipIfMissingCommands, get_qemu_img
>   
>   
>   class Aarch64VirtMachine(QemuSystemTest):
> @@ -132,5 +134,73 @@ def test_aarch64_virt_gicv2(self):
>           self.common_aarch64_virt("virt,gic-version=2")
>   
>   
> +    ASSET_VIRT_GPU_KERNEL = Asset(
> +        'https://fileserver.linaro.org/s/ce5jXBFinPxtEdx/'
> +        'download?path=%2F&files='
> +        'Image',
> +        '89e5099d26166204cc5ca4bb6d1a11b92c217e1f82ec67e3ba363d09157462f6')
> +
> +    ASSET_VIRT_GPU_ROOTFS = Asset(
> +        'https://fileserver.linaro.org/s/ce5jXBFinPxtEdx/'
> +        'download?path=%2F&files='
> +        'rootfs.ext4.zstd',
> +        '792da7573f5dc2913ddb7c638151d4a6b2d028a4cb2afb38add513c1924bdad4')
> +
> +    @skipIfMissingCommands('zstd')
> +    def test_aarch64_virt_with_gpu(self):
> +        # This tests boots with a buildroot test image that contains
> +        # vkmark and other GPU exercising tools. We run a headless
> +        # weston that nevertheless still exercises the virtio-gpu
> +        # backend.
> +
> +        self.set_machine('virt')
> +        self.require_accelerator("tcg")
> +
> +        kernel_path = self.ASSET_VIRT_GPU_KERNEL.fetch()
> +        image_path = self.uncompress(self.ASSET_VIRT_GPU_ROOTFS, format="zstd")
> +
> +        self.vm.set_console()
> +        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
> +                               'console=ttyAMA0 root=/dev/vda')
> +
> +        self.vm.add_args("-accel", "tcg")
> +        self.vm.add_args("-cpu", "neoverse-v1,pauth-impdef=on")
> +        self.vm.add_args("-machine", "virt,gic-version=max",
> +                         '-kernel', kernel_path,
> +                         '-append', kernel_command_line)
> +        self.vm.add_args("-smp", "2", "-m", "2048")
> +        self.vm.add_args("-device",
> +                         "virtio-gpu-gl-pci,hostmem=4G,blob=on,venus=on")
> +        self.vm.add_args("-display", "egl-headless")
> +        self.vm.add_args("-display", "dbus,gl=on")

[*]

> +        self.vm.add_args("-device", "virtio-blk-device,drive=hd0")
> +        self.vm.add_args("-blockdev",
> +                         "driver=raw,file.driver=file,"
> +                         "node-name=hd0,read-only=on,"
> +                         f"file.filename={image_path}")
> +        self.vm.add_args("-snapshot")
> +
> +        try:
> +            self.vm.launch()
> +        except VMLaunchFailure as excp:
> +            if "old virglrenderer, blob resources unsupported" in excp.output:
> +                self.skipTest("No blob support for virtio-gpu")
> +            elif "old virglrenderer, venus unsupported" in excp.output:
> +                self.skipTest("No venus support for virtio-gpu")

This seems dependent on the order of the CLI arguments, as I got:

qemu-system-aarch64: -device 
virtio-gpu-gl-pci,hostmem=4G,blob=on,venus=on: 'virtio-gpu-gl-pci' is 
not a valid device model name

I understand it is too complex to check this device availability with
meson, in order to avoid running the test.

Can we use device introspection instead, like we do in QTest with
qtest_qom_has_concrete_type() for accelerators? Maybe in the lines of:

   @skipIfMissingQOMType('virtio-gpu-gl-pci')

Or having a skipIfMissingDevice -> skipIfMissingQOMType alias:

   @skipIfMissingDevice('virtio-gpu-gl-pci')

tests/functional/test_virtio_version.py already uses some of it:

  def devtype_implements(vm, devtype, implements):
      return devtype in [d['name'] for d in
                         vm.cmd('qom-list-types', implements=implements)]

> +            else:
> +                self.log.info("unhandled launch failure: {excp.output}")
> +                raise excp
> +
> +        self.wait_for_console_pattern('buildroot login:')
> +        exec_command(self, 'root')
> +        exec_command(self, 'export XDG_RUNTIME_DIR=/tmp')
> +        exec_command_and_wait_for_pattern(self,
> +                                          "weston -B headless "
> +                                          "--renderer gl "
> +                                          "--shell kiosk "
> +                                          "-- vkmark -b:duration=1.0",
> +                                          "vkmark Score")
> +
> +
>   if __name__ == '__main__':
>       QemuSystemTest.main()

Re: [PULL 23/32] tests/functional: extend test_aarch64_virt with vulkan test

[Thomas Huth]

On 19/02/2025 14.25, Philippe Mathieu-Daudé wrote:
> (+Markus for CLI)
> 
> On 10/1/25 14:17, Alex Bennée wrote:
>> Now that we have virtio-gpu Vulkan support, let's add a test for it.
>> Currently this is using images build by buildroot:
>>
>>    https://lists.buildroot.org/pipermail/buildroot/2024-December/768196.html
>>
>> Reviewed-by: Thomas Huth <thuth@redhat.com>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> Message-Id: <20250108121054.1126164-24-alex.bennee@linaro.org>
>>
>> diff --git a/tests/functional/test_aarch64_virt.py b/tests/functional/ 
>> test_aarch64_virt.py
>> index 201c5ed023..6b2336a28d 100755
>> --- a/tests/functional/test_aarch64_virt.py
>> +++ b/tests/functional/test_aarch64_virt.py
>> @@ -13,10 +13,12 @@
>>   import logging
>>   from subprocess import check_call, DEVNULL
>> +from qemu.machine.machine import VMLaunchFailure
>> +
>>   from qemu_test import QemuSystemTest, Asset
>> -from qemu_test import exec_command_and_wait_for_pattern
>> +from qemu_test import exec_command, exec_command_and_wait_for_pattern
>>   from qemu_test import wait_for_console_pattern
>> -from qemu_test import get_qemu_img
>> +from qemu_test import skipIfMissingCommands, get_qemu_img
>>   class Aarch64VirtMachine(QemuSystemTest):
>> @@ -132,5 +134,73 @@ def test_aarch64_virt_gicv2(self):
>>           self.common_aarch64_virt("virt,gic-version=2")
>> +    ASSET_VIRT_GPU_KERNEL = Asset(
>> +        'https://fileserver.linaro.org/s/ce5jXBFinPxtEdx/'
>> +        'download?path=%2F&files='
>> +        'Image',
>> +        '89e5099d26166204cc5ca4bb6d1a11b92c217e1f82ec67e3ba363d09157462f6')
>> +
>> +    ASSET_VIRT_GPU_ROOTFS = Asset(
>> +        'https://fileserver.linaro.org/s/ce5jXBFinPxtEdx/'
>> +        'download?path=%2F&files='
>> +        'rootfs.ext4.zstd',
>> +        '792da7573f5dc2913ddb7c638151d4a6b2d028a4cb2afb38add513c1924bdad4')
>> +
>> +    @skipIfMissingCommands('zstd')
>> +    def test_aarch64_virt_with_gpu(self):
>> +        # This tests boots with a buildroot test image that contains
>> +        # vkmark and other GPU exercising tools. We run a headless
>> +        # weston that nevertheless still exercises the virtio-gpu
>> +        # backend.
>> +
>> +        self.set_machine('virt')
>> +        self.require_accelerator("tcg")
>> +
>> +        kernel_path = self.ASSET_VIRT_GPU_KERNEL.fetch()
>> +        image_path = self.uncompress(self.ASSET_VIRT_GPU_ROOTFS, 
>> format="zstd")
>> +
>> +        self.vm.set_console()
>> +        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
>> +                               'console=ttyAMA0 root=/dev/vda')
>> +
>> +        self.vm.add_args("-accel", "tcg")
>> +        self.vm.add_args("-cpu", "neoverse-v1,pauth-impdef=on")
>> +        self.vm.add_args("-machine", "virt,gic-version=max",
>> +                         '-kernel', kernel_path,
>> +                         '-append', kernel_command_line)
>> +        self.vm.add_args("-smp", "2", "-m", "2048")
>> +        self.vm.add_args("-device",
>> +                         "virtio-gpu-gl-pci,hostmem=4G,blob=on,venus=on")
>> +        self.vm.add_args("-display", "egl-headless")
>> +        self.vm.add_args("-display", "dbus,gl=on")
> 
> [*]
> 
>> +        self.vm.add_args("-device", "virtio-blk-device,drive=hd0")
>> +        self.vm.add_args("-blockdev",
>> +                         "driver=raw,file.driver=file,"
>> +                         "node-name=hd0,read-only=on,"
>> +                         f"file.filename={image_path}")
>> +        self.vm.add_args("-snapshot")
>> +
>> +        try:
>> +            self.vm.launch()
>> +        except VMLaunchFailure as excp:
>> +            if "old virglrenderer, blob resources unsupported" in 
>> excp.output:
>> +                self.skipTest("No blob support for virtio-gpu")
>> +            elif "old virglrenderer, venus unsupported" in excp.output:
>> +                self.skipTest("No venus support for virtio-gpu")
> 
> This seems dependent on the order of the CLI arguments, as I got:
> 
> qemu-system-aarch64: -device virtio-gpu-gl-pci,hostmem=4G,blob=on,venus=on: 
> 'virtio-gpu-gl-pci' is not a valid device model name
> 
> I understand it is too complex to check this device availability with
> meson, in order to avoid running the test.
> 
> Can we use device introspection instead, like we do in QTest with
> qtest_qom_has_concrete_type() for accelerators? Maybe in the lines of:
> 
>    @skipIfMissingQOMType('virtio-gpu-gl-pci')

We already have "self.require_device('...')" that can be used to check for 
the availability of devices and skip the test if it is not built in ... 
would that be suitable here?

  Thomas

Re: [PULL 23/32] tests/functional: extend test_aarch64_virt with vulkan test

[Philippe Mathieu-Daudé]

On 19/2/25 14:43, Thomas Huth wrote:
> On 19/02/2025 14.25, Philippe Mathieu-Daudé wrote:
>> (+Markus for CLI)
>>
>> On 10/1/25 14:17, Alex Bennée wrote:
>>> Now that we have virtio-gpu Vulkan support, let's add a test for it.
>>> Currently this is using images build by buildroot:
>>>
>>>    https://lists.buildroot.org/pipermail/buildroot/2024- 
>>> December/768196.html
>>>
>>> Reviewed-by: Thomas Huth <thuth@redhat.com>
>>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>>> Message-Id: <20250108121054.1126164-24-alex.bennee@linaro.org>
>>>
>>> diff --git a/tests/functional/test_aarch64_virt.py b/tests/ 
>>> functional/ test_aarch64_virt.py
>>> index 201c5ed023..6b2336a28d 100755
>>> --- a/tests/functional/test_aarch64_virt.py
>>> +++ b/tests/functional/test_aarch64_virt.py
>>> @@ -13,10 +13,12 @@
>>>   import logging
>>>   from subprocess import check_call, DEVNULL
>>> +from qemu.machine.machine import VMLaunchFailure
>>> +
>>>   from qemu_test import QemuSystemTest, Asset
>>> -from qemu_test import exec_command_and_wait_for_pattern
>>> +from qemu_test import exec_command, exec_command_and_wait_for_pattern
>>>   from qemu_test import wait_for_console_pattern
>>> -from qemu_test import get_qemu_img
>>> +from qemu_test import skipIfMissingCommands, get_qemu_img
>>>   class Aarch64VirtMachine(QemuSystemTest):
>>> @@ -132,5 +134,73 @@ def test_aarch64_virt_gicv2(self):
>>>           self.common_aarch64_virt("virt,gic-version=2")
>>> +    ASSET_VIRT_GPU_KERNEL = Asset(
>>> +        'https://fileserver.linaro.org/s/ce5jXBFinPxtEdx/'
>>> +        'download?path=%2F&files='
>>> +        'Image',
>>> +        
>>> '89e5099d26166204cc5ca4bb6d1a11b92c217e1f82ec67e3ba363d09157462f6')
>>> +
>>> +    ASSET_VIRT_GPU_ROOTFS = Asset(
>>> +        'https://fileserver.linaro.org/s/ce5jXBFinPxtEdx/'
>>> +        'download?path=%2F&files='
>>> +        'rootfs.ext4.zstd',
>>> +        
>>> '792da7573f5dc2913ddb7c638151d4a6b2d028a4cb2afb38add513c1924bdad4')
>>> +
>>> +    @skipIfMissingCommands('zstd')
>>> +    def test_aarch64_virt_with_gpu(self):
>>> +        # This tests boots with a buildroot test image that contains
>>> +        # vkmark and other GPU exercising tools. We run a headless
>>> +        # weston that nevertheless still exercises the virtio-gpu
>>> +        # backend.
>>> +
>>> +        self.set_machine('virt')
>>> +        self.require_accelerator("tcg")
>>> +
>>> +        kernel_path = self.ASSET_VIRT_GPU_KERNEL.fetch()
>>> +        image_path = self.uncompress(self.ASSET_VIRT_GPU_ROOTFS, 
>>> format="zstd")
>>> +
>>> +        self.vm.set_console()
>>> +        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
>>> +                               'console=ttyAMA0 root=/dev/vda')
>>> +
>>> +        self.vm.add_args("-accel", "tcg")
>>> +        self.vm.add_args("-cpu", "neoverse-v1,pauth-impdef=on")
>>> +        self.vm.add_args("-machine", "virt,gic-version=max",
>>> +                         '-kernel', kernel_path,
>>> +                         '-append', kernel_command_line)
>>> +        self.vm.add_args("-smp", "2", "-m", "2048")
>>> +        self.vm.add_args("-device",
>>> +                         "virtio-gpu-gl- 
>>> pci,hostmem=4G,blob=on,venus=on")
>>> +        self.vm.add_args("-display", "egl-headless")
>>> +        self.vm.add_args("-display", "dbus,gl=on")
>>
>> [*]
>>
>>> +        self.vm.add_args("-device", "virtio-blk-device,drive=hd0")
>>> +        self.vm.add_args("-blockdev",
>>> +                         "driver=raw,file.driver=file,"
>>> +                         "node-name=hd0,read-only=on,"
>>> +                         f"file.filename={image_path}")
>>> +        self.vm.add_args("-snapshot")
>>> +
>>> +        try:
>>> +            self.vm.launch()
>>> +        except VMLaunchFailure as excp:
>>> +            if "old virglrenderer, blob resources unsupported" in 
>>> excp.output:
>>> +                self.skipTest("No blob support for virtio-gpu")
>>> +            elif "old virglrenderer, venus unsupported" in excp.output:
>>> +                self.skipTest("No venus support for virtio-gpu")
>>
>> This seems dependent on the order of the CLI arguments, as I got:
>>
>> qemu-system-aarch64: -device virtio-gpu-gl- 
>> pci,hostmem=4G,blob=on,venus=on: 'virtio-gpu-gl-pci' is not a valid 
>> device model name
>>
>> I understand it is too complex to check this device availability with
>> meson, in order to avoid running the test.
>>
>> Can we use device introspection instead, like we do in QTest with
>> qtest_qom_has_concrete_type() for accelerators? Maybe in the lines of:
>>
>>    @skipIfMissingQOMType('virtio-gpu-gl-pci')
> 
> We already have "self.require_device('...')" that can be used to check 
> for the availability of devices and skip the test if it is not built 
> in ... would that be suitable here?

Yes, perfect, thanks!

> 
>   Thomas
>