[PULL 0/6] Functional test fixes for 10.0-rc2

[PULL 0/6] Functional test fixes for 10.0-rc2

[Thomas Huth]

 Hi Stefan!

The following changes since commit b876e721f1c939f3e83ac85bd3c1c2821e12b3fa:

  Update version for v10.0.0-rc1 release (2025-03-25 13:58:14 -0400)

are available in the Git repository at:

  https://gitlab.com/thuth/qemu.git tags/pull-request-2025-03-27

for you to fetch changes up to 5a8d024f8d3f55846cb8385b025d795fee4afd35:

  tests/functional/test_vnc: Skip test if VNC support is not available (2025-03-27 08:59:35 +0100)

----------------------------------------------------------------
* Make some functional tests more robust
* Fix a typo in the s390x code

----------------------------------------------------------------
Peter Maydell (1):
      tests/functional/meson.build: Bump arm_aspeed_bletchley timeout

Philippe Mathieu-Daudé (1):
      target/s390x: Fix a typo in s390_cpu_class_init()

Thomas Huth (4):
      tests/functional/test_ppc64_replay: Mark the e500 test as flaky
      tests/functional: Add missing require_netdev('user') statements
      tests/functional/test_aarch64_virt_gpu: Skip if "dbus" display isn't available
      tests/functional/test_vnc: Skip test if VNC support is not available

 MAINTAINERS                                  |  2 +-
 target/s390x/cpu.c                           |  2 +-
 tests/functional/meson.build                 |  2 +-
 tests/functional/test_aarch64_rme_sbsaref.py |  1 +
 tests/functional/test_aarch64_rme_virt.py    |  4 +++-
 tests/functional/test_aarch64_virt_gpu.py    |  2 ++
 tests/functional/test_arm_bpim2u.py          |  2 ++
 tests/functional/test_arm_cubieboard.py      |  2 ++
 tests/functional/test_arm_orangepi.py        |  4 ++++
 tests/functional/test_ppc64_hv.py            |  3 +++
 tests/functional/test_ppc64_replay.py        |  3 ++-
 tests/functional/test_vnc.py                 | 26 ++++++++++++++++++++++----
 tests/functional/test_x86_64_kvm_xen.py      |  1 +
 13 files changed, 45 insertions(+), 9 deletions(-)

[PULL 1/6] tests/functional/test_ppc64_replay: Mark the e500 test as flaky

[Thomas Huth]

From: Thomas Huth <thuth@redhat.com>

The test_ppc64_e500 occasionally fails (it just hangs and finally times
out). The same issue could also be reproduced with the former Avocado
test already (especially if the host system is under heavy load), so
it's not a new regression. It's very likely the issue that has been
filed at https://gitlab.com/qemu-project/qemu/-/issues/2523 instead (e.g.
I could not reproduce the issue in older commits before commit 578912ad),
so use this URL for the reasoning in the the decorator.

Message-ID: <20250320065012.309520-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/functional/test_ppc64_replay.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tests/functional/test_ppc64_replay.py b/tests/functional/test_ppc64_replay.py
index 48ce1b7f1e1..e8c9c4bcbf8 100755
--- a/tests/functional/test_ppc64_replay.py
+++ b/tests/functional/test_ppc64_replay.py
@@ -5,7 +5,7 @@
 #
 # SPDX-License-Identifier: GPL-2.0-or-later
 
-from qemu_test import Asset
+from qemu_test import Asset, skipFlakyTest
 from replay_kernel import ReplayKernelBase
 
 
@@ -16,6 +16,7 @@ class Ppc64Replay(ReplayKernelBase):
          'day19.tar.xz'),
         '20b1bb5a8488c664defbb5d283addc91a05335a936c63b3f5ff7eee74b725755')
 
+    @skipFlakyTest('https://gitlab.com/qemu-project/qemu/-/issues/2523')
     def test_ppc64_e500(self):
         self.set_machine('ppce500')
         self.cpu = 'e5500'
-- 
2.49.0

[PULL 2/6] tests/functional/meson.build: Bump arm_aspeed_bletchley timeout

[Thomas Huth]

From: Peter Maydell <peter.maydell@linaro.org>

The arm_aspeed_bletchley timeout takes more than its
current 120s timeout on a debug Rust config:

$ time (cd build/rust; PYTHONPATH=../../python:../../tests/functional QEMU_TEST_QEMU_BINARY=./qemu-system-arm ./pyvenv/bin/python3 ../../tests/functional/test_arm_aspeed_bletchley.py)
TAP version 13
ok 1 test_arm_aspeed_bletchley.BletchleyMachine.test_arm_ast2600_bletchley_openbmc
1..1

real    2m15.536s
user    3m21.444s
sys     0m11.558s

Bump it up to 480s, same as arm_aspeed_rainier.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250321155806.1888867-1-peter.maydell@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/functional/meson.build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 96d28289279..0f8be30fe27 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -26,7 +26,7 @@ test_timeouts = {
   'arm_aspeed_witherspoon' : 120,
   'arm_aspeed_ast2500' : 720,
   'arm_aspeed_ast2600' : 1200,
-  'arm_aspeed_bletchley' : 120,
+  'arm_aspeed_bletchley' : 480,
   'arm_aspeed_rainier' : 480,
   'arm_bpim2u' : 500,
   'arm_collie' : 180,
-- 
2.49.0

[PULL 3/6] tests/functional: Add missing require_netdev('user') statements

[Thomas Huth]

From: Thomas Huth <thuth@redhat.com>

A bunch of tests are using "-netdev user" but fail to check
for the availability of SLIRP in the binary, so these tests
fail if QEMU has been configured with "--disable-slirp"
(most of the tests are disabled by default with a decorator,
that's likely why nobody noticed this problem yet). Add the
missing self.require_netdev('user') statements to skip the
tests if SLIRP is not available.

Message-ID: <20250324123450.111307-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/functional/test_aarch64_rme_sbsaref.py | 1 +
 tests/functional/test_aarch64_rme_virt.py    | 4 +++-
 tests/functional/test_arm_bpim2u.py          | 2 ++
 tests/functional/test_arm_cubieboard.py      | 2 ++
 tests/functional/test_arm_orangepi.py        | 4 ++++
 tests/functional/test_ppc64_hv.py            | 3 +++
 tests/functional/test_x86_64_kvm_xen.py      | 1 +
 7 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/tests/functional/test_aarch64_rme_sbsaref.py b/tests/functional/test_aarch64_rme_sbsaref.py
index ddcc9493a6c..0f4f6103a1e 100755
--- a/tests/functional/test_aarch64_rme_sbsaref.py
+++ b/tests/functional/test_aarch64_rme_sbsaref.py
@@ -33,6 +33,7 @@ class Aarch64RMESbsaRefMachine(QemuSystemTest):
     def test_aarch64_rme_sbsaref(self):
         self.set_machine('sbsa-ref')
         self.require_accelerator('tcg')
+        self.require_netdev('user')
 
         self.vm.set_console()
 
diff --git a/tests/functional/test_aarch64_rme_virt.py b/tests/functional/test_aarch64_rme_virt.py
index 38e01721a4e..f4ad4d33d58 100755
--- a/tests/functional/test_aarch64_rme_virt.py
+++ b/tests/functional/test_aarch64_rme_virt.py
@@ -60,8 +60,10 @@ class Aarch64RMEVirtMachine(QemuSystemTest):
     # and launching a nested VM using it.
     def test_aarch64_rme_virt(self):
         self.set_machine('virt')
-        self.vm.set_console()
         self.require_accelerator('tcg')
+        self.require_netdev('user')
+
+        self.vm.set_console()
 
         stack_path_tar_gz = self.ASSET_RME_STACK_VIRT.fetch()
         self.archive_extract(stack_path_tar_gz, format="tar")
diff --git a/tests/functional/test_arm_bpim2u.py b/tests/functional/test_arm_bpim2u.py
index 12cd3597460..8de6ccba881 100755
--- a/tests/functional/test_arm_bpim2u.py
+++ b/tests/functional/test_arm_bpim2u.py
@@ -140,6 +140,8 @@ def test_arm_bpim2u_gmac(self):
     @skipBigDataTest()
     def test_arm_bpim2u_openwrt_22_03_3(self):
         self.set_machine('bpim2u')
+        self.require_netdev('user')
+
         # This test download a 8.9 MiB compressed image and expand it
         # to 127 MiB.
         image_path = self.uncompress(self.ASSET_SD_IMAGE)
diff --git a/tests/functional/test_arm_cubieboard.py b/tests/functional/test_arm_cubieboard.py
index 423db710e82..b87a28154da 100755
--- a/tests/functional/test_arm_cubieboard.py
+++ b/tests/functional/test_arm_cubieboard.py
@@ -107,6 +107,8 @@ def test_arm_cubieboard_openwrt_22_03_2(self):
         # This test download a 7.5 MiB compressed image and expand it
         # to 126 MiB.
         self.set_machine('cubieboard')
+        self.require_netdev('user')
+
         image_path = self.uncompress(self.ASSET_OPENWRT)
         image_pow2ceil_expand(image_path)
 
diff --git a/tests/functional/test_arm_orangepi.py b/tests/functional/test_arm_orangepi.py
index 28919391e5d..1815f56e027 100755
--- a/tests/functional/test_arm_orangepi.py
+++ b/tests/functional/test_arm_orangepi.py
@@ -147,6 +147,8 @@ def test_arm_orangepi_sd(self):
     @skipBigDataTest()
     def test_arm_orangepi_armbian(self):
         self.set_machine('orangepi-pc')
+        self.require_netdev('user')
+
         # This test download a 275 MiB compressed image and expand it
         # to 1036 MiB, but the underlying filesystem is 1552 MiB...
         # As we expand it to 2 GiB we are safe.
@@ -181,6 +183,8 @@ def test_arm_orangepi_armbian(self):
     @skipBigDataTest()
     def test_arm_orangepi_uboot_netbsd9(self):
         self.set_machine('orangepi-pc')
+        self.require_netdev('user')
+
         # This test download a 304MB compressed image and expand it to 2GB
         # We use the common OrangePi PC 'plus' build of U-Boot for our secondary
         # program loader (SPL). We will then set the path to the more specific
diff --git a/tests/functional/test_ppc64_hv.py b/tests/functional/test_ppc64_hv.py
index 62f996adf6d..1920e91f18b 100755
--- a/tests/functional/test_ppc64_hv.py
+++ b/tests/functional/test_ppc64_hv.py
@@ -125,6 +125,7 @@ def do_test_kvm(self, hpt=False):
 
     def test_hv_pseries(self):
         self.require_accelerator("tcg")
+        self.require_netdev('user')
         self.set_machine('pseries')
         self.vm.add_args("-accel", "tcg,thread=multi")
         self.vm.add_args('-device', 'nvme,serial=1234,drive=drive0')
@@ -136,6 +137,7 @@ def test_hv_pseries(self):
 
     def test_hv_pseries_kvm(self):
         self.require_accelerator("kvm")
+        self.require_netdev('user')
         self.set_machine('pseries')
         self.vm.add_args("-accel", "kvm")
         self.vm.add_args('-device', 'nvme,serial=1234,drive=drive0')
@@ -147,6 +149,7 @@ def test_hv_pseries_kvm(self):
 
     def test_hv_powernv(self):
         self.require_accelerator("tcg")
+        self.require_netdev('user')
         self.set_machine('powernv')
         self.vm.add_args("-accel", "tcg,thread=multi")
         self.vm.add_args('-device', 'nvme,bus=pcie.2,addr=0x0,serial=1234,drive=drive0',
diff --git a/tests/functional/test_x86_64_kvm_xen.py b/tests/functional/test_x86_64_kvm_xen.py
index 3bedef6c98c..c6abf6bba3b 100755
--- a/tests/functional/test_x86_64_kvm_xen.py
+++ b/tests/functional/test_x86_64_kvm_xen.py
@@ -41,6 +41,7 @@ class KVMXenGuest(QemuSystemTest):
     def common_vm_setup(self):
         # We also catch lack of KVM_XEN support if we fail to launch
         self.require_accelerator("kvm")
+        self.require_netdev('user')
 
         self.vm.set_console()
 
-- 
2.49.0

[PULL 4/6] target/s390x: Fix a typo in s390_cpu_class_init()

[Thomas Huth]

From: Philippe Mathieu-Daudé <philmd@linaro.org>

Replace the comma at the end of the line by a semicolon.

Fixes: 41868f846d2 ("s390x/cpumodel: "host" and "qemu" as CPU subclasses")
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20250324165356.39540-1-philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 target/s390x/cpu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
index d73142600bf..1f75629ddc2 100644
--- a/target/s390x/cpu.c
+++ b/target/s390x/cpu.c
@@ -377,7 +377,7 @@ static void s390_cpu_class_init(ObjectClass *oc, void *data)
     resettable_class_set_parent_phases(rc, NULL, s390_cpu_reset_hold, NULL,
                                        &scc->parent_phases);
 
-    cc->class_by_name = s390_cpu_class_by_name,
+    cc->class_by_name = s390_cpu_class_by_name;
     cc->mmu_index = s390x_cpu_mmu_index;
     cc->dump_state = s390_cpu_dump_state;
     cc->query_cpu_fast = s390_query_cpu_fast;
-- 
2.49.0

[PULL 5/6] tests/functional/test_aarch64_virt_gpu: Skip if "dbus" display isn't available

[Thomas Huth]

From: Thomas Huth <thuth@redhat.com>

This test currently fails if the "dbus" display has not been compiled
into the binary (which can happen when CFI has been enabled, for example).
Check for the error message to skip the test in that case.

While we're at it, also make sure that this test is covered in the
right section in the MAINTAINERS file.

Message-ID: <20250325061609.272847-1-thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 MAINTAINERS                               | 2 +-
 tests/functional/test_aarch64_virt_gpu.py | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index 8f470a1c9b7..c1ccb0acd14 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1018,7 +1018,7 @@ S: Maintained
 F: hw/arm/virt*
 F: include/hw/arm/virt.h
 F: docs/system/arm/virt.rst
-F: tests/functional/test_aarch64_virt.py
+F: tests/functional/test_aarch64_virt*.py
 F: tests/functional/test_aarch64_tuxrun.py
 F: tests/functional/test_arm_tuxrun.py
 
diff --git a/tests/functional/test_aarch64_virt_gpu.py b/tests/functional/test_aarch64_virt_gpu.py
index 314d994a7aa..38447278579 100755
--- a/tests/functional/test_aarch64_virt_gpu.py
+++ b/tests/functional/test_aarch64_virt_gpu.py
@@ -74,6 +74,8 @@ def _launch_virt_gpu(self, gpu_device):
                 self.skipTest("Can't access host DRM render node")
             elif "'type' does not accept value 'egl-headless'" in excp.output:
                 self.skipTest("egl-headless support is not available")
+            elif "'type' does not accept value 'dbus'" in excp.output:
+                self.skipTest("dbus display support is not available")
             else:
                 self.log.info("unhandled launch failure: %s", excp.output)
                 raise excp
-- 
2.49.0

[PULL 6/6] tests/functional/test_vnc: Skip test if VNC support is not available

[Thomas Huth]

From: Thomas Huth <thuth@redhat.com>

These tests currently fail if VNC support has not been compiled into
the QEMU binary. Let's add some checks to skip the tests in that
case instead.

Message-ID: <20250325064715.278876-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/functional/test_vnc.py | 26 ++++++++++++++++++++++----
 1 file changed, 22 insertions(+), 4 deletions(-)

diff --git a/tests/functional/test_vnc.py b/tests/functional/test_vnc.py
index 1916be0103f..8c9953bdb00 100755
--- a/tests/functional/test_vnc.py
+++ b/tests/functional/test_vnc.py
@@ -12,6 +12,7 @@
 
 import socket
 from typing import List
+from qemu.machine.machine import VMLaunchFailure
 
 from qemu_test import QemuSystemTest
 from qemu_test.ports import Ports
@@ -32,7 +33,14 @@ class Vnc(QemuSystemTest):
     def test_no_vnc_change_password(self):
         self.vm.add_args('-nodefaults', '-S')
         self.vm.launch()
-        self.assertFalse(self.vm.qmp('query-vnc')['return']['enabled'])
+
+        query_vnc_response = self.vm.qmp('query-vnc')
+        if 'error' in query_vnc_response:
+            self.assertEqual(query_vnc_response['error']['class'],
+                             'CommandNotFound')
+            self.skipTest('VNC support not available')
+        self.assertFalse(query_vnc_response['return']['enabled'])
+
         set_password_response = self.vm.qmp('change-vnc-password',
                                             password='new_password')
         self.assertIn('error', set_password_response)
@@ -41,9 +49,19 @@ def test_no_vnc_change_password(self):
         self.assertEqual(set_password_response['error']['desc'],
                          'Could not set password')
 
+    def launch_guarded(self):
+        try:
+            self.vm.launch()
+        except VMLaunchFailure as excp:
+            if "-vnc: invalid option" in excp.output:
+                self.skipTest("VNC support not available")
+            else:
+                self.log.info("unhandled launch failure: %s", excp.output)
+                raise excp
+
     def test_change_password_requires_a_password(self):
         self.vm.add_args('-nodefaults', '-S', '-vnc', ':1,to=999')
-        self.vm.launch()
+        self.launch_guarded()
         self.assertTrue(self.vm.qmp('query-vnc')['return']['enabled'])
         set_password_response = self.vm.qmp('change-vnc-password',
                                             password='new_password')
@@ -55,7 +73,7 @@ def test_change_password_requires_a_password(self):
 
     def test_change_password(self):
         self.vm.add_args('-nodefaults', '-S', '-vnc', ':1,to=999,password=on')
-        self.vm.launch()
+        self.launch_guarded()
         self.assertTrue(self.vm.qmp('query-vnc')['return']['enabled'])
         self.vm.cmd('change-vnc-password',
                     password='new_password')
@@ -66,7 +84,7 @@ def do_test_change_listen(self, a, b, c):
         self.assertFalse(check_connect(c))
 
         self.vm.add_args('-nodefaults', '-S', '-vnc', f'{VNC_ADDR}:{a - 5900}')
-        self.vm.launch()
+        self.launch_guarded()
         self.assertEqual(self.vm.qmp('query-vnc')['return']['service'], str(a))
         self.assertTrue(check_connect(a))
         self.assertFalse(check_connect(b))
-- 
2.49.0

Re: [PULL 0/6] Functional test fixes for 10.0-rc2

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 116 bytes --]

Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/10.0 for any user-visible changes.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]