[PULL 00/23] Functional tests and misc patches

[PULL 00/23] Functional tests and misc patches

[Thomas Huth]

 Hi Richard!

The following changes since commit 6a9fa5ef3230a7d51e0d953a59ee9ef10af705b8:

  Merge tag 'pull-tcg-20250905' of https://gitlab.com/rth7680/qemu into staging (2025-09-05 09:51:27 +0200)

are available in the Git repository at:

  https://gitlab.com/thuth/qemu.git tags/pull-request-2025-09-09

for you to fetch changes up to 2fc170bcdc4d2f05534c68572b4f72a7d18c2119:

  tests/functional: purge scratch dir on test startup (2025-09-09 15:45:33 +0200)

----------------------------------------------------------------
* Silence warnings from the undefined-behaviour sanitizer
* Many small improvements to various functional tests
* Remove remainders from storing avocado artifacts in the Gitlab CI
* Keep more meson log files as artifacts in the Gitlab CI instead
* Re-enable -fzero-call-used-regs on OpenBSD

----------------------------------------------------------------
Daniel P. Berrangé (11):
      tests/functional: enable force refresh of cached assets
      tests/functional: fix formatting of exception args
      tests/functional: handle URLError when fetching assets
      gitlab: replace avocado results files with meson results files
      gitlab: always include entire of meson-logs directory
      gitlab: include all junit XML files from meson
      gitlab: prevent duplicated meson log artifacts in test jobs
      tests/functional: fix infinite loop on console EOF
      tests/functional: avoid duplicate messages on failures
      tests/functional: avoid tearDown failure when QEMU dies
      tests/functional: purge scratch dir on test startup

John Levon (2):
      tests/functional: return output from cmd.py helpers
      tests/functional: add vm param to cmd.py helpers

Kane-Chen-AS (4):
      tests/functional/arm: Update test ASPEED SDK v03.02 for AST1030
      tests/functional/arm: Update test ASPEED SDK v09.07 for AST2500
      tests/functional/arm: Update test ASPEED SDK v09.07 for AST2600
      tests/functional/arm: Update test ASPEED SDK v09.07 for AST2700 vbootrom

Thomas Huth (6):
      hw/display/bcm2835_fb: Move inclusion of console.h to the .c file
      Revert "meson.build: Disable -fzero-call-used-regs on OpenBSD"
      hw/mips/malta: Silence warning from ubsan
      system/physmem: Silence warning from ubsan
      ui/vnc: Fix crash when specifying [vnc] without id in the config file
      tests/functional/m68k: Avoid ResourceWarning in the nextcube test

 docs/devel/testing/functional.rst               |  3 ++
 meson.build                                     |  6 +--
 include/hw/display/bcm2835_fb.h                 |  1 -
 hw/display/bcm2835_fb.c                         |  1 +
 hw/mips/malta.c                                 |  2 +-
 system/physmem.c                                |  4 +-
 ui/vnc.c                                        | 18 +++----
 .gitlab-ci.d/buildtest-template.yml             | 16 ++++--
 .gitlab-ci.d/buildtest.yml                      |  4 +-
 .gitlab-ci.d/crossbuild-template.yml            |  4 +-
 .gitlab-ci.d/custom-runners.yml                 |  2 +-
 .gitlab-ci.d/windows.yml                        |  4 +-
 tests/functional/aarch64/test_aspeed_ast2700.py |  8 ++-
 tests/functional/arm/test_aspeed_ast1030.py     | 12 ++---
 tests/functional/arm/test_aspeed_ast2500.py     |  8 +--
 tests/functional/arm/test_aspeed_ast2600.py     |  8 +--
 tests/functional/m68k/test_nextcube.py          |  3 +-
 tests/functional/qemu_test/asset.py             | 16 +++++-
 tests/functional/qemu_test/cmd.py               | 67 ++++++++++++++++++++-----
 tests/functional/qemu_test/testcase.py          | 14 ++++--
 20 files changed, 137 insertions(+), 64 deletions(-)

[PULL 01/23] hw/display/bcm2835_fb: Move inclusion of console.h to the .c file

[Thomas Huth]

From: Thomas Huth <thuth@redhat.com>

The definitions from console.h are not needed in the bcm2835_fb.h
header file yet, so let's move it to the place that really needs
its definitions, i.e. into the bcm2835_fb.c file.
This way the header can also be used by code that is not compiled
with the CFLAGS that are required for pixman or OpenGL (in case
their headers do not reside under /usr/include).

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250508144120.163009-3-thuth@redhat.com>
---
 include/hw/display/bcm2835_fb.h | 1 -
 hw/display/bcm2835_fb.c         | 1 +
 2 files changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/hw/display/bcm2835_fb.h b/include/hw/display/bcm2835_fb.h
index 49541bf08f4..acc9230b6a8 100644
--- a/include/hw/display/bcm2835_fb.h
+++ b/include/hw/display/bcm2835_fb.h
@@ -13,7 +13,6 @@
 #define BCM2835_FB_H
 
 #include "hw/sysbus.h"
-#include "ui/console.h"
 #include "qom/object.h"
 
 #define UPPER_RAM_BASE 0x40000000
diff --git a/hw/display/bcm2835_fb.c b/hw/display/bcm2835_fb.c
index 820e67ac8bb..1bb2ee45a01 100644
--- a/hw/display/bcm2835_fb.c
+++ b/hw/display/bcm2835_fb.c
@@ -27,6 +27,7 @@
 #include "hw/display/bcm2835_fb.h"
 #include "hw/hw.h"
 #include "hw/irq.h"
+#include "ui/console.h"
 #include "framebuffer.h"
 #include "ui/pixel_ops.h"
 #include "hw/misc/bcm2835_mbox_defs.h"
-- 
2.51.0

[PULL 02/23] Revert "meson.build: Disable -fzero-call-used-regs on OpenBSD"

[Thomas Huth]

From: Thomas Huth <thuth@redhat.com>

This reverts commit 2d6d995709482cc8b6a76dbb5334a28001a14a9a.

OpenBSD 7.7 fixed the problem with the -fzero-call-used-regs on OpenBSD,
see https://github.com/openbsd/src/commit/03eca72d1e030b7a542cd6aec1 for
the fix there.

Suggested-by: Brad Smith <brad@comstyle.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250508144120.163009-6-thuth@redhat.com>
---
 meson.build | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/meson.build b/meson.build
index fa6186db334..3d738733566 100644
--- a/meson.build
+++ b/meson.build
@@ -709,11 +709,7 @@ hardening_flags = [
 #
 # NB: Clang 17 is broken and SEGVs
 # https://github.com/llvm/llvm-project/issues/75168
-#
-# NB2: This clashes with the "retguard" extension of OpenBSD's Clang
-# https://gitlab.com/qemu-project/qemu/-/issues/2278
-if host_os != 'openbsd' and \
-   cc.compiles('extern struct { void (*cb)(void); } s; void f(void) { s.cb(); }',
+if cc.compiles('extern struct { void (*cb)(void); } s; void f(void) { s.cb(); }',
                name: '-fzero-call-used-regs=used-gpr',
                args: ['-O2', '-fzero-call-used-regs=used-gpr'])
     hardening_flags += '-fzero-call-used-regs=used-gpr'
-- 
2.51.0

[PULL 03/23] hw/mips/malta: Silence warning from ubsan

[Thomas Huth]

From: Thomas Huth <thuth@redhat.com>

When compiling QEMU with --enable-ubsan there is a undefined behavior
warning when using the malta machine:

 hw/mips/malta.c:1200:32: runtime error: addition of unsigned offset
  to 0x7fb620600000 overflowed to 0x7fb6205fffff
 SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior hw/mips/malta.c:1200:32

To fix the issue, check the bios_size whether we really loaded the
firmware before trying to byte-swap the instructions here.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250728115152.187728-1-thuth@redhat.com>
---
 hw/mips/malta.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/mips/malta.c b/hw/mips/malta.c
index 344dc8ca766..02da629b5af 100644
--- a/hw/mips/malta.c
+++ b/hw/mips/malta.c
@@ -1191,7 +1191,7 @@ void mips_malta_init(MachineState *machine)
          * In little endian mode the 32bit words in the bios are swapped,
          * a neat trick which allows bi-endian firmware.
          */
-        if (!TARGET_BIG_ENDIAN) {
+        if (!TARGET_BIG_ENDIAN && bios_size > 0) {
             uint32_t *end, *addr;
             const size_t swapsize = MIN(bios_size, 0x3e0000);
             addr = rom_ptr(FLASH_ADDRESS, swapsize);
-- 
2.51.0

[PULL 04/23] system/physmem: Silence warning from ubsan

[Thomas Huth]

From: Thomas Huth <thuth@redhat.com>

When compiling QEMU with --enable-ubsan there is a undefined behavior
warning when running the bios-tables-test for example:

 .../system/physmem.c:3243:13: runtime error: applying non-zero offset 262144 to null pointer
    #0 0x55ac1df5fbc4 in address_space_write_rom_internal .../system/physmem.c:3243:13

The problem is that buf is indeed NULL if the function is e.g. called
with type == FLUSH_CACHE. Add a check to fix the issue.

Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250728172545.314178-1-thuth@redhat.com>
---
 system/physmem.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/system/physmem.c b/system/physmem.c
index f498572fc82..311011156c7 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -3231,8 +3231,10 @@ static inline MemTxResult address_space_write_rom_internal(AddressSpace *as,
             }
         }
         len -= l;
-        buf += l;
         addr += l;
+        if (buf) {
+            buf += l;
+        }
     }
     return MEMTX_OK;
 }
-- 
2.51.0

Re: [PULL 04/23] system/physmem: Silence warning from ubsan

[Richard Henderson]

On 9/9/25 06:51, Thomas Huth wrote:
> From: Thomas Huth <thuth@redhat.com>
> 
> When compiling QEMU with --enable-ubsan there is a undefined behavior
> warning when running the bios-tables-test for example:
> 
>   .../system/physmem.c:3243:13: runtime error: applying non-zero offset 262144 to null pointer
>      #0 0x55ac1df5fbc4 in address_space_write_rom_internal .../system/physmem.c:3243:13
> 
> The problem is that buf is indeed NULL if the function is e.g. called
> with type == FLUSH_CACHE. Add a check to fix the issue.
> 
> Reviewed-by: David Hildenbrand <david@redhat.com>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> Message-ID: <20250728172545.314178-1-thuth@redhat.com>
> ---
>   system/physmem.c | 4 +++-
>   1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/system/physmem.c b/system/physmem.c
> index f498572fc82..311011156c7 100644
> --- a/system/physmem.c
> +++ b/system/physmem.c
> @@ -3231,8 +3231,10 @@ static inline MemTxResult address_space_write_rom_internal(AddressSpace *as,
>               }
>           }
>           len -= l;
> -        buf += l;
>           addr += l;
> +        if (buf) {
> +            buf += l;
> +        }
>       }
>       return MEMTX_OK;
>   }

This has caused a new warning from Coverity:

Line 3224:
CID 1621220: (#1 of 1): Dereference after null check (FORWARD_NULL)
10. var_deref_model: Passing null pointer buf to memcpy, which dereferences it. [Note: The 
source code implementation of the function has been overridden by a builtin model.]

Line 3235:
5. var_compare_op: Comparing buf to null implies that buf might be null.

I think the best solution would be to split this function, so that the flush cache 
operation doesn't have the ptr argument at all.

An easy short-term solution could be to move the increment of buf into the WRITE_DATA case 
of the switch.


r~

[PULL 05/23] ui/vnc: Fix crash when specifying [vnc] without id in the config file

[Thomas Huth]

From: Thomas Huth <thuth@redhat.com>

QEMU currently crashes when there is a [vnc] section in the config
file that does not have an "id = ..." line:

 $ echo "[vnc]" > /tmp/qemu.conf
 $ ./qemu-system-x86_64 -readconfig /tmp/qemu.conf
 qemu-system-x86_64: ../../devel/qemu/ui/vnc.c:4347: vnc_init_func:
  Assertion `id' failed.
 Aborted (core dumped)

The required "id" is only set up automatically while parsing the command
line, but not when reading the options from the config file.
Thus let's move code that automatically adds the id (if it does not
exist yet) to the init function that needs the id for the first time,
replacing the assert() statement there.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2836
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250821145130.845104-1-thuth@redhat.com>
---
 ui/vnc.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/ui/vnc.c b/ui/vnc.c
index 68ca4a68e7a..9054fc81253 100644
--- a/ui/vnc.c
+++ b/ui/vnc.c
@@ -4309,8 +4309,9 @@ void vnc_display_add_client(const char *id, int csock, bool skipauth)
     }
 }
 
-static void vnc_auto_assign_id(QemuOptsList *olist, QemuOpts *opts)
+static char *vnc_auto_assign_id(QemuOpts *opts)
 {
+    QemuOptsList *olist = qemu_find_opts("vnc");
     int i = 2;
     char *id;
 
@@ -4320,23 +4321,18 @@ static void vnc_auto_assign_id(QemuOptsList *olist, QemuOpts *opts)
         id = g_strdup_printf("vnc%d", i++);
     }
     qemu_opts_set_id(opts, id);
+
+    return id;
 }
 
 void vnc_parse(const char *str)
 {
     QemuOptsList *olist = qemu_find_opts("vnc");
     QemuOpts *opts = qemu_opts_parse_noisily(olist, str, !is_help_option(str));
-    const char *id;
 
     if (!opts) {
         exit(1);
     }
-
-    id = qemu_opts_id(opts);
-    if (!id) {
-        /* auto-assign id if not present */
-        vnc_auto_assign_id(olist, opts);
-    }
 }
 
 int vnc_init_func(void *opaque, QemuOpts *opts, Error **errp)
@@ -4344,7 +4340,11 @@ int vnc_init_func(void *opaque, QemuOpts *opts, Error **errp)
     Error *local_err = NULL;
     char *id = (char *)qemu_opts_id(opts);
 
-    assert(id);
+    if (!id) {
+        /* auto-assign id if not present */
+        id = vnc_auto_assign_id(opts);
+    }
+
     vnc_display_init(id, &local_err);
     if (local_err) {
         error_propagate(errp, local_err);
-- 
2.51.0

[PULL 06/23] tests/functional/m68k: Avoid ResourceWarning in the nextcube test

[Thomas Huth]

From: Thomas Huth <thuth@redhat.com>

Since commit c3fd296cf7b1 ("functional: always enable all python
warnings") we enabled more warnings for the functional tests. This
triggers now a warning in the nextcube test:

 tests/functional/m68k/test_nextcube.py:47: ResourceWarning:
  unclosed file <_io.BufferedReader name='tests/functional/m68k/test_nextcube.NextCubeMachine.test_bootrom_framebuffer_size/scratch/dump.ppm'>
   width, height = Image.open(screenshot_path).size

Use a proper "with" context to avoid it.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250829142000.62320-1-thuth@redhat.com>
---
 tests/functional/m68k/test_nextcube.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tests/functional/m68k/test_nextcube.py b/tests/functional/m68k/test_nextcube.py
index 13c72bd136a..c1610e58456 100755
--- a/tests/functional/m68k/test_nextcube.py
+++ b/tests/functional/m68k/test_nextcube.py
@@ -44,7 +44,8 @@ def test_bootrom_framebuffer_size(self):
         self.check_bootrom_framebuffer(screenshot_path)
 
         from PIL import Image
-        width, height = Image.open(screenshot_path).size
+        with Image.open(screenshot_path) as image:
+                width, height = image.size
         self.assertEqual(width, 1120)
         self.assertEqual(height, 832)
 
-- 
2.51.0

[PULL 07/23] tests/functional: enable force refresh of cached assets

[Thomas Huth]

From: Daniel P. Berrangé <berrange@redhat.com>

If the 'QEMU_TEST_REFRESH_CACHE' environment variable is set, then
ignore any existing cached asset and download a fresh copy.

This can be used to selectively refresh assets if set before running
a single test script.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20250829142616.2633254-2-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 docs/devel/testing/functional.rst   | 3 +++
 tests/functional/qemu_test/asset.py | 4 ++++
 2 files changed, 7 insertions(+)

diff --git a/docs/devel/testing/functional.rst b/docs/devel/testing/functional.rst
index 3728bab6c0c..fdeaebaadcb 100644
--- a/docs/devel/testing/functional.rst
+++ b/docs/devel/testing/functional.rst
@@ -312,6 +312,9 @@ The cache is populated in the ``~/.cache/qemu/download`` directory by
 default, but the location can be changed by setting the
 ``QEMU_TEST_CACHE_DIR`` environment variable.
 
+To force the test suite to re-download the cache, even if still valid,
+set the ``QEMU_TEST_REFRESH_CACHE`` environment variable.
+
 Skipping tests
 --------------
 
diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_test/asset.py
index 704b84d0ea6..b5a6136d365 100644
--- a/tests/functional/qemu_test/asset.py
+++ b/tests/functional/qemu_test/asset.py
@@ -72,6 +72,10 @@ def _check(self, cache_file):
         return self.hash == hl.hexdigest()
 
     def valid(self):
+        if os.getenv("QEMU_TEST_REFRESH_CACHE", None) is not None:
+            self.log.info("Force refresh of asset %s", self.url)
+            return False
+
         return self.cache_file.exists() and self._check(self.cache_file)
 
     def fetchable(self):
-- 
2.51.0

[PULL 08/23] tests/functional: fix formatting of exception args

[Thomas Huth]

From: Daniel P. Berrangé <berrange@redhat.com>

The catch-all exception handler forgot the placeholder for
the exception details.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20250829142616.2633254-3-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/functional/qemu_test/asset.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_test/asset.py
index b5a6136d365..5c74adf2241 100644
--- a/tests/functional/qemu_test/asset.py
+++ b/tests/functional/qemu_test/asset.py
@@ -173,7 +173,7 @@ def fetch(self):
                 continue
             except Exception as e:
                 tmp_cache_file.unlink()
-                raise AssetError(self, "Unable to download: " % e)
+                raise AssetError(self, "Unable to download: %s" % e)
 
         if not os.path.exists(tmp_cache_file):
             raise AssetError(self, "Download retries exceeded", transient=True)
-- 
2.51.0

[PULL 09/23] tests/functional: handle URLError when fetching assets

[Thomas Huth]

From: Daniel P. Berrangé <berrange@redhat.com>

We treat most HTTP errors as non-fatal when fetching assets,
but forgot to handle network level errors. This adds catching
of URLError so that we retry on failure, and will ultimately
trigger graceful skipping in the pre-cache task.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250829142616.2633254-4-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/functional/qemu_test/asset.py | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_test/asset.py
index 5c74adf2241..2dd32bf28d9 100644
--- a/tests/functional/qemu_test/asset.py
+++ b/tests/functional/qemu_test/asset.py
@@ -15,7 +15,7 @@
 from time import sleep
 from pathlib import Path
 from shutil import copyfileobj
-from urllib.error import HTTPError
+from urllib.error import HTTPError, URLError
 
 class AssetError(Exception):
     def __init__(self, asset, msg, transient=False):
@@ -171,6 +171,14 @@ def fetch(self):
                     raise AssetError(self, "Unable to download: "
                                      "HTTP error %d" % e.code)
                 continue
+            except URLError as e:
+                # This is typically a network/service level error
+                # eg urlopen error [Errno 110] Connection timed out>
+                tmp_cache_file.unlink()
+                self.log.error("Unable to download %s: URL error %s",
+                               self.url, e.reason)
+                raise AssetError(self, "Unable to download: URL error %s" %
+                                 e.reason, transient=True)
             except Exception as e:
                 tmp_cache_file.unlink()
                 raise AssetError(self, "Unable to download: %s" % e)
-- 
2.51.0

[PULL 10/23] tests/functional/arm: Update test ASPEED SDK v03.02 for AST1030

[Thomas Huth]

From: Kane-Chen-AS <kane_chen@aspeedtech.com>

Signed-off-by: Kane-Chen-AS <kane_chen@aspeedtech.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Message-ID: <20250904100556.1729604-2-kane_chen@aspeedtech.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/functional/arm/test_aspeed_ast1030.py | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/tests/functional/arm/test_aspeed_ast1030.py b/tests/functional/arm/test_aspeed_ast1030.py
index 77037f01793..42126b514ff 100755
--- a/tests/functional/arm/test_aspeed_ast1030.py
+++ b/tests/functional/arm/test_aspeed_ast1030.py
@@ -12,17 +12,17 @@
 
 class AST1030Machine(LinuxKernelTest):
 
-    ASSET_ZEPHYR_3_00 = Asset(
+    ASSET_ZEPHYR_3_02 = Asset(
         ('https://github.com/AspeedTech-BMC'
-         '/zephyr/releases/download/v00.03.00/ast1030-evb-demo.zip'),
-        '37fe3ecd4a1b9d620971a15b96492a81093435396eeac69b6f3e384262ff555f')
+         '/zephyr/releases/download/v00.03.02/ast1030-evb-demo.zip'),
+        '1ec83caab3ddd5d09481772801be7210e222cb015ce22ec6fffb8a76956dcd4f')
 
-    def test_ast1030_zephyros_3_00(self):
+    def test_ast1030_zephyros_3_02(self):
         self.set_machine('ast1030-evb')
 
-        kernel_name = "ast1030-evb-demo/zephyr.elf"
+        kernel_name = "ast1030-evb-demo-3/zephyr.elf"
         kernel_file = self.archive_extract(
-            self.ASSET_ZEPHYR_3_00, member=kernel_name)
+            self.ASSET_ZEPHYR_3_02, member=kernel_name)
 
         self.vm.set_console()
         self.vm.add_args('-kernel', kernel_file, '-nographic')
-- 
2.51.0

[PULL 11/23] tests/functional/arm: Update test ASPEED SDK v09.07 for AST2500

[Thomas Huth]

From: Kane-Chen-AS <kane_chen@aspeedtech.com>

Signed-off-by: Kane-Chen-AS <kane_chen@aspeedtech.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Message-ID: <20250904100556.1729604-3-kane_chen@aspeedtech.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/functional/arm/test_aspeed_ast2500.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/tests/functional/arm/test_aspeed_ast2500.py b/tests/functional/arm/test_aspeed_ast2500.py
index 6923fe87017..4fdd81e2f95 100755
--- a/tests/functional/arm/test_aspeed_ast2500.py
+++ b/tests/functional/arm/test_aspeed_ast2500.py
@@ -37,14 +37,14 @@ def test_arm_ast2500_evb_buildroot(self):
 
         self.do_test_arm_aspeed_buildroot_poweroff()
 
-    ASSET_SDK_V906_AST2500 = Asset(
-        'https://github.com/AspeedTech-BMC/openbmc/releases/download/v09.06/ast2500-default-obmc.tar.gz',
-        '542db84645b4efd8aed50385d7f4dd1caff379a987032311cfa7b563a3addb2a')
+    ASSET_SDK_V907_AST2500 = Asset(
+        'https://github.com/AspeedTech-BMC/openbmc/releases/download/v09.07/ast2500-default-obmc.tar.gz',
+        'd52bcc279a37c8d7679b3e4ef22cc77c36f0f6624c687b37334f798828afb077')
 
     def test_arm_ast2500_evb_sdk(self):
         self.set_machine('ast2500-evb')
 
-        self.archive_extract(self.ASSET_SDK_V906_AST2500)
+        self.archive_extract(self.ASSET_SDK_V907_AST2500)
 
         self.do_test_arm_aspeed_sdk_start(
             self.scratch_file("ast2500-default", "image-bmc"))
-- 
2.51.0

[PULL 12/23] tests/functional/arm: Update test ASPEED SDK v09.07 for AST2600

[Thomas Huth]

From: Kane-Chen-AS <kane_chen@aspeedtech.com>

Signed-off-by: Kane-Chen-AS <kane_chen@aspeedtech.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Message-ID: <20250904100556.1729604-4-kane_chen@aspeedtech.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/functional/arm/test_aspeed_ast2600.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/tests/functional/arm/test_aspeed_ast2600.py b/tests/functional/arm/test_aspeed_ast2600.py
index fdae4c939d8..129695ca4ec 100755
--- a/tests/functional/arm/test_aspeed_ast2600.py
+++ b/tests/functional/arm/test_aspeed_ast2600.py
@@ -97,14 +97,14 @@ def test_arm_ast2600_evb_buildroot_tpm(self):
 
         self.do_test_arm_aspeed_buildroot_poweroff()
 
-    ASSET_SDK_V906_AST2600 = Asset(
-        'https://github.com/AspeedTech-BMC/openbmc/releases/download/v09.06/ast2600-default-obmc.tar.gz',
-        '768d76e247896ad78c154b9cff4f766da2ce65f217d620b286a4a03a8a4f68f5')
+    ASSET_SDK_V907_AST2600 = Asset(
+        'https://github.com/AspeedTech-BMC/openbmc/releases/download/v09.07/ast2600-default-obmc.tar.gz',
+        'cb6c08595bcbba1672ce716b068ba4e48eda1ed9abe78a07b30392ba2278feba')
 
     def test_arm_ast2600_evb_sdk(self):
         self.set_machine('ast2600-evb')
 
-        self.archive_extract(self.ASSET_SDK_V906_AST2600)
+        self.archive_extract(self.ASSET_SDK_V907_AST2600)
 
         self.vm.add_args('-device',
             'tmp105,bus=aspeed.i2c.bus.5,address=0x4d,id=tmp-test')
-- 
2.51.0

[PULL 13/23] tests/functional/arm: Update test ASPEED SDK v09.07 for AST2700 vbootrom

[Thomas Huth]

From: Kane-Chen-AS <kane_chen@aspeedtech.com>

Signed-off-by: Kane-Chen-AS <kane_chen@aspeedtech.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Message-ID: <20250904100556.1729604-5-kane_chen@aspeedtech.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/functional/aarch64/test_aspeed_ast2700.py | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/tests/functional/aarch64/test_aspeed_ast2700.py b/tests/functional/aarch64/test_aspeed_ast2700.py
index d02dc7991c1..8a08bc46888 100755
--- a/tests/functional/aarch64/test_aspeed_ast2700.py
+++ b/tests/functional/aarch64/test_aspeed_ast2700.py
@@ -54,6 +54,10 @@ def verify_openbmc_boot_and_login(self, name):
             'https://github.com/AspeedTech-BMC/openbmc/releases/download/v09.06/ast2700-default-obmc.tar.gz',
             'f1d53e0be8a404ecce3e105f72bc50fa4e090ad13160ffa91b10a6e0233a9dc6')
 
+    ASSET_SDK_V907_AST2700A1_VBOOROM = Asset(
+            'https://github.com/AspeedTech-BMC/openbmc/releases/download/v09.07/ast2700-default-obmc.tar.gz',
+            '6e9e0c4b13e0f26040eca3f4a7f17cf09fc0f5c37c820500ff79370cc3c44add')
+
     def do_ast2700_i2c_test(self):
         exec_command_and_wait_for_pattern(self,
             'echo lm75 0x4d > /sys/class/i2c-dev/i2c-1/device/new_device ',
@@ -127,10 +131,10 @@ def test_aarch64_ast2700a1_evb_sdk_v09_06(self):
         self.verify_openbmc_boot_and_login('ast2700-default')
         self.do_ast2700_i2c_test()
 
-    def test_aarch64_ast2700a1_evb_sdk_vbootrom_v09_06(self):
+    def test_aarch64_ast2700a1_evb_sdk_vbootrom_v09_07(self):
         self.set_machine('ast2700a1-evb')
 
-        self.archive_extract(self.ASSET_SDK_V906_AST2700A1)
+        self.archive_extract(self.ASSET_SDK_V907_AST2700A1_VBOOROM)
         self.start_ast2700_test_vbootrom('ast2700-default')
         self.verify_vbootrom_firmware_flow()
         self.verify_openbmc_boot_and_login('ast2700-default')
-- 
2.51.0

[PULL 14/23] gitlab: replace avocado results files with meson results files

[Thomas Huth]

From: Daniel P. Berrangé <berrange@redhat.com>

The 'results.xml' file and 'test-results' directory were both outputs
of the avovcado test runner. Since we're now using meson with the new
functional test framework, we must reference meson results files as the
CI artifacts.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <20250908190901.3571859-2-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.d/buildtest-template.yml | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index 038c3c9540a..46722982149 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -104,11 +104,10 @@
     when: always
     expire_in: 7 days
     paths:
-      - build/tests/results/latest/results.xml
-      - build/tests/results/latest/test-results
+      - build/meson-logs/testlog.txt
       - build/tests/functional/*/*/*.log
     reports:
-      junit: build/tests/results/latest/results.xml
+      junit: build/meson-logs/testlog.junit.xml
   before_script:
     - export QEMU_TEST_ALLOW_UNTRUSTED_CODE=1
     - export QEMU_TEST_CACHE_DIR=${CI_PROJECT_DIR}/functional-cache
-- 
2.51.0

[PULL 15/23] gitlab: always include entire of meson-logs directory

[Thomas Huth]

From: Daniel P. Berrangé <berrange@redhat.com>

There are files besides testlog.txt that may be useful as published
CI artifacts.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <20250908190901.3571859-3-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.d/buildtest-template.yml  | 4 ++--
 .gitlab-ci.d/buildtest.yml           | 2 +-
 .gitlab-ci.d/crossbuild-template.yml | 2 +-
 .gitlab-ci.d/windows.yml             | 2 +-
 4 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index 46722982149..4cc5b20790f 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -88,7 +88,7 @@
     when: always
     expire_in: 7 days
     paths:
-      - build/meson-logs/testlog.txt
+      - build/meson-logs
     reports:
       junit: build/meson-logs/testlog.junit.xml
 
@@ -104,7 +104,7 @@
     when: always
     expire_in: 7 days
     paths:
-      - build/meson-logs/testlog.txt
+      - build/meson-logs
       - build/tests/functional/*/*/*.log
     reports:
       junit: build/meson-logs/testlog.junit.xml
diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index d888a600637..778289267f1 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -613,7 +613,7 @@ gcov:
     when: always
     expire_in: 2 days
     paths:
-      - build/meson-logs/testlog.txt
+      - build/meson-logs
     reports:
       junit: build/meson-logs/testlog.junit.xml
       coverage_report:
diff --git a/.gitlab-ci.d/crossbuild-template.yml b/.gitlab-ci.d/crossbuild-template.yml
index 303943f818f..7e70376cfcc 100644
--- a/.gitlab-ci.d/crossbuild-template.yml
+++ b/.gitlab-ci.d/crossbuild-template.yml
@@ -128,6 +128,6 @@
     when: always
     expire_in: 7 days
     paths:
-      - build/meson-logs/testlog.txt
+      - build/meson-logs
     reports:
       junit: build/meson-logs/testlog.junit.xml
diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
index beac39e5bde..f14e9ca1341 100644
--- a/.gitlab-ci.d/windows.yml
+++ b/.gitlab-ci.d/windows.yml
@@ -24,7 +24,7 @@ msys2-64bit:
     name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
     expire_in: 7 days
     paths:
-      - build/meson-logs/testlog.txt
+      - build/meson-logs
     reports:
       junit: "build/meson-logs/testlog.junit.xml"
   before_script:
-- 
2.51.0

[PULL 16/23] gitlab: include all junit XML files from meson

[Thomas Huth]

From: Daniel P. Berrangé <berrange@redhat.com>

The junit XML file produced by meson does not always have the
name 'testlog.junit.xml' - in the case of 'make check-functional'
there is a 'testlog-thorough.junit.xml' file too.

Improve CI debugging robustness by capturing all junit files that
meson produces.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <20250908190901.3571859-4-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.d/buildtest-template.yml  | 4 ++--
 .gitlab-ci.d/buildtest.yml           | 2 +-
 .gitlab-ci.d/crossbuild-template.yml | 2 +-
 .gitlab-ci.d/custom-runners.yml      | 2 +-
 .gitlab-ci.d/windows.yml             | 2 +-
 5 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index 4cc5b20790f..308490a35ac 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -90,7 +90,7 @@
     paths:
       - build/meson-logs
     reports:
-      junit: build/meson-logs/testlog.junit.xml
+      junit: build/meson-logs/*.junit.xml
 
 .functional_test_job_template:
   extends: .common_test_job_template
@@ -107,7 +107,7 @@
       - build/meson-logs
       - build/tests/functional/*/*/*.log
     reports:
-      junit: build/meson-logs/testlog.junit.xml
+      junit: build/meson-logs/*.junit.xml
   before_script:
     - export QEMU_TEST_ALLOW_UNTRUSTED_CODE=1
     - export QEMU_TEST_CACHE_DIR=${CI_PROJECT_DIR}/functional-cache
diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index 778289267f1..83c2867295a 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -615,7 +615,7 @@ gcov:
     paths:
       - build/meson-logs
     reports:
-      junit: build/meson-logs/testlog.junit.xml
+      junit: build/meson-logs/*.junit.xml
       coverage_report:
         coverage_format: cobertura
         path: build/coverage.xml
diff --git a/.gitlab-ci.d/crossbuild-template.yml b/.gitlab-ci.d/crossbuild-template.yml
index 7e70376cfcc..58136d06e4a 100644
--- a/.gitlab-ci.d/crossbuild-template.yml
+++ b/.gitlab-ci.d/crossbuild-template.yml
@@ -130,4 +130,4 @@
     paths:
       - build/meson-logs
     reports:
-      junit: build/meson-logs/testlog.junit.xml
+      junit: build/meson-logs/*.junit.xml
diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
index 1aa3c60efe9..2d493f70f7a 100644
--- a/.gitlab-ci.d/custom-runners.yml
+++ b/.gitlab-ci.d/custom-runners.yml
@@ -26,7 +26,7 @@
       - build/build.ninja
       - build/meson-logs
     reports:
-      junit: build/meson-logs/testlog.junit.xml
+      junit: build/meson-logs/*.junit.xml
 
 include:
   - local: '/.gitlab-ci.d/custom-runners/ubuntu-22.04-s390x.yml'
diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
index f14e9ca1341..1e6a01bd9ac 100644
--- a/.gitlab-ci.d/windows.yml
+++ b/.gitlab-ci.d/windows.yml
@@ -26,7 +26,7 @@ msys2-64bit:
     paths:
       - build/meson-logs
     reports:
-      junit: "build/meson-logs/testlog.junit.xml"
+      junit: build/meson-logs/*.junit.xml
   before_script:
   - Write-Output "Acquiring msys2.exe installer at $(Get-Date -Format u)"
   - If ( !(Test-Path -Path msys64\var\cache ) ) {
-- 
2.51.0

[PULL 17/23] gitlab: prevent duplicated meson log artifacts in test jobs

[Thomas Huth]

From: Daniel P. Berrangé <berrange@redhat.com>

The build jobs will populate build/meson-logs/ with various files
that are added as artifacts.

The test jobs preserve the state of the build jobs, so we must
delete any pre-existing logs to prevent confusion from duplicate
artifacts.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <20250908190901.3571859-5-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.d/buildtest-template.yml | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index 308490a35ac..d866cb12bb1 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -83,6 +83,10 @@
 
 .native_test_job_template:
   extends: .common_test_job_template
+  before_script:
+    # Prevent logs from the build job that run earlier
+    # from being duplicated in the test job artifacts
+    - rm -f build/meson-logs/*
   artifacts:
     name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
     when: always
@@ -111,6 +115,9 @@
   before_script:
     - export QEMU_TEST_ALLOW_UNTRUSTED_CODE=1
     - export QEMU_TEST_CACHE_DIR=${CI_PROJECT_DIR}/functional-cache
+    # Prevent logs from the build job that run earlier
+    # from being duplicated in the test job artifacts
+    - rm -f build/meson-logs/*
   after_script:
     - cd build
     - du -chs ${CI_PROJECT_DIR}/*-cache
-- 
2.51.0

[PULL 18/23] tests/functional: return output from cmd.py helpers

[Thomas Huth]

From: John Levon <john.levon@nutanix.com>

Tests might want to look at the whole output from a command execution,
as well as just logging it. Add support for this.

Signed-off-by: John Levon <john.levon@nutanix.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250903201931.168317-2-john.levon@nutanix.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/functional/qemu_test/cmd.py | 53 +++++++++++++++++++++++++------
 1 file changed, 44 insertions(+), 9 deletions(-)

diff --git a/tests/functional/qemu_test/cmd.py b/tests/functional/qemu_test/cmd.py
index dc5f422b77d..c19dfc577ff 100644
--- a/tests/functional/qemu_test/cmd.py
+++ b/tests/functional/qemu_test/cmd.py
@@ -45,6 +45,9 @@ def is_readable_executable_file(path):
 # If end of line is seen, with neither @success or @failure
 # return False
 #
+# In both cases, also return the contents of the line (in bytes)
+# up to that point.
+#
 # If @failure is seen, then mark @test as failed
 def _console_read_line_until_match(test, vm, success, failure):
     msg = bytes([])
@@ -76,10 +79,23 @@ def _console_read_line_until_match(test, vm, success, failure):
     except:
         console_logger.debug(msg)
 
-    return done
+    return done, msg
 
 def _console_interaction(test, success_message, failure_message,
                          send_string, keep_sending=False, vm=None):
+    """
+    Interact with the console until either message is seen.
+
+    :param success_message: if this message appears, finish interaction
+    :param failure_message: if this message appears, test fails
+    :param send_string: a string to send to the console before trying
+                        to read a new line
+    :param keep_sending: keep sending the send string each time
+    :param vm: the VM to interact with
+
+    :return: The collected output (in bytes form).
+    """
+
     assert not keep_sending or send_string
     assert success_message or send_string
 
@@ -101,6 +117,8 @@ def _console_interaction(test, success_message, failure_message,
     if failure_message is not None:
         failure_message_b = failure_message.encode()
 
+    out = bytes([])
+
     while True:
         if send_string:
             vm.console_socket.sendall(send_string.encode())
@@ -113,11 +131,17 @@ def _console_interaction(test, success_message, failure_message,
                 break
             continue
 
-        if _console_read_line_until_match(test, vm,
-                                          success_message_b,
-                                          failure_message_b):
+        done, line = _console_read_line_until_match(test, vm,
+                                                    success_message_b,
+                                                    failure_message_b)
+
+        out += line
+
+        if done:
             break
 
+    return out
+
 def interrupt_interactive_console_until_pattern(test, success_message,
                                                 failure_message=None,
                                                 interrupt_string='\r'):
@@ -140,10 +164,12 @@ def interrupt_interactive_console_until_pattern(test, success_message,
     :param failure_message: if this message appears, test fails
     :param interrupt_string: a string to send to the console before trying
                              to read a new line
+
+    :return: The collected output (in bytes form).
     """
     assert success_message
-    _console_interaction(test, success_message, failure_message,
-                         interrupt_string, True)
+    return _console_interaction(test, success_message, failure_message,
+                                interrupt_string, True)
 
 def wait_for_console_pattern(test, success_message, failure_message=None,
                              vm=None):
@@ -155,9 +181,12 @@ def wait_for_console_pattern(test, success_message, failure_message=None,
     :type test: :class:`qemu_test.QemuSystemTest`
     :param success_message: if this message appears, test succeeds
     :param failure_message: if this message appears, test fails
+
+    :return: The collected output (in bytes form).
     """
     assert success_message
-    _console_interaction(test, success_message, failure_message, None, vm=vm)
+    return _console_interaction(test, success_message, failure_message,
+                                None, vm=vm)
 
 def exec_command(test, command):
     """
@@ -168,8 +197,10 @@ def exec_command(test, command):
     :type test: :class:`qemu_test.QemuSystemTest`
     :param command: the command to send
     :type command: str
+
+    :return: The collected output (in bytes form).
     """
-    _console_interaction(test, None, None, command + '\r')
+    return _console_interaction(test, None, None, command + '\r')
 
 def exec_command_and_wait_for_pattern(test, command,
                                       success_message, failure_message=None):
@@ -184,9 +215,13 @@ def exec_command_and_wait_for_pattern(test, command,
     :param command: the command to send
     :param success_message: if this message appears, test succeeds
     :param failure_message: if this message appears, test fails
+
+    :return: The collected output (in bytes form).
     """
     assert success_message
-    _console_interaction(test, success_message, failure_message, command + '\r')
+
+    return _console_interaction(test, success_message, failure_message,
+                                command + '\r')
 
 def get_qemu_img(test):
     test.log.debug('Looking for and selecting a qemu-img binary')
-- 
2.51.0

[PULL 19/23] tests/functional: add vm param to cmd.py helpers

[Thomas Huth]

From: John Levon <john.levon@nutanix.com>

Extend the "vm" parameter of wait_for_console_pattern() to all the other
utility functions; this allows them to be used on a VM other than
test.vm.

Signed-off-by: John Levon <john.levon@nutanix.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250903201931.168317-3-john.levon@nutanix.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/functional/qemu_test/cmd.py | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/tests/functional/qemu_test/cmd.py b/tests/functional/qemu_test/cmd.py
index c19dfc577ff..8069c89730b 100644
--- a/tests/functional/qemu_test/cmd.py
+++ b/tests/functional/qemu_test/cmd.py
@@ -144,7 +144,8 @@ def _console_interaction(test, success_message, failure_message,
 
 def interrupt_interactive_console_until_pattern(test, success_message,
                                                 failure_message=None,
-                                                interrupt_string='\r'):
+                                                interrupt_string='\r',
+                                                vm=None):
     """
     Keep sending a string to interrupt a console prompt, while logging the
     console output. Typical use case is to break a boot loader prompt, such:
@@ -164,12 +165,13 @@ def interrupt_interactive_console_until_pattern(test, success_message,
     :param failure_message: if this message appears, test fails
     :param interrupt_string: a string to send to the console before trying
                              to read a new line
+    :param vm: VM to use
 
     :return: The collected output (in bytes form).
     """
     assert success_message
     return _console_interaction(test, success_message, failure_message,
-                                interrupt_string, True)
+                                interrupt_string, True, vm=vm)
 
 def wait_for_console_pattern(test, success_message, failure_message=None,
                              vm=None):
@@ -181,6 +183,7 @@ def wait_for_console_pattern(test, success_message, failure_message=None,
     :type test: :class:`qemu_test.QemuSystemTest`
     :param success_message: if this message appears, test succeeds
     :param failure_message: if this message appears, test fails
+    :param vm: VM to use
 
     :return: The collected output (in bytes form).
     """
@@ -188,7 +191,7 @@ def wait_for_console_pattern(test, success_message, failure_message=None,
     return _console_interaction(test, success_message, failure_message,
                                 None, vm=vm)
 
-def exec_command(test, command):
+def exec_command(test, command, vm=None):
     """
     Send a command to a console (appending CRLF characters), while logging
     the content.
@@ -196,14 +199,16 @@ def exec_command(test, command):
     :param test: a test containing a VM.
     :type test: :class:`qemu_test.QemuSystemTest`
     :param command: the command to send
+    :param vm: VM to use
     :type command: str
 
     :return: The collected output (in bytes form).
     """
-    return _console_interaction(test, None, None, command + '\r')
+    return _console_interaction(test, None, None, command + '\r', vm=vm)
 
 def exec_command_and_wait_for_pattern(test, command,
-                                      success_message, failure_message=None):
+                                      success_message, failure_message=None,
+                                      vm=None):
     """
     Send a command to a console (appending CRLF characters), then wait
     for success_message to appear on the console, while logging the.
@@ -215,13 +220,14 @@ def exec_command_and_wait_for_pattern(test, command,
     :param command: the command to send
     :param success_message: if this message appears, test succeeds
     :param failure_message: if this message appears, test fails
+    :param vm: VM to use
 
     :return: The collected output (in bytes form).
     """
     assert success_message
 
     return _console_interaction(test, success_message, failure_message,
-                                command + '\r')
+                                command + '\r', vm=vm)
 
 def get_qemu_img(test):
     test.log.debug('Looking for and selecting a qemu-img binary')
-- 
2.51.0

[PULL 20/23] tests/functional: fix infinite loop on console EOF

[Thomas Huth]

From: Daniel P. Berrangé <berrange@redhat.com>

The 'recv' method will return an empty byte array, not None, when
the socket has EOF.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <20250908135722.3375580-2-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/functional/qemu_test/cmd.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/functional/qemu_test/cmd.py b/tests/functional/qemu_test/cmd.py
index 8069c89730b..f544566245b 100644
--- a/tests/functional/qemu_test/cmd.py
+++ b/tests/functional/qemu_test/cmd.py
@@ -54,7 +54,7 @@ def _console_read_line_until_match(test, vm, success, failure):
     done = False
     while True:
         c = vm.console_socket.recv(1)
-        if c is None:
+        if not c:
             done = True
             test.fail(
                 f"EOF in console, expected '{success}'")
-- 
2.51.0

[PULL 21/23] tests/functional: avoid duplicate messages on failures

[Thomas Huth]

From: Daniel P. Berrangé <berrange@redhat.com>

In some scenarios the same tests is mentioned in both the
'res.results.errors' and 'res.results.failures' array returned
by unittest.main(). This was seen when the 'tearDown' method
raised an exception.

In such a case, we printed out the same information about where
to find a log file twice for each test. Track which tests we
have already reported on, to avoid the duplication.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250908135722.3375580-3-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/functional/qemu_test/testcase.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu_test/testcase.py
index fbeb1710587..82a7724404b 100644
--- a/tests/functional/qemu_test/testcase.py
+++ b/tests/functional/qemu_test/testcase.py
@@ -251,13 +251,14 @@ def main():
                                    test_output_log = pycotap.LogMode.LogToError)
         res = unittest.main(module = None, testRunner = tr, exit = False,
                             argv=[sys.argv[0], path] + sys.argv[1:])
+        failed = {}
         for (test, message) in res.result.errors + res.result.failures:
-
-            if hasattr(test, "log_filename"):
+            if hasattr(test, "log_filename") and not test.id() in failed:
                 print('More information on ' + test.id() + ' could be found here:'
                       '\n %s' % test.log_filename, file=sys.stderr)
                 if hasattr(test, 'console_log_name'):
                     print(' %s' % test.console_log_name, file=sys.stderr)
+                failed[test.id()] = True
         sys.exit(not res.result.wasSuccessful())
 
 
-- 
2.51.0

[PULL 22/23] tests/functional: avoid tearDown failure when QEMU dies

[Thomas Huth]

From: Daniel P. Berrangé <berrange@redhat.com>

In a QEMU process under test dies unexpectedly, the 'shutdown'
method may well raise an exception. This causes the tearDown
method to fail, which means any later cleanup code fails to
get run. Most notably the log handlers don't get removed so
the base.log file from an earlier test will get polluted with
messages from any subsequent tests. The tearDown failure also
results in pages of exceptions printed on the console, which
obscures the real failure message / trace printed by the test.

Ignore any shutdown failures in the tearDown method, since any
test which cares about clean shutdown should have already
cleaned up any running VMs. The tearDown method is just there
as a safety net to cleanup resources. The base.log file will
still containing log messages from the failed 'vm.shutdown'
call too.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250908135722.3375580-4-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/functional/qemu_test/testcase.py | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu_test/testcase.py
index 82a7724404b..faa0a4f0db2 100644
--- a/tests/functional/qemu_test/testcase.py
+++ b/tests/functional/qemu_test/testcase.py
@@ -404,7 +404,10 @@ def set_vm_arg(self, arg, value):
 
     def tearDown(self):
         for vm in self._vms.values():
-            vm.shutdown()
+            try:
+                vm.shutdown()
+            except Exception as ex:
+                self.log.error("Failed to teardown VM: %s" % ex)
         logging.getLogger('console').removeHandler(self._console_log_fh)
         self._console_log_fh.close()
         super().tearDown()
-- 
2.51.0

[PULL 23/23] tests/functional: purge scratch dir on test startup

[Thomas Huth]

From: Daniel P. Berrangé <berrange@redhat.com>

The test suite purges the scratch dir in the tearDown method, but
if python crashes (or is non-gracefully killed) this won't get run.
Also the user can set QEMU_TEST_KEEP_SCRATCH to disable cleanup.

Purging the scratch dir on startup ensures that tests always run
from a clean state.

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <20250908135722.3375580-5-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/functional/qemu_test/testcase.py | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu_test/testcase.py
index faa0a4f0db2..2c0abde3957 100644
--- a/tests/functional/qemu_test/testcase.py
+++ b/tests/functional/qemu_test/testcase.py
@@ -205,6 +205,10 @@ def setUp(self):
         self.outputdir = self.build_file('tests', 'functional',
                                          self.arch, self.id())
         self.workdir = os.path.join(self.outputdir, 'scratch')
+        if os.path.exists(self.workdir):
+            # Purge as safety net in case of unclean termination of
+            # previous test, or use of QEMU_TEST_KEEP_SCRATCH
+            shutil.rmtree(self.workdir)
         os.makedirs(self.workdir, exist_ok=True)
 
         self.log_filename = self.log_file('base.log')
-- 
2.51.0

Re: [PULL 00/23] Functional tests and misc patches

[Peter Maydell]

On Tue, 9 Sept 2025 at 14:54, Thomas Huth <thuth@redhat.com> wrote:
>
>  Hi Richard!
>
> The following changes since commit 6a9fa5ef3230a7d51e0d953a59ee9ef10af705b8:
>
>   Merge tag 'pull-tcg-20250905' of https://gitlab.com/rth7680/qemu into staging (2025-09-05 09:51:27 +0200)
>
> are available in the Git repository at:
>
>   https://gitlab.com/thuth/qemu.git tags/pull-request-2025-09-09
>
> for you to fetch changes up to 2fc170bcdc4d2f05534c68572b4f72a7d18c2119:
>
>   tests/functional: purge scratch dir on test startup (2025-09-09 15:45:33 +0200)
>
> ----------------------------------------------------------------
> * Silence warnings from the undefined-behaviour sanitizer
> * Many small improvements to various functional tests
> * Remove remainders from storing avocado artifacts in the Gitlab CI
> * Keep more meson log files as artifacts in the Gitlab CI instead
> * Re-enable -fzero-call-used-regs on OpenBSD
>



Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/10.2
for any user-visible changes.

-- PMM