* [PATCH 01/15] gitlab-ci: Remove the avocado tests from the CI pipelines
2025-03-25 20:00 [PATCH for-10.1 00/15] Convert remaining Avocado tests to functional Thomas Huth
@ 2025-03-25 20:00 ` Thomas Huth
2025-03-26 9:52 ` Daniel P. Berrangé
2025-03-25 20:00 ` [PATCH 02/15] tests/functional: Move the check for the parameters from avocado to functional Thomas Huth
` (13 subsequent siblings)
14 siblings, 1 reply; 34+ messages in thread
From: Thomas Huth @ 2025-03-25 20:00 UTC (permalink / raw)
To: qemu-devel; +Cc: Daniel P. Berrangé, Philippe Mathieu-Daudé
From: Thomas Huth <thuth@redhat.com>
We are going to move the remaining Avocado tests step by step
into the functional test framework. Unfortunately, Avocado fails
with an error if it cannot determine a test to run, so disable
the tests here now to avoid failures in the Gitlab-CI during the
next steps.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
.gitlab-ci.d/buildtest-template.yml | 11 ----------
.gitlab-ci.d/buildtest.yml | 33 +++++++++++------------------
2 files changed, 12 insertions(+), 32 deletions(-)
diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index 39da7698b09..13fa4f4a4fc 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -95,7 +95,6 @@
cache:
key: "${CI_JOB_NAME}-cache"
paths:
- - ${CI_PROJECT_DIR}/avocado-cache
- ${CI_PROJECT_DIR}/functional-cache
policy: pull-push
artifacts:
@@ -109,16 +108,6 @@
reports:
junit: build/tests/results/latest/results.xml
before_script:
- - mkdir -p ~/.config/avocado
- - echo "[datadir.paths]" > ~/.config/avocado/avocado.conf
- - echo "cache_dirs = ['${CI_PROJECT_DIR}/avocado-cache']"
- >> ~/.config/avocado/avocado.conf
- - echo -e '[job.output.testlogs]\nstatuses = ["FAIL", "INTERRUPT"]'
- >> ~/.config/avocado/avocado.conf
- - if [ -d ${CI_PROJECT_DIR}/avocado-cache ]; then
- du -chs ${CI_PROJECT_DIR}/*-cache ;
- fi
- - export AVOCADO_ALLOW_UNTRUSTED_CODE=1
- export QEMU_TEST_ALLOW_UNTRUSTED_CODE=1
- export QEMU_TEST_CACHE_DIR=${CI_PROJECT_DIR}/functional-cache
after_script:
diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index 00f4bfcd9f3..431bc07d8fb 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -29,8 +29,7 @@ functional-system-alpine:
artifacts: true
variables:
IMAGE: alpine
- MAKE_CHECK_ARGS: check-avocado check-functional
- AVOCADO_TAGS: arch:avr arch:loongarch64 arch:mips64 arch:mipsel
+ MAKE_CHECK_ARGS: check-functional
build-system-ubuntu:
extends:
@@ -60,8 +59,7 @@ functional-system-ubuntu:
artifacts: true
variables:
IMAGE: ubuntu2204
- MAKE_CHECK_ARGS: check-avocado check-functional
- AVOCADO_TAGS: arch:alpha arch:microblazeel arch:mips64el
+ MAKE_CHECK_ARGS: check-functional
build-system-debian:
extends:
@@ -92,8 +90,7 @@ functional-system-debian:
artifacts: true
variables:
IMAGE: debian
- MAKE_CHECK_ARGS: check-avocado check-functional
- AVOCADO_TAGS: arch:arm arch:i386 arch:riscv64 arch:sh4 arch:sparc arch:xtensa
+ MAKE_CHECK_ARGS: check-functional
crash-test-debian:
extends: .native_test_job_template
@@ -155,9 +152,7 @@ functional-system-fedora:
artifacts: true
variables:
IMAGE: fedora
- MAKE_CHECK_ARGS: check-avocado check-functional
- AVOCADO_TAGS: arch:microblaze arch:mips arch:xtensa arch:m68k
- arch:riscv32 arch:ppc arch:sparc64
+ MAKE_CHECK_ARGS: check-functional
crash-test-fedora:
extends: .native_test_job_template
@@ -278,9 +273,7 @@ functional-system-centos:
artifacts: true
variables:
IMAGE: centos9
- MAKE_CHECK_ARGS: check-avocado check-functional
- AVOCADO_TAGS: arch:ppc64 arch:or1k arch:s390x arch:x86_64 arch:rx
- arch:sh4
+ MAKE_CHECK_ARGS: check-functional
build-system-opensuse:
extends:
@@ -309,8 +302,7 @@ functional-system-opensuse:
artifacts: true
variables:
IMAGE: opensuse-leap
- MAKE_CHECK_ARGS: check-avocado check-functional
- AVOCADO_TAGS: arch:s390x arch:x86_64 arch:aarch64
+ MAKE_CHECK_ARGS: check-functional
#
# Flaky tests. We don't run these by default and they are allow fail
@@ -338,10 +330,9 @@ functional-system-flaky:
allow_failure: true
variables:
IMAGE: debian
- MAKE_CHECK_ARGS: check-avocado check-functional
+ MAKE_CHECK_ARGS: check-functional
QEMU_JOB_OPTIONAL: 1
QEMU_TEST_FLAKY_TESTS: 1
- AVOCADO_TAGS: flaky
# This jobs explicitly disable TCG (--disable-tcg), KVM is detected by
# the configure script. The container doesn't contain Xen headers so
@@ -482,8 +473,8 @@ clang-user:
# Since slirp callbacks are used in QEMU Timers, we cannot use libslirp with
# CFI builds, and thus have to disable it here.
#
-# Split in three sets of build/check/avocado to limit the execution time of each
-# job
+# Split in three sets of build/check/functional to limit the execution time
+# of each job
build-cfi-aarch64:
extends:
- .native_build_job_template
@@ -520,7 +511,7 @@ functional-cfi-aarch64:
artifacts: true
variables:
IMAGE: fedora
- MAKE_CHECK_ARGS: check-avocado check-functional
+ MAKE_CHECK_ARGS: check-functional
build-cfi-ppc64-s390x:
extends:
@@ -558,7 +549,7 @@ functional-cfi-ppc64-s390x:
artifacts: true
variables:
IMAGE: fedora
- MAKE_CHECK_ARGS: check-avocado check-functional
+ MAKE_CHECK_ARGS: check-functional
build-cfi-x86_64:
extends:
@@ -592,7 +583,7 @@ functional-cfi-x86_64:
artifacts: true
variables:
IMAGE: fedora
- MAKE_CHECK_ARGS: check-avocado check-functional
+ MAKE_CHECK_ARGS: check-functional
tsan-build:
extends: .native_build_job_template
--
2.49.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* Re: [PATCH 01/15] gitlab-ci: Remove the avocado tests from the CI pipelines
2025-03-25 20:00 ` [PATCH 01/15] gitlab-ci: Remove the avocado tests from the CI pipelines Thomas Huth
@ 2025-03-26 9:52 ` Daniel P. Berrangé
0 siblings, 0 replies; 34+ messages in thread
From: Daniel P. Berrangé @ 2025-03-26 9:52 UTC (permalink / raw)
To: Thomas Huth; +Cc: qemu-devel, Philippe Mathieu-Daudé
On Tue, Mar 25, 2025 at 09:00:09PM +0100, Thomas Huth wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> We are going to move the remaining Avocado tests step by step
> into the functional test framework. Unfortunately, Avocado fails
> with an error if it cannot determine a test to run, so disable
> the tests here now to avoid failures in the Gitlab-CI during the
> next steps.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> .gitlab-ci.d/buildtest-template.yml | 11 ----------
> .gitlab-ci.d/buildtest.yml | 33 +++++++++++------------------
> 2 files changed, 12 insertions(+), 32 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH 02/15] tests/functional: Move the check for the parameters from avocado to functional
2025-03-25 20:00 [PATCH for-10.1 00/15] Convert remaining Avocado tests to functional Thomas Huth
2025-03-25 20:00 ` [PATCH 01/15] gitlab-ci: Remove the avocado tests from the CI pipelines Thomas Huth
@ 2025-03-25 20:00 ` Thomas Huth
2025-03-26 9:27 ` Daniel P. Berrangé
2025-03-25 20:00 ` [PATCH 03/15] tests/functional: Convert reverse_debugging tests to the functional framework Thomas Huth
` (12 subsequent siblings)
14 siblings, 1 reply; 34+ messages in thread
From: Thomas Huth @ 2025-03-25 20:00 UTC (permalink / raw)
To: qemu-devel; +Cc: Daniel P. Berrangé, Philippe Mathieu-Daudé
From: Thomas Huth <thuth@redhat.com>
test_x86_64_pc in tests/avocado/boot_linux_console.py only checks
whether the kernel parameters have correctly been passed to the
kernel in the guest by looking for them in the console output of the
guest. Let's move that to the functional test framework now, but
instead of doing it in a separate test, let's do it for all tuxrun
tests instead, so it is done automatically for all targets that have
a tuxrun test.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/avocado/boot_linux_console.py | 34 ------------------------
tests/functional/qemu_test/tuxruntest.py | 9 ++++---
2 files changed, 5 insertions(+), 38 deletions(-)
diff --git a/tests/avocado/boot_linux_console.py b/tests/avocado/boot_linux_console.py
index c15f39ae1f3..cbb1e2fb506 100644
--- a/tests/avocado/boot_linux_console.py
+++ b/tests/avocado/boot_linux_console.py
@@ -9,17 +9,9 @@
# later. See the COPYING file in the top-level directory.
import os
-import lzma
-import gzip
import shutil
-from avocado import skip
-from avocado import skipUnless
-from avocado import skipUnless
from avocado_qemu import QemuSystemTest
-from avocado_qemu import exec_command
-from avocado_qemu import exec_command_and_wait_for_pattern
-from avocado_qemu import interrupt_interactive_console_until_pattern
from avocado_qemu import wait_for_console_pattern
from avocado.utils import process
from avocado.utils import archive
@@ -68,29 +60,3 @@ def extract_from_rpm(self, rpm, path):
process.run("rpm2cpio %s | cpio -id %s" % (rpm, path), shell=True)
os.chdir(cwd)
return os.path.normpath(os.path.join(self.workdir, path))
-
-class BootLinuxConsole(LinuxKernelTest):
- """
- Boots a Linux kernel and checks that the console is operational and the
- kernel command line is properly passed from QEMU to the kernel
- """
- timeout = 90
-
- def test_x86_64_pc(self):
- """
- :avocado: tags=arch:x86_64
- :avocado: tags=machine:pc
- """
- kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
- '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
- '/vmlinuz')
- kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
- kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
-
- self.vm.set_console()
- kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
- self.vm.add_args('-kernel', kernel_path,
- '-append', kernel_command_line)
- self.vm.launch()
- console_pattern = 'Kernel command line: %s' % kernel_command_line
- self.wait_for_console_pattern(console_pattern)
diff --git a/tests/functional/qemu_test/tuxruntest.py b/tests/functional/qemu_test/tuxruntest.py
index ad74156f9c5..c2bd5baaae9 100644
--- a/tests/functional/qemu_test/tuxruntest.py
+++ b/tests/functional/qemu_test/tuxruntest.py
@@ -77,12 +77,12 @@ def prepare_run(self, kernel, disk, drive, dtb=None, console_index=0):
blockdev = "driver=raw,file.driver=file," \
+ f"file.filename={disk},node-name=hd0"
- kcmd_line = self.KERNEL_COMMON_COMMAND_LINE
- kcmd_line += f" root=/dev/{self.root}"
- kcmd_line += f" console={self.console}"
+ self.kcmd_line = self.KERNEL_COMMON_COMMAND_LINE
+ self.kcmd_line += f" root=/dev/{self.root}"
+ self.kcmd_line += f" console={self.console}"
self.vm.add_args('-kernel', kernel,
- '-append', kcmd_line,
+ '-append', self.kcmd_line,
'-blockdev', blockdev)
# Sometimes we need extra devices attached
@@ -103,6 +103,7 @@ def run_tuxtest_tests(self, haltmsg):
wait to exit cleanly.
"""
ps1='root@tuxtest:~#'
+ self.wait_for_console_pattern(self.kcmd_line)
self.wait_for_console_pattern('tuxtest login:')
exec_command_and_wait_for_pattern(self, 'root', ps1)
exec_command_and_wait_for_pattern(self, 'cat /proc/interrupts', ps1)
--
2.49.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* Re: [PATCH 02/15] tests/functional: Move the check for the parameters from avocado to functional
2025-03-25 20:00 ` [PATCH 02/15] tests/functional: Move the check for the parameters from avocado to functional Thomas Huth
@ 2025-03-26 9:27 ` Daniel P. Berrangé
0 siblings, 0 replies; 34+ messages in thread
From: Daniel P. Berrangé @ 2025-03-26 9:27 UTC (permalink / raw)
To: Thomas Huth; +Cc: qemu-devel, Philippe Mathieu-Daudé
On Tue, Mar 25, 2025 at 09:00:10PM +0100, Thomas Huth wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> test_x86_64_pc in tests/avocado/boot_linux_console.py only checks
> whether the kernel parameters have correctly been passed to the
> kernel in the guest by looking for them in the console output of the
> guest. Let's move that to the functional test framework now, but
> instead of doing it in a separate test, let's do it for all tuxrun
> tests instead, so it is done automatically for all targets that have
> a tuxrun test.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/avocado/boot_linux_console.py | 34 ------------------------
> tests/functional/qemu_test/tuxruntest.py | 9 ++++---
> 2 files changed, 5 insertions(+), 38 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH 03/15] tests/functional: Convert reverse_debugging tests to the functional framework
2025-03-25 20:00 [PATCH for-10.1 00/15] Convert remaining Avocado tests to functional Thomas Huth
2025-03-25 20:00 ` [PATCH 01/15] gitlab-ci: Remove the avocado tests from the CI pipelines Thomas Huth
2025-03-25 20:00 ` [PATCH 02/15] tests/functional: Move the check for the parameters from avocado to functional Thomas Huth
@ 2025-03-25 20:00 ` Thomas Huth
2025-03-26 9:30 ` Daniel P. Berrangé
2025-03-25 20:00 ` [PATCH 04/15] tests/functional: Convert the i386 replay avocado test Thomas Huth
` (11 subsequent siblings)
14 siblings, 1 reply; 34+ messages in thread
From: Thomas Huth @ 2025-03-25 20:00 UTC (permalink / raw)
To: qemu-devel; +Cc: Daniel P. Berrangé, Philippe Mathieu-Daudé
From: Thomas Huth <thuth@redhat.com>
These tests are using the gdb-related library functions from the
Avocado framework which we don't have in the functional framework
yet. So for the time being, keep those imports and skip the test
if the Avocado framework is not installed on the host.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 2 +-
tests/functional/meson.build | 4 +
.../reverse_debugging.py | 114 +++---------------
.../functional/test_aarch64_reverse_debug.py | 37 ++++++
tests/functional/test_ppc64_reverse_debug.py | 41 +++++++
tests/functional/test_x86_64_reverse_debug.py | 36 ++++++
6 files changed, 138 insertions(+), 96 deletions(-)
rename tests/{avocado => functional}/reverse_debugging.py (66%)
create mode 100755 tests/functional/test_aarch64_reverse_debug.py
create mode 100755 tests/functional/test_ppc64_reverse_debug.py
create mode 100755 tests/functional/test_x86_64_reverse_debug.py
diff --git a/MAINTAINERS b/MAINTAINERS
index 8f470a1c9b7..73ccf5e5176 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3670,7 +3670,7 @@ F: docs/system/replay.rst
F: stubs/replay.c
F: tests/avocado/replay_kernel.py
F: tests/avocado/replay_linux.py
-F: tests/avocado/reverse_debugging.py
+F: tests/functional/*reverse_debug*.py
F: tests/functional/*replay*.py
F: qapi/replay.json
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 96d28289279..bcb4f50a62f 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -13,6 +13,7 @@ endif
test_timeouts = {
'aarch64_aspeed' : 600,
'aarch64_raspi4' : 480,
+ 'aarch64_reverse_debug' : 180,
'aarch64_rme_virt' : 1200,
'aarch64_rme_sbsaref' : 1200,
'aarch64_sbsaref_alpine' : 1200,
@@ -78,6 +79,7 @@ tests_aarch64_system_thorough = [
'aarch64_raspi3',
'aarch64_raspi4',
'aarch64_replay',
+ 'aarch64_reverse_debug',
'aarch64_rme_virt',
'aarch64_rme_sbsaref',
'aarch64_sbsaref',
@@ -229,6 +231,7 @@ tests_ppc64_system_thorough = [
'ppc64_powernv',
'ppc64_pseries',
'ppc64_replay',
+ 'ppc64_reverse_debug',
'ppc64_tuxrun',
'ppc64_mac99',
]
@@ -311,6 +314,7 @@ tests_x86_64_system_thorough = [
'x86_64_hotplug_cpu',
'x86_64_kvm_xen',
'x86_64_replay',
+ 'x86_64_reverse_debug',
'x86_64_tuxrun',
]
diff --git a/tests/avocado/reverse_debugging.py b/tests/functional/reverse_debugging.py
similarity index 66%
rename from tests/avocado/reverse_debugging.py
rename to tests/functional/reverse_debugging.py
index f24287cd0a0..f9a1d395f1d 100644
--- a/tests/avocado/reverse_debugging.py
+++ b/tests/functional/reverse_debugging.py
@@ -1,5 +1,7 @@
# Reverse debugging test
#
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
# Copyright (c) 2020 ISP RAS
#
# Author:
@@ -10,14 +12,9 @@
import os
import logging
-from avocado import skipUnless
-from avocado_qemu import BUILD_DIR
-from avocado.utils import datadrainer
-from avocado.utils import gdb
-from avocado.utils import process
-from avocado.utils.network.ports import find_free_port
-from avocado.utils.path import find_command
-from boot_linux_console import LinuxKernelTest
+from qemu_test import LinuxKernelTest, get_qemu_img
+from qemu_test.ports import Ports
+
class ReverseDebugging(LinuxKernelTest):
"""
@@ -36,8 +33,10 @@ class ReverseDebugging(LinuxKernelTest):
endian_is_le = True
def run_vm(self, record, shift, args, replay_path, image_path, port):
+ from avocado.utils import datadrainer
+
logger = logging.getLogger('replay')
- vm = self.get_vm()
+ vm = self.get_vm(name='record' if record else 'replay')
vm.set_console()
if record:
logger.info('recording the execution...')
@@ -100,25 +99,25 @@ def vm_get_icount(vm):
return vm.qmp('query-replay')['return']['icount']
def reverse_debugging(self, shift=7, args=None):
+ from avocado.utils import gdb
+ from avocado.utils import process
+
logger = logging.getLogger('replay')
# create qcow2 for snapshots
logger.info('creating qcow2 image for VM snapshots')
image_path = os.path.join(self.workdir, 'disk.qcow2')
- qemu_img = os.path.join(BUILD_DIR, 'qemu-img')
- if not os.path.exists(qemu_img):
- qemu_img = find_command('qemu-img', False)
- if qemu_img is False:
- self.cancel('Could not find "qemu-img", which is required to '
- 'create the temporary qcow2 image')
+ qemu_img = get_qemu_img(self)
+ if qemu_img is None:
+ self.skipTest('Could not find "qemu-img", which is required to '
+ 'create the temporary qcow2 image')
cmd = '%s create -f qcow2 %s 128M' % (qemu_img, image_path)
process.run(cmd)
replay_path = os.path.join(self.workdir, 'replay.bin')
- port = find_free_port()
# record the log
- vm = self.run_vm(True, shift, args, replay_path, image_path, port)
+ vm = self.run_vm(True, shift, args, replay_path, image_path, -1)
while self.vm_get_icount(vm) <= self.STEPS:
pass
last_icount = self.vm_get_icount(vm)
@@ -127,7 +126,9 @@ def reverse_debugging(self, shift=7, args=None):
logger.info("recorded log with %s+ steps" % last_icount)
# replay and run debug commands
- vm = self.run_vm(False, shift, args, replay_path, image_path, port)
+ with Ports() as ports:
+ port = ports.find_free_port()
+ vm = self.run_vm(False, shift, args, replay_path, image_path, port)
logger.info('connecting to gdbstub')
g = gdb.GDBRemote('127.0.0.1', port, False, False)
g.connect()
@@ -193,80 +194,3 @@ def reverse_debugging(self, shift=7, args=None):
logger.info('exiting gdb and qemu')
vm.shutdown()
-
-class ReverseDebugging_X86_64(ReverseDebugging):
- """
- :avocado: tags=accel:tcg
- """
-
- REG_PC = 0x10
- REG_CS = 0x12
- def get_pc(self, g):
- return self.get_reg_le(g, self.REG_PC) \
- + self.get_reg_le(g, self.REG_CS) * 0x10
-
- # unidentified gitlab timeout problem
- @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab')
- def test_x86_64_pc(self):
- """
- :avocado: tags=arch:x86_64
- :avocado: tags=machine:pc
- """
- # start with BIOS only
- self.reverse_debugging()
-
-class ReverseDebugging_AArch64(ReverseDebugging):
- """
- :avocado: tags=accel:tcg
- """
-
- REG_PC = 32
-
- # unidentified gitlab timeout problem
- @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab')
- def test_aarch64_virt(self):
- """
- :avocado: tags=arch:aarch64
- :avocado: tags=machine:virt
- :avocado: tags=cpu:cortex-a53
- """
- kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
- '/linux/releases/29/Everything/aarch64/os/images/pxeboot'
- '/vmlinuz')
- kernel_hash = '8c73e469fc6ea06a58dc83a628fc695b693b8493'
- kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
-
- self.reverse_debugging(
- args=('-kernel', kernel_path))
-
-class ReverseDebugging_ppc64(ReverseDebugging):
- """
- :avocado: tags=accel:tcg
- """
-
- REG_PC = 0x40
-
- # unidentified gitlab timeout problem
- @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab')
- def test_ppc64_pseries(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:pseries
- :avocado: tags=flaky
- """
- # SLOF branches back to its entry point, which causes this test
- # to take the 'hit a breakpoint again' path. That's not a problem,
- # just slightly different than the other machines.
- self.endian_is_le = False
- self.reverse_debugging()
-
- # See https://gitlab.com/qemu-project/qemu/-/issues/1992
- @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab')
- def test_ppc64_powernv(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:powernv
- :avocado: tags=flaky
- """
- self.endian_is_le = False
- self.reverse_debugging()
diff --git a/tests/functional/test_aarch64_reverse_debug.py b/tests/functional/test_aarch64_reverse_debug.py
new file mode 100755
index 00000000000..82925bf5908
--- /dev/null
+++ b/tests/functional/test_aarch64_reverse_debug.py
@@ -0,0 +1,37 @@
+#!/usr/bin/env python3
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# Reverse debugging test
+#
+# Copyright (c) 2020 ISP RAS
+#
+# Author:
+# Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+from qemu_test import Asset, skipIfMissingImports
+from reverse_debugging import ReverseDebugging
+
+
+@skipIfMissingImports('avocado')
+class ReverseDebugging_AArch64(ReverseDebugging):
+
+ REG_PC = 32
+
+ KERNEL_ASSET = Asset(
+ ('https://archives.fedoraproject.org/pub/archive/fedora/linux/'
+ 'releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz'),
+ '7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7')
+
+ def test_aarch64_virt(self):
+ self.set_machine('virt')
+ self.cpu = 'cortex-a53'
+ kernel_path = self.KERNEL_ASSET.fetch()
+ self.reverse_debugging(args=('-kernel', kernel_path))
+
+
+if __name__ == '__main__':
+ ReverseDebugging.main()
diff --git a/tests/functional/test_ppc64_reverse_debug.py b/tests/functional/test_ppc64_reverse_debug.py
new file mode 100755
index 00000000000..567b9422008
--- /dev/null
+++ b/tests/functional/test_ppc64_reverse_debug.py
@@ -0,0 +1,41 @@
+#!/usr/bin/env python3
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# Reverse debugging test
+#
+# Copyright (c) 2020 ISP RAS
+#
+# Author:
+# Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+from qemu_test import skipIfMissingImports, skipFlakyTest
+from reverse_debugging import ReverseDebugging
+
+
+@skipIfMissingImports('avocado')
+class ReverseDebugging_ppc64(ReverseDebugging):
+
+ REG_PC = 0x40
+
+ @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/1992")
+ def test_ppc64_pseries(self):
+ self.set_machine('pseries')
+ # SLOF branches back to its entry point, which causes this test
+ # to take the 'hit a breakpoint again' path. That's not a problem,
+ # just slightly different than the other machines.
+ self.endian_is_le = False
+ self.reverse_debugging()
+
+ @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/1992")
+ def test_ppc64_powernv(self):
+ self.set_machine('powernv')
+ self.endian_is_le = False
+ self.reverse_debugging()
+
+
+if __name__ == '__main__':
+ ReverseDebugging.main()
diff --git a/tests/functional/test_x86_64_reverse_debug.py b/tests/functional/test_x86_64_reverse_debug.py
new file mode 100755
index 00000000000..aba31f68748
--- /dev/null
+++ b/tests/functional/test_x86_64_reverse_debug.py
@@ -0,0 +1,36 @@
+#!/usr/bin/env python3
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# Reverse debugging test
+#
+# Copyright (c) 2020 ISP RAS
+#
+# Author:
+# Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+from qemu_test import skipIfMissingImports, skipFlakyTest
+from reverse_debugging import ReverseDebugging
+
+
+@skipIfMissingImports('avocado')
+class ReverseDebugging_X86_64(ReverseDebugging):
+
+ REG_PC = 0x10
+ REG_CS = 0x12
+ def get_pc(self, g):
+ return self.get_reg_le(g, self.REG_PC) \
+ + self.get_reg_le(g, self.REG_CS) * 0x10
+
+ @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/1992")
+ def test_x86_64_pc(self):
+ self.set_machine('pc')
+ # start with BIOS only
+ self.reverse_debugging()
+
+
+if __name__ == '__main__':
+ ReverseDebugging.main()
--
2.49.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* Re: [PATCH 03/15] tests/functional: Convert reverse_debugging tests to the functional framework
2025-03-25 20:00 ` [PATCH 03/15] tests/functional: Convert reverse_debugging tests to the functional framework Thomas Huth
@ 2025-03-26 9:30 ` Daniel P. Berrangé
2025-04-14 9:52 ` Thomas Huth
0 siblings, 1 reply; 34+ messages in thread
From: Daniel P. Berrangé @ 2025-03-26 9:30 UTC (permalink / raw)
To: Thomas Huth; +Cc: qemu-devel, Philippe Mathieu-Daudé
On Tue, Mar 25, 2025 at 09:00:11PM +0100, Thomas Huth wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> These tests are using the gdb-related library functions from the
> Avocado framework which we don't have in the functional framework
> yet. So for the time being, keep those imports and skip the test
> if the Avocado framework is not installed on the host.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> MAINTAINERS | 2 +-
> tests/functional/meson.build | 4 +
> .../reverse_debugging.py | 114 +++---------------
> .../functional/test_aarch64_reverse_debug.py | 37 ++++++
> tests/functional/test_ppc64_reverse_debug.py | 41 +++++++
> tests/functional/test_x86_64_reverse_debug.py | 36 ++++++
> 6 files changed, 138 insertions(+), 96 deletions(-)
> rename tests/{avocado => functional}/reverse_debugging.py (66%)
> create mode 100755 tests/functional/test_aarch64_reverse_debug.py
> create mode 100755 tests/functional/test_ppc64_reverse_debug.py
> create mode 100755 tests/functional/test_x86_64_reverse_debug.py
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 8f470a1c9b7..73ccf5e5176 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -3670,7 +3670,7 @@ F: docs/system/replay.rst
> F: stubs/replay.c
> F: tests/avocado/replay_kernel.py
> F: tests/avocado/replay_linux.py
> -F: tests/avocado/reverse_debugging.py
> +F: tests/functional/*reverse_debug*.py
> F: tests/functional/*replay*.py
> F: qapi/replay.json
>
> diff --git a/tests/functional/test_aarch64_reverse_debug.py b/tests/functional/test_aarch64_reverse_debug.py
> new file mode 100755
> index 00000000000..82925bf5908
> --- /dev/null
> +++ b/tests/functional/test_aarch64_reverse_debug.py
> @@ -0,0 +1,37 @@
> +#!/usr/bin/env python3
> +#
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +#
> +# Reverse debugging test
> +#
> +# Copyright (c) 2020 ISP RAS
> +#
> +# Author:
> +# Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or
> +# later. See the COPYING file in the top-level directory.
> +
> +from qemu_test import Asset, skipIfMissingImports
> +from reverse_debugging import ReverseDebugging
> +
> +
> +@skipIfMissingImports('avocado')
Would it make sense to specialize this to 'avocado.utils' ?
> +class ReverseDebugging_AArch64(ReverseDebugging):
> +
> + REG_PC = 32
> +
> + KERNEL_ASSET = Asset(
> + ('https://archives.fedoraproject.org/pub/archive/fedora/linux/'
> + 'releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz'),
> + '7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7')
> +
> + def test_aarch64_virt(self):
> + self.set_machine('virt')
> + self.cpu = 'cortex-a53'
> + kernel_path = self.KERNEL_ASSET.fetch()
> + self.reverse_debugging(args=('-kernel', kernel_path))
> +
> +
> +if __name__ == '__main__':
> + ReverseDebugging.main()
> diff --git a/tests/functional/test_x86_64_reverse_debug.py b/tests/functional/test_x86_64_reverse_debug.py
> new file mode 100755
> index 00000000000..aba31f68748
> --- /dev/null
> +++ b/tests/functional/test_x86_64_reverse_debug.py
> @@ -0,0 +1,36 @@
> +#!/usr/bin/env python3
> +#
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +#
> +# Reverse debugging test
> +#
> +# Copyright (c) 2020 ISP RAS
> +#
> +# Author:
> +# Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or
> +# later. See the COPYING file in the top-level directory.
> +
> +from qemu_test import skipIfMissingImports, skipFlakyTest
> +from reverse_debugging import ReverseDebugging
> +
> +
> +@skipIfMissingImports('avocado')
> +class ReverseDebugging_X86_64(ReverseDebugging):
> +
> + REG_PC = 0x10
> + REG_CS = 0x12
> + def get_pc(self, g):
> + return self.get_reg_le(g, self.REG_PC) \
> + + self.get_reg_le(g, self.REG_CS) * 0x10
> +
> + @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/1992")
This bug refers to ppc64. Was this a copy and paste mistake, does
that bug need updating to also mention x86_64 ?
> + def test_x86_64_pc(self):
> + self.set_machine('pc')
> + # start with BIOS only
> + self.reverse_debugging()
> +
> +
> +if __name__ == '__main__':
> + ReverseDebugging.main()
> --
> 2.49.0
>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 34+ messages in thread* Re: [PATCH 03/15] tests/functional: Convert reverse_debugging tests to the functional framework
2025-03-26 9:30 ` Daniel P. Berrangé
@ 2025-04-14 9:52 ` Thomas Huth
0 siblings, 0 replies; 34+ messages in thread
From: Thomas Huth @ 2025-04-14 9:52 UTC (permalink / raw)
To: Daniel P. Berrangé; +Cc: qemu-devel, Philippe Mathieu-Daudé
On 26/03/2025 10.30, Daniel P. Berrangé wrote:
> On Tue, Mar 25, 2025 at 09:00:11PM +0100, Thomas Huth wrote:
>> From: Thomas Huth <thuth@redhat.com>
>>
>> These tests are using the gdb-related library functions from the
>> Avocado framework which we don't have in the functional framework
>> yet. So for the time being, keep those imports and skip the test
>> if the Avocado framework is not installed on the host.
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>> MAINTAINERS | 2 +-
>> tests/functional/meson.build | 4 +
>> .../reverse_debugging.py | 114 +++---------------
>> .../functional/test_aarch64_reverse_debug.py | 37 ++++++
>> tests/functional/test_ppc64_reverse_debug.py | 41 +++++++
>> tests/functional/test_x86_64_reverse_debug.py | 36 ++++++
>> 6 files changed, 138 insertions(+), 96 deletions(-)
>> rename tests/{avocado => functional}/reverse_debugging.py (66%)
>> create mode 100755 tests/functional/test_aarch64_reverse_debug.py
>> create mode 100755 tests/functional/test_ppc64_reverse_debug.py
>> create mode 100755 tests/functional/test_x86_64_reverse_debug.py
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index 8f470a1c9b7..73ccf5e5176 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -3670,7 +3670,7 @@ F: docs/system/replay.rst
>> F: stubs/replay.c
>> F: tests/avocado/replay_kernel.py
>> F: tests/avocado/replay_linux.py
>> -F: tests/avocado/reverse_debugging.py
>> +F: tests/functional/*reverse_debug*.py
>> F: tests/functional/*replay*.py
>> F: qapi/replay.json
>>
>
>> diff --git a/tests/functional/test_aarch64_reverse_debug.py b/tests/functional/test_aarch64_reverse_debug.py
>> new file mode 100755
>> index 00000000000..82925bf5908
>> --- /dev/null
>> +++ b/tests/functional/test_aarch64_reverse_debug.py
>> @@ -0,0 +1,37 @@
>> +#!/usr/bin/env python3
>> +#
>> +# SPDX-License-Identifier: GPL-2.0-or-later
>> +#
>> +# Reverse debugging test
>> +#
>> +# Copyright (c) 2020 ISP RAS
>> +#
>> +# Author:
>> +# Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2 or
>> +# later. See the COPYING file in the top-level directory.
>> +
>> +from qemu_test import Asset, skipIfMissingImports
>> +from reverse_debugging import ReverseDebugging
>> +
>> +
>> +@skipIfMissingImports('avocado')
>
> Would it make sense to specialize this to 'avocado.utils' ?
Sure, can do!
>> +class ReverseDebugging_AArch64(ReverseDebugging):
>> +
>> + REG_PC = 32
>> +
>> + KERNEL_ASSET = Asset(
>> + ('https://archives.fedoraproject.org/pub/archive/fedora/linux/'
>> + 'releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz'),
>> + '7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7')
>> +
>> + def test_aarch64_virt(self):
>> + self.set_machine('virt')
>> + self.cpu = 'cortex-a53'
>> + kernel_path = self.KERNEL_ASSET.fetch()
>> + self.reverse_debugging(args=('-kernel', kernel_path))
>> +
>> +
>> +if __name__ == '__main__':
>> + ReverseDebugging.main()
>
>> diff --git a/tests/functional/test_x86_64_reverse_debug.py b/tests/functional/test_x86_64_reverse_debug.py
>> new file mode 100755
>> index 00000000000..aba31f68748
>> --- /dev/null
>> +++ b/tests/functional/test_x86_64_reverse_debug.py
>> @@ -0,0 +1,36 @@
>> +#!/usr/bin/env python3
>> +#
>> +# SPDX-License-Identifier: GPL-2.0-or-later
>> +#
>> +# Reverse debugging test
>> +#
>> +# Copyright (c) 2020 ISP RAS
>> +#
>> +# Author:
>> +# Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2 or
>> +# later. See the COPYING file in the top-level directory.
>> +
>> +from qemu_test import skipIfMissingImports, skipFlakyTest
>> +from reverse_debugging import ReverseDebugging
>> +
>> +
>> +@skipIfMissingImports('avocado')
>> +class ReverseDebugging_X86_64(ReverseDebugging):
>> +
>> + REG_PC = 0x10
>> + REG_CS = 0x12
>> + def get_pc(self, g):
>> + return self.get_reg_le(g, self.REG_PC) \
>> + + self.get_reg_le(g, self.REG_CS) * 0x10
>> +
>> + @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/1992")
>
> This bug refers to ppc64. Was this a copy and paste mistake, does
> that bug need updating to also mention x86_64 ?
I'm not sure whether it's the same bug, so I opened another one for x86 here:
https://gitlab.com/qemu-project/qemu/-/issues/2922
I also noticed that it fails on aarch64 in some rare cases, and in a
different way. So here's another bug for aarch64:
https://gitlab.com/qemu-project/qemu/-/issues/2921
I'll use those in the decorators in the next version of the patch.
Thomas
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH 04/15] tests/functional: Convert the i386 replay avocado test
2025-03-25 20:00 [PATCH for-10.1 00/15] Convert remaining Avocado tests to functional Thomas Huth
` (2 preceding siblings ...)
2025-03-25 20:00 ` [PATCH 03/15] tests/functional: Convert reverse_debugging tests to the functional framework Thomas Huth
@ 2025-03-25 20:00 ` Thomas Huth
2025-03-26 9:53 ` Daniel P. Berrangé
2025-03-25 20:00 ` [PATCH 05/15] tests/avocado: Remove the LinuxKernelTest class Thomas Huth
` (10 subsequent siblings)
14 siblings, 1 reply; 34+ messages in thread
From: Thomas Huth @ 2025-03-25 20:00 UTC (permalink / raw)
To: qemu-devel; +Cc: Daniel P. Berrangé, Philippe Mathieu-Daudé
From: Thomas Huth <thuth@redhat.com>
Since this was the last test in tests/avocado/replay_kernel.py,
we can remove that Avocado file now.
Signed-off-by: Thomas Huth <thuth@redhat.com
---
MAINTAINERS | 1 -
tests/avocado/replay_kernel.py | 110 ---------------------------
tests/functional/meson.build | 1 +
tests/functional/test_i386_replay.py | 28 +++++++
4 files changed, 29 insertions(+), 111 deletions(-)
delete mode 100644 tests/avocado/replay_kernel.py
create mode 100755 tests/functional/test_i386_replay.py
diff --git a/MAINTAINERS b/MAINTAINERS
index 73ccf5e5176..4e1d816f2b4 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3668,7 +3668,6 @@ F: include/system/replay.h
F: docs/devel/replay.rst
F: docs/system/replay.rst
F: stubs/replay.c
-F: tests/avocado/replay_kernel.py
F: tests/avocado/replay_linux.py
F: tests/functional/*reverse_debug*.py
F: tests/functional/*replay*.py
diff --git a/tests/avocado/replay_kernel.py b/tests/avocado/replay_kernel.py
deleted file mode 100644
index 35515323723..00000000000
--- a/tests/avocado/replay_kernel.py
+++ /dev/null
@@ -1,110 +0,0 @@
-# Record/replay test that boots a Linux kernel
-#
-# Copyright (c) 2020 ISP RAS
-#
-# Author:
-# Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or
-# later. See the COPYING file in the top-level directory.
-
-import os
-import lzma
-import shutil
-import logging
-import time
-import subprocess
-
-from avocado import skip
-from avocado import skipUnless
-from avocado import skipUnless
-from avocado_qemu import wait_for_console_pattern
-from avocado.utils import archive
-from avocado.utils import process
-from boot_linux_console import LinuxKernelTest
-
-class ReplayKernelBase(LinuxKernelTest):
- """
- Boots a Linux kernel in record mode and checks that the console
- is operational and the kernel command line is properly passed
- from QEMU to the kernel.
- Then replays the same scenario and verifies, that QEMU correctly
- terminates.
- """
-
- timeout = 180
- KERNEL_COMMON_COMMAND_LINE = 'printk.time=1 panic=-1 '
-
- def run_vm(self, kernel_path, kernel_command_line, console_pattern,
- record, shift, args, replay_path):
- # icount requires TCG to be available
- self.require_accelerator('tcg')
-
- logger = logging.getLogger('replay')
- start_time = time.time()
- vm = self.get_vm()
- vm.set_console()
- if record:
- logger.info('recording the execution...')
- mode = 'record'
- else:
- logger.info('replaying the execution...')
- mode = 'replay'
- vm.add_args('-icount', 'shift=%s,rr=%s,rrfile=%s' %
- (shift, mode, replay_path),
- '-kernel', kernel_path,
- '-append', kernel_command_line,
- '-net', 'none',
- '-no-reboot')
- if args:
- vm.add_args(*args)
- vm.launch()
- self.wait_for_console_pattern(console_pattern, vm)
- if record:
- vm.shutdown()
- logger.info('finished the recording with log size %s bytes'
- % os.path.getsize(replay_path))
- self.run_replay_dump(replay_path)
- logger.info('successfully tested replay-dump.py')
- else:
- vm.wait()
- logger.info('successfully finished the replay')
- elapsed = time.time() - start_time
- logger.info('elapsed time %.2f sec' % elapsed)
- return elapsed
-
- def run_replay_dump(self, replay_path):
- try:
- subprocess.check_call(["./scripts/replay-dump.py",
- "-f", replay_path],
- stdout=subprocess.DEVNULL)
- except subprocess.CalledProcessError:
- self.fail('replay-dump.py failed')
-
- def run_rr(self, kernel_path, kernel_command_line, console_pattern,
- shift=7, args=None):
- replay_path = os.path.join(self.workdir, 'replay.bin')
- t1 = self.run_vm(kernel_path, kernel_command_line, console_pattern,
- True, shift, args, replay_path)
- t2 = self.run_vm(kernel_path, kernel_command_line, console_pattern,
- False, shift, args, replay_path)
- logger = logging.getLogger('replay')
- logger.info('replay overhead {:.2%}'.format(t2 / t1 - 1))
-
-class ReplayKernelNormal(ReplayKernelBase):
-
- def test_i386_pc(self):
- """
- :avocado: tags=arch:i386
- :avocado: tags=machine:pc
- """
- kernel_url = ('https://storage.tuxboot.com/20230331/i386/bzImage')
- kernel_hash = 'a3e5b32a354729e65910f5a1ffcda7c14a6c12a55e8213fb86e277f1b76ed956'
- kernel_path = self.fetch_asset(kernel_url,
- asset_hash=kernel_hash,
- algorithm = "sha256")
-
- kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
- console_pattern = 'VFS: Cannot open root device'
-
- self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5)
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index bcb4f50a62f..452d863f187 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -151,6 +151,7 @@ tests_i386_system_quick = [
]
tests_i386_system_thorough = [
+ 'i386_replay',
'i386_tuxrun',
]
diff --git a/tests/functional/test_i386_replay.py b/tests/functional/test_i386_replay.py
new file mode 100755
index 00000000000..7c4c2602da9
--- /dev/null
+++ b/tests/functional/test_i386_replay.py
@@ -0,0 +1,28 @@
+#!/usr/bin/env python3
+#
+# Replay test that boots a Linux kernel on a i386 machine
+# and checks the console
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+from qemu_test import Asset
+from replay_kernel import ReplayKernelBase
+
+
+class I386Replay(ReplayKernelBase):
+
+ ASSET_KERNEL = Asset(
+ 'https://storage.tuxboot.com/20230331/i386/bzImage',
+ 'a3e5b32a354729e65910f5a1ffcda7c14a6c12a55e8213fb86e277f1b76ed956')
+
+ def test_pc(self):
+ self.set_machine('pc')
+ kernel_url = ()
+ kernel_path = self.ASSET_KERNEL.fetch()
+ kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
+ console_pattern = 'VFS: Cannot open root device'
+ self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5)
+
+
+if __name__ == '__main__':
+ ReplayKernelBase.main()
--
2.49.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 05/15] tests/avocado: Remove the LinuxKernelTest class
2025-03-25 20:00 [PATCH for-10.1 00/15] Convert remaining Avocado tests to functional Thomas Huth
` (3 preceding siblings ...)
2025-03-25 20:00 ` [PATCH 04/15] tests/functional: Convert the i386 replay avocado test Thomas Huth
@ 2025-03-25 20:00 ` Thomas Huth
2025-03-26 9:40 ` Daniel P. Berrangé
2025-03-25 20:00 ` [PATCH 06/15] tests/functional: Convert the 32-bit big endian Wheezy mips test Thomas Huth
` (9 subsequent siblings)
14 siblings, 1 reply; 34+ messages in thread
From: Thomas Huth @ 2025-03-25 20:00 UTC (permalink / raw)
To: qemu-devel; +Cc: Daniel P. Berrangé, Philippe Mathieu-Daudé
From: Thomas Huth <thuth@redhat.com>
All tests that used this class have been converted to the functional
framework, so we can remove the boot_linux_console.py file now.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/avocado/boot_linux_console.py | 62 -----------------------------
1 file changed, 62 deletions(-)
delete mode 100644 tests/avocado/boot_linux_console.py
diff --git a/tests/avocado/boot_linux_console.py b/tests/avocado/boot_linux_console.py
deleted file mode 100644
index cbb1e2fb506..00000000000
--- a/tests/avocado/boot_linux_console.py
+++ /dev/null
@@ -1,62 +0,0 @@
-# Functional test that boots a Linux kernel and checks the console
-#
-# Copyright (c) 2018 Red Hat, Inc.
-#
-# Author:
-# Cleber Rosa <crosa@redhat.com>
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or
-# later. See the COPYING file in the top-level directory.
-
-import os
-import shutil
-
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
-from avocado.utils import process
-from avocado.utils import archive
-
-class LinuxKernelTest(QemuSystemTest):
- KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
-
- def wait_for_console_pattern(self, success_message, vm=None):
- wait_for_console_pattern(self, success_message,
- failure_message='Kernel panic - not syncing',
- vm=vm)
-
- def extract_from_deb(self, deb, path):
- """
- Extracts a file from a deb package into the test workdir
-
- :param deb: path to the deb archive
- :param path: path within the deb archive of the file to be extracted
- :returns: path of the extracted file
- """
- cwd = os.getcwd()
- os.chdir(self.workdir)
- file_path = process.run("ar t %s" % deb).stdout_text.split()[2]
- process.run("ar x %s %s" % (deb, file_path))
- archive.extract(file_path, self.workdir)
- os.chdir(cwd)
- # Return complete path to extracted file. Because callers to
- # extract_from_deb() specify 'path' with a leading slash, it is
- # necessary to use os.path.relpath() as otherwise os.path.join()
- # interprets it as an absolute path and drops the self.workdir part.
- return os.path.normpath(os.path.join(self.workdir,
- os.path.relpath(path, '/')))
-
- def extract_from_rpm(self, rpm, path):
- """
- Extracts a file from an RPM package into the test workdir.
-
- :param rpm: path to the rpm archive
- :param path: path within the rpm archive of the file to be extracted
- needs to be a relative path (starting with './') because
- cpio(1), which is used to extract the file, expects that.
- :returns: path of the extracted file
- """
- cwd = os.getcwd()
- os.chdir(self.workdir)
- process.run("rpm2cpio %s | cpio -id %s" % (rpm, path), shell=True)
- os.chdir(cwd)
- return os.path.normpath(os.path.join(self.workdir, path))
--
2.49.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 06/15] tests/functional: Convert the 32-bit big endian Wheezy mips test
2025-03-25 20:00 [PATCH for-10.1 00/15] Convert remaining Avocado tests to functional Thomas Huth
` (4 preceding siblings ...)
2025-03-25 20:00 ` [PATCH 05/15] tests/avocado: Remove the LinuxKernelTest class Thomas Huth
@ 2025-03-25 20:00 ` Thomas Huth
2025-03-26 9:54 ` Daniel P. Berrangé
2025-03-25 20:00 ` [PATCH 07/15] tests/functional: Convert the 32-bit little " Thomas Huth
` (8 subsequent siblings)
14 siblings, 1 reply; 34+ messages in thread
From: Thomas Huth @ 2025-03-25 20:00 UTC (permalink / raw)
To: qemu-devel; +Cc: Daniel P. Berrangé, Philippe Mathieu-Daudé
From: Thomas Huth <thuth@redhat.com>
The test checks some entries in /proc and the output of some commands ...
we put these checks into exportable functions now so that they can
be reused more easily.
Additionally the linux_ssh_mips_malta.py uses SSH to test the networking
of the guest. Since we don't have a SSH module in the functional
framework yet, let's use the check_http_download() function here instead.
And while we're at it, also switch the NIC to e1000 now to get some more
test coverage, since the "pcnet" device is already tested in the test
test_mips_malta_cpio.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/avocado/linux_ssh_mips_malta.py | 8 --
tests/functional/meson.build | 2 +-
tests/functional/test_mips_malta.py | 108 +++++++++++++++++++++++++-
3 files changed, 107 insertions(+), 11 deletions(-)
diff --git a/tests/avocado/linux_ssh_mips_malta.py b/tests/avocado/linux_ssh_mips_malta.py
index d9bb525ad9c..73d294ad5cf 100644
--- a/tests/avocado/linux_ssh_mips_malta.py
+++ b/tests/avocado/linux_ssh_mips_malta.py
@@ -172,14 +172,6 @@ def check_mips_malta(self, uname_m, endianess):
# Wait for VM to shut down gracefully
self.vm.wait()
- def test_mips_malta32eb_kernel3_2_0(self):
- """
- :avocado: tags=arch:mips
- :avocado: tags=endian:big
- :avocado: tags=device:pcnet32
- """
- self.check_mips_malta('mips', 'be')
-
def test_mips_malta32el_kernel3_2_0(self):
"""
:avocado: tags=arch:mipsel
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 452d863f187..3f8fabd218d 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -39,7 +39,7 @@ test_timeouts = {
'arm_tuxrun' : 240,
'arm_sx1' : 360,
'intel_iommu': 300,
- 'mips_malta' : 120,
+ 'mips_malta' : 480,
'mipsel_replay' : 480,
'mips64el_replay' : 180,
'netdev_ethtool' : 180,
diff --git a/tests/functional/test_mips_malta.py b/tests/functional/test_mips_malta.py
index 9697c7d63f8..89b9556f30d 100755
--- a/tests/functional/test_mips_malta.py
+++ b/tests/functional/test_mips_malta.py
@@ -6,10 +6,93 @@
#
# SPDX-License-Identifier: GPL-2.0-or-later
-from qemu_test import LinuxKernelTest, Asset
+import os
+
+from qemu_test import LinuxKernelTest, Asset, wait_for_console_pattern
from qemu_test import exec_command_and_wait_for_pattern
+def mips_run_common_commands(test, prompt='#'):
+ exec_command_and_wait_for_pattern(test,
+ 'uname -m',
+ 'mips')
+ exec_command_and_wait_for_pattern(test,
+ 'grep XT-PIC /proc/interrupts',
+ 'timer')
+ wait_for_console_pattern(test, prompt)
+ exec_command_and_wait_for_pattern(test,
+ 'grep XT-PIC /proc/interrupts',
+ 'serial')
+ wait_for_console_pattern(test, prompt)
+ exec_command_and_wait_for_pattern(test,
+ 'grep XT-PIC /proc/interrupts',
+ 'ata_piix')
+ wait_for_console_pattern(test, prompt)
+ exec_command_and_wait_for_pattern(test,
+ 'grep XT-PIC /proc/interrupts',
+ 'rtc')
+ wait_for_console_pattern(test, prompt)
+ exec_command_and_wait_for_pattern(test,
+ 'cat /proc/devices',
+ 'input')
+ wait_for_console_pattern(test, prompt)
+ exec_command_and_wait_for_pattern(test,
+ 'cat /proc/devices',
+ 'fb')
+ wait_for_console_pattern(test, prompt)
+ exec_command_and_wait_for_pattern(test,
+ 'cat /proc/ioports',
+ ' : serial')
+ wait_for_console_pattern(test, prompt)
+ exec_command_and_wait_for_pattern(test,
+ 'cat /proc/ioports',
+ ' : ata_piix')
+ wait_for_console_pattern(test, prompt)
+
+def mips_check_wheezy(test, kernel_path, image_path, kernel_command_line,
+ dl_file, hsum, nic='pcnet', cpuinfo='MIPS 24Kc'):
+ test.require_netdev('user')
+ test.require_device(nic)
+ test.set_machine('malta')
+
+ port=8080
+ test.vm.add_args('-kernel', kernel_path,
+ '-append', kernel_command_line,
+ '-drive', 'file=%s,snapshot=on' % image_path,
+ '-netdev', 'user,id=n1' +
+ ',tftp=' + os.path.basename(kernel_path) +
+ ',hostfwd=tcp:127.0.0.1:0-:%d' % port,
+ '-device', f'{nic},netdev=n1',
+ '-no-reboot')
+ test.vm.set_console()
+ test.vm.launch()
+
+ wait_for_console_pattern(test, 'login: ', 'Oops')
+ exec_command_and_wait_for_pattern(test, 'root', 'Password:')
+ exec_command_and_wait_for_pattern(test, 'root', ':~# ')
+ mips_run_common_commands(test)
+
+ exec_command_and_wait_for_pattern(test, 'cd /', '# ')
+ test.check_http_download(dl_file, hsum, port,
+ pythoncmd='python -m SimpleHTTPServer')
+
+ exec_command_and_wait_for_pattern(test, 'cat /proc/cpuinfo', cpuinfo)
+ exec_command_and_wait_for_pattern(test, 'cat /proc/devices', 'usb')
+ exec_command_and_wait_for_pattern(test, 'cat /proc/ioports',
+ ' : piix4_smbus')
+ # lspci for the host bridge does not work on big endian targets:
+ # https://gitlab.com/qemu-project/qemu/-/issues/2826
+ # exec_command_and_wait_for_pattern(test, 'lspci -d 11ab:4620',
+ # 'GT-64120')
+ exec_command_and_wait_for_pattern(test,
+ 'cat /sys/bus/i2c/devices/i2c-0/name',
+ 'SMBus PIIX4 adapter')
+ exec_command_and_wait_for_pattern(test, 'cat /proc/mtd', 'YAMON')
+ # Empty 'Board Config' (64KB)
+ exec_command_and_wait_for_pattern(test, 'md5sum /dev/mtd2ro',
+ '0dfbe8aa4c20b52e1b8bf3cb6cbdf193')
+
+
class MaltaMachineConsole(LinuxKernelTest):
ASSET_KERNEL_2_63_2 = Asset(
@@ -70,7 +153,8 @@ def test_mips_malta_cpio(self):
exec_command_and_wait_for_pattern(self, 'cat /proc/cpuinfo',
'BogoMIPS')
exec_command_and_wait_for_pattern(self, 'uname -a',
- 'Debian')
+ '4.5.0-2-4kc-malta #1 Debian')
+ mips_run_common_commands(self)
exec_command_and_wait_for_pattern(self, 'ip link set eth0 up',
'eth0: link up')
@@ -89,6 +173,26 @@ def test_mips_malta_cpio(self):
# Wait for VM to shut down gracefully
self.vm.wait()
+ ASSET_WHEEZY_KERNEL = Asset(
+ ('https://people.debian.org/~aurel32/qemu/mips/'
+ 'vmlinux-3.2.0-4-4kc-malta'),
+ '0377fcda31299213c10b8e5babe7260ef99188b3ae1aca6f56594abb71e7f67e')
+
+ ASSET_WHEEZY_DISK = Asset(
+ ('https://people.debian.org/~aurel32/qemu/mips/'
+ 'debian_wheezy_mips_standard.qcow2'),
+ 'de03599285b8382ad309309a6c4869f6c6c42a5cfc983342bab9ec0dfa7849a2')
+
+ def test_wheezy(self):
+ kernel_path = self.ASSET_WHEEZY_KERNEL.fetch()
+ image_path = self.ASSET_WHEEZY_DISK.fetch()
+ kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
+ + 'console=ttyS0 root=/dev/sda1')
+ mips_check_wheezy(self,
+ kernel_path, image_path, kernel_command_line, nic='e1000',
+ dl_file='/boot/initrd.img-3.2.0-4-4kc-malta',
+ hsum='ff0c0369143d9bbb9a6e6bc79322a2be535619df639e84103237f406e87493dc')
+
if __name__ == '__main__':
LinuxKernelTest.main()
--
2.49.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* Re: [PATCH 06/15] tests/functional: Convert the 32-bit big endian Wheezy mips test
2025-03-25 20:00 ` [PATCH 06/15] tests/functional: Convert the 32-bit big endian Wheezy mips test Thomas Huth
@ 2025-03-26 9:54 ` Daniel P. Berrangé
0 siblings, 0 replies; 34+ messages in thread
From: Daniel P. Berrangé @ 2025-03-26 9:54 UTC (permalink / raw)
To: Thomas Huth; +Cc: qemu-devel, Philippe Mathieu-Daudé
On Tue, Mar 25, 2025 at 09:00:14PM +0100, Thomas Huth wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> The test checks some entries in /proc and the output of some commands ...
> we put these checks into exportable functions now so that they can
> be reused more easily.
>
> Additionally the linux_ssh_mips_malta.py uses SSH to test the networking
> of the guest. Since we don't have a SSH module in the functional
> framework yet, let's use the check_http_download() function here instead.
>
> And while we're at it, also switch the NIC to e1000 now to get some more
> test coverage, since the "pcnet" device is already tested in the test
> test_mips_malta_cpio.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/avocado/linux_ssh_mips_malta.py | 8 --
> tests/functional/meson.build | 2 +-
> tests/functional/test_mips_malta.py | 108 +++++++++++++++++++++++++-
> 3 files changed, 107 insertions(+), 11 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH 07/15] tests/functional: Convert the 32-bit little endian Wheezy mips test
2025-03-25 20:00 [PATCH for-10.1 00/15] Convert remaining Avocado tests to functional Thomas Huth
` (5 preceding siblings ...)
2025-03-25 20:00 ` [PATCH 06/15] tests/functional: Convert the 32-bit big endian Wheezy mips test Thomas Huth
@ 2025-03-25 20:00 ` Thomas Huth
2025-03-26 9:55 ` Daniel P. Berrangé
2025-03-25 20:00 ` [PATCH 08/15] tests/functional: Convert the 64-bit " Thomas Huth
` (7 subsequent siblings)
14 siblings, 1 reply; 34+ messages in thread
From: Thomas Huth @ 2025-03-25 20:00 UTC (permalink / raw)
To: qemu-devel; +Cc: Daniel P. Berrangé, Philippe Mathieu-Daudé
From: Thomas Huth <thuth@redhat.com>
Reuse the test function from the big endian test to easily
convert the 32-bit little endian Wheezy mips test.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/avocado/linux_ssh_mips_malta.py | 8 --------
tests/functional/meson.build | 1 +
tests/functional/test_mipsel_malta.py | 22 ++++++++++++++++++++++
3 files changed, 23 insertions(+), 8 deletions(-)
diff --git a/tests/avocado/linux_ssh_mips_malta.py b/tests/avocado/linux_ssh_mips_malta.py
index 73d294ad5cf..c1300aec961 100644
--- a/tests/avocado/linux_ssh_mips_malta.py
+++ b/tests/avocado/linux_ssh_mips_malta.py
@@ -172,14 +172,6 @@ def check_mips_malta(self, uname_m, endianess):
# Wait for VM to shut down gracefully
self.vm.wait()
- def test_mips_malta32el_kernel3_2_0(self):
- """
- :avocado: tags=arch:mipsel
- :avocado: tags=endian:little
- :avocado: tags=device:pcnet32
- """
- self.check_mips_malta('mips', 'le')
-
def test_mips_malta64eb_kernel3_2_0(self):
"""
:avocado: tags=arch:mips64
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 3f8fabd218d..c7bd4eb4edf 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -40,6 +40,7 @@ test_timeouts = {
'arm_sx1' : 360,
'intel_iommu': 300,
'mips_malta' : 480,
+ 'mipsel_malta' : 420,
'mipsel_replay' : 480,
'mips64el_replay' : 180,
'netdev_ethtool' : 180,
diff --git a/tests/functional/test_mipsel_malta.py b/tests/functional/test_mipsel_malta.py
index fe9c3a172e6..9ee2884da8e 100755
--- a/tests/functional/test_mipsel_malta.py
+++ b/tests/functional/test_mipsel_malta.py
@@ -13,6 +13,8 @@
from qemu_test import interrupt_interactive_console_until_pattern
from qemu_test import wait_for_console_pattern
+from test_mips_malta import mips_check_wheezy
+
class MaltaMachineConsole(LinuxKernelTest):
@@ -57,6 +59,26 @@ def test_mips_malta32el_nanomips_16k_up(self):
def test_mips_malta32el_nanomips_64k_dbg(self):
self.do_test_mips_malta32el_nanomips(self.ASSET_KERNEL_64K)
+ ASSET_WHEEZY_KERNEL = Asset(
+ ('https://people.debian.org/~aurel32/qemu/mipsel/'
+ 'vmlinux-3.2.0-4-4kc-malta'),
+ 'dc8a3648305b0201ca7a5cd135fe2890067a65d93c38728022bb0e656ad2bf9a')
+
+ ASSET_WHEEZY_DISK = Asset(
+ ('https://people.debian.org/~aurel32/qemu/mipsel/'
+ 'debian_wheezy_mipsel_standard.qcow2'),
+ '454f09ae39f7e6461c84727b927100d2c7813841f2a0a5dce328114887ecf914')
+
+ def test_wheezy(self):
+ kernel_path = self.ASSET_WHEEZY_KERNEL.fetch()
+ image_path = self.ASSET_WHEEZY_DISK.fetch()
+ kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
+ + 'console=ttyS0 root=/dev/sda1')
+ mips_check_wheezy(self,
+ kernel_path, image_path, kernel_command_line,
+ dl_file='/boot/initrd.img-3.2.0-4-4kc-malta',
+ hsum='9fc9f250ed56a74e35e704ddfd5a1c5a5625adefc5c9da91f649288d3ca000f0')
+
class MaltaMachineYAMON(QemuSystemTest):
--
2.49.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 08/15] tests/functional: Convert the 64-bit little endian Wheezy mips test
2025-03-25 20:00 [PATCH for-10.1 00/15] Convert remaining Avocado tests to functional Thomas Huth
` (6 preceding siblings ...)
2025-03-25 20:00 ` [PATCH 07/15] tests/functional: Convert the 32-bit little " Thomas Huth
@ 2025-03-25 20:00 ` Thomas Huth
2025-03-26 9:56 ` Daniel P. Berrangé
2025-03-25 20:00 ` [PATCH 09/15] tests/functional: Convert the 64-bit big " Thomas Huth
` (6 subsequent siblings)
14 siblings, 1 reply; 34+ messages in thread
From: Thomas Huth @ 2025-03-25 20:00 UTC (permalink / raw)
To: qemu-devel; +Cc: Daniel P. Berrangé, Philippe Mathieu-Daudé
From: Thomas Huth <thuth@redhat.com>
Reuse the test function from the 32-bit big endian test to easily
convert the 64-bit little endian Wheezy mips test.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/avocado/linux_ssh_mips_malta.py | 8 --------
tests/functional/meson.build | 1 +
tests/functional/test_mips64el_malta.py | 22 ++++++++++++++++++++++
3 files changed, 23 insertions(+), 8 deletions(-)
diff --git a/tests/avocado/linux_ssh_mips_malta.py b/tests/avocado/linux_ssh_mips_malta.py
index c1300aec961..2fa5cf9a6c1 100644
--- a/tests/avocado/linux_ssh_mips_malta.py
+++ b/tests/avocado/linux_ssh_mips_malta.py
@@ -179,11 +179,3 @@ def test_mips_malta64eb_kernel3_2_0(self):
:avocado: tags=device:pcnet32
"""
self.check_mips_malta('mips64', 'be')
-
- def test_mips_malta64el_kernel3_2_0(self):
- """
- :avocado: tags=arch:mips64el
- :avocado: tags=endian:little
- :avocado: tags=device:pcnet32
- """
- self.check_mips_malta('mips64', 'le')
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index c7bd4eb4edf..63876bc7c42 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -42,6 +42,7 @@ test_timeouts = {
'mips_malta' : 480,
'mipsel_malta' : 420,
'mipsel_replay' : 480,
+ 'mips64el_malta' : 420,
'mips64el_replay' : 180,
'netdev_ethtool' : 180,
'ppc_40p' : 240,
diff --git a/tests/functional/test_mips64el_malta.py b/tests/functional/test_mips64el_malta.py
index a8da15a26b3..dd37212f9df 100755
--- a/tests/functional/test_mips64el_malta.py
+++ b/tests/functional/test_mips64el_malta.py
@@ -16,6 +16,8 @@
from qemu_test import exec_command_and_wait_for_pattern
from qemu_test import skipIfMissingImports, skipFlakyTest, skipUntrustedTest
+from test_mips_malta import mips_check_wheezy
+
class MaltaMachineConsole(LinuxKernelTest):
@@ -90,6 +92,26 @@ def test_mips64el_malta_5KEc_cpio(self):
# Wait for VM to shut down gracefully
self.vm.wait()
+ ASSET_WHEEZY_KERNEL = Asset(
+ ('https://people.debian.org/~aurel32/qemu/mipsel/'
+ 'vmlinux-3.2.0-4-5kc-malta'),
+ '5e8b725244c59745bb8b64f5d8f49f25fecfa549f3395fb6d19a3b9e5065b85b')
+
+ ASSET_WHEEZY_DISK = Asset(
+ ('https://people.debian.org/~aurel32/qemu/mipsel/'
+ 'debian_wheezy_mipsel_standard.qcow2'),
+ '454f09ae39f7e6461c84727b927100d2c7813841f2a0a5dce328114887ecf914')
+
+ def test_wheezy(self):
+ kernel_path = self.ASSET_WHEEZY_KERNEL.fetch()
+ image_path = self.ASSET_WHEEZY_DISK.fetch()
+ kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
+ + 'console=ttyS0 root=/dev/sda1')
+ mips_check_wheezy(self,
+ kernel_path, image_path, kernel_command_line, cpuinfo='MIPS 20Kc',
+ dl_file='/boot/initrd.img-3.2.0-4-5kc-malta',
+ hsum='7579f8b56c1187c7c04d0dc3c0c56c7a6314c5ddd3a9bf8803ecc7cf8a3be9f8')
+
@skipIfMissingImports('numpy', 'cv2')
class MaltaMachineFramebuffer(LinuxKernelTest):
--
2.49.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 09/15] tests/functional: Convert the 64-bit big endian Wheezy mips test
2025-03-25 20:00 [PATCH for-10.1 00/15] Convert remaining Avocado tests to functional Thomas Huth
` (7 preceding siblings ...)
2025-03-25 20:00 ` [PATCH 08/15] tests/functional: Convert the 64-bit " Thomas Huth
@ 2025-03-25 20:00 ` Thomas Huth
2025-03-26 10:00 ` Daniel P. Berrangé
2025-03-25 20:00 ` [PATCH 10/15] tests/avocado: Remove the boot_linux.py tests Thomas Huth
` (5 subsequent siblings)
14 siblings, 1 reply; 34+ messages in thread
From: Thomas Huth @ 2025-03-25 20:00 UTC (permalink / raw)
To: qemu-devel; +Cc: Daniel P. Berrangé, Philippe Mathieu-Daudé
From: Thomas Huth <thuth@redhat.com>
Reuse the test function from the 32-bit big endian test to easily
convert the 64-bit big endian Wheezy mips test.
Since this was the last test in tests/avocado/linux_ssh_mips_malta.py,
we can remove this avocado file now, too.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 1 -
tests/avocado/linux_ssh_mips_malta.py | 181 --------------------------
tests/functional/meson.build | 2 +
tests/functional/test_mips64_malta.py | 35 +++++
4 files changed, 37 insertions(+), 182 deletions(-)
delete mode 100644 tests/avocado/linux_ssh_mips_malta.py
create mode 100755 tests/functional/test_mips64_malta.py
diff --git a/MAINTAINERS b/MAINTAINERS
index 4e1d816f2b4..7b843c543ff 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1354,7 +1354,6 @@ F: hw/acpi/piix4.c
F: hw/mips/malta.c
F: hw/pci-host/gt64120.c
F: include/hw/southbridge/piix.h
-F: tests/avocado/linux_ssh_mips_malta.py
F: tests/functional/test_mips*_malta.py
F: tests/functional/test_mips*_tuxrun.py
diff --git a/tests/avocado/linux_ssh_mips_malta.py b/tests/avocado/linux_ssh_mips_malta.py
deleted file mode 100644
index 2fa5cf9a6c1..00000000000
--- a/tests/avocado/linux_ssh_mips_malta.py
+++ /dev/null
@@ -1,181 +0,0 @@
-# Functional test that boots a VM and run commands via a SSH session
-#
-# Copyright (c) Philippe Mathieu-Daudé <f4bug@amsat.org>
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or
-# later. See the COPYING file in the top-level directory.
-
-import os
-import re
-import base64
-import logging
-import time
-
-from avocado import skipUnless
-from avocado_qemu import LinuxSSHMixIn
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
-from avocado.utils import process
-from avocado.utils import archive
-from avocado.utils import ssh
-
-
-@skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
-@skipUnless(ssh.SSH_CLIENT_BINARY, 'No SSH client available')
-class LinuxSSH(QemuSystemTest, LinuxSSHMixIn):
- """
- :avocado: tags=accel:tcg
- """
-
- timeout = 150 # Not for 'configure --enable-debug --enable-debug-tcg'
-
- KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
- VM_IP = '127.0.0.1'
-
- BASE_URL = 'https://people.debian.org/~aurel32/qemu/'
- IMAGE_INFO = {
- 'be': {'base_url': 'mips',
- 'image_name': 'debian_wheezy_mips_standard.qcow2',
- 'image_hash': '8987a63270df67345b2135a6b7a4885a35e392d5',
- 'kernel_hash': {
- 32: '592e384a4edc16dade52a6cd5c785c637bcbc9ad',
- 64: 'db6eea7de35d36c77d8c165b6bcb222e16eb91db'}
- },
- 'le': {'base_url': 'mipsel',
- 'image_name': 'debian_wheezy_mipsel_standard.qcow2',
- 'image_hash': '7866764d9de3ef536ffca24c9fb9f04ffdb45802',
- 'kernel_hash': {
- 32: 'a66bea5a8adaa2cb3d36a1d4e0ccdb01be8f6c2a',
- 64: '6a7f77245acf231415a0e8b725d91ed2f3487794'}
- }
- }
- CPU_INFO = {
- 32: {'cpu': 'MIPS 24Kc', 'kernel_release': '3.2.0-4-4kc-malta'},
- 64: {'cpu': 'MIPS 20Kc', 'kernel_release': '3.2.0-4-5kc-malta'}
- }
-
- def get_url(self, endianess, path=''):
- qkey = {'le': 'el', 'be': ''}
- return '%s/mips%s/%s' % (self.BASE_URL, qkey[endianess], path)
-
- def get_image_info(self, endianess):
- dinfo = self.IMAGE_INFO[endianess]
- image_url = self.get_url(endianess, dinfo['image_name'])
- image_hash = dinfo['image_hash']
- return (image_url, image_hash)
-
- def get_kernel_info(self, endianess, wordsize):
- minfo = self.CPU_INFO[wordsize]
- kernel_url = self.get_url(endianess,
- 'vmlinux-%s' % minfo['kernel_release'])
- kernel_hash = self.IMAGE_INFO[endianess]['kernel_hash'][wordsize]
- return kernel_url, kernel_hash
-
- def ssh_disconnect_vm(self):
- self.ssh_session.quit()
-
- def boot_debian_wheezy_image_and_ssh_login(self, endianess, kernel_path):
- image_url, image_hash = self.get_image_info(endianess)
- image_path = self.fetch_asset(image_url, asset_hash=image_hash)
-
- self.vm.set_console()
- kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
- + 'console=ttyS0 root=/dev/sda1')
- self.vm.add_args('-no-reboot',
- '-kernel', kernel_path,
- '-append', kernel_command_line,
- '-drive', 'file=%s,snapshot=on' % image_path,
- '-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22',
- '-device', 'pcnet,netdev=vnet')
- self.vm.launch()
-
- self.log.info('VM launched, waiting for sshd')
- console_pattern = 'Starting OpenBSD Secure Shell server: sshd'
- wait_for_console_pattern(self, console_pattern, 'Oops')
- self.log.info('sshd ready')
-
- self.ssh_connect('root', 'root', False)
-
- def shutdown_via_ssh(self):
- self.ssh_command('poweroff')
- self.ssh_disconnect_vm()
- wait_for_console_pattern(self, 'Power down', 'Oops')
-
- def run_common_commands(self, wordsize):
- self.ssh_command_output_contains(
- 'cat /proc/cpuinfo',
- self.CPU_INFO[wordsize]['cpu'])
- self.ssh_command_output_contains(
- 'uname -m',
- 'mips')
- self.ssh_command_output_contains(
- 'uname -r',
- self.CPU_INFO[wordsize]['kernel_release'])
- self.ssh_command_output_contains(
- 'cat /proc/interrupts',
- 'XT-PIC timer')
- self.ssh_command_output_contains(
- 'cat /proc/interrupts',
- 'XT-PIC i8042')
- self.ssh_command_output_contains(
- 'cat /proc/interrupts',
- 'XT-PIC serial')
- self.ssh_command_output_contains(
- 'cat /proc/interrupts',
- 'XT-PIC ata_piix')
- self.ssh_command_output_contains(
- 'cat /proc/interrupts',
- 'XT-PIC eth0')
- self.ssh_command_output_contains(
- 'cat /proc/devices',
- 'input')
- self.ssh_command_output_contains(
- 'cat /proc/devices',
- 'usb')
- self.ssh_command_output_contains(
- 'cat /proc/devices',
- 'fb')
- self.ssh_command_output_contains(
- 'cat /proc/ioports',
- ' : serial')
- self.ssh_command_output_contains(
- 'cat /proc/ioports',
- ' : ata_piix')
- self.ssh_command_output_contains(
- 'cat /proc/ioports',
- ' : piix4_smbus')
- self.ssh_command_output_contains(
- 'lspci -d 11ab:4620',
- 'GT-64120')
- self.ssh_command_output_contains(
- 'cat /sys/bus/i2c/devices/i2c-0/name',
- 'SMBus PIIX4 adapter')
- self.ssh_command_output_contains(
- 'cat /proc/mtd',
- 'YAMON')
- # Empty 'Board Config' (64KB)
- self.ssh_command_output_contains(
- 'md5sum /dev/mtd2ro',
- '0dfbe8aa4c20b52e1b8bf3cb6cbdf193')
-
- def check_mips_malta(self, uname_m, endianess):
- wordsize = 64 if '64' in uname_m else 32
- kernel_url, kernel_hash = self.get_kernel_info(endianess, wordsize)
- kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
- self.boot_debian_wheezy_image_and_ssh_login(endianess, kernel_path)
-
- stdout, _ = self.ssh_command('uname -a')
- self.assertIn(True, [uname_m + " GNU/Linux" in line for line in stdout])
-
- self.run_common_commands(wordsize)
- self.shutdown_via_ssh()
- # Wait for VM to shut down gracefully
- self.vm.wait()
-
- def test_mips_malta64eb_kernel3_2_0(self):
- """
- :avocado: tags=arch:mips64
- :avocado: tags=endian:big
- :avocado: tags=device:pcnet32
- """
- self.check_mips_malta('mips64', 'be')
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 63876bc7c42..b36fcab0896 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -42,6 +42,7 @@ test_timeouts = {
'mips_malta' : 480,
'mipsel_malta' : 420,
'mipsel_replay' : 480,
+ 'mips64_malta' : 240,
'mips64el_malta' : 420,
'mips64el_replay' : 180,
'netdev_ethtool' : 180,
@@ -191,6 +192,7 @@ tests_mipsel_system_thorough = [
]
tests_mips64_system_thorough = [
+ 'mips64_malta',
'mips64_tuxrun',
]
diff --git a/tests/functional/test_mips64_malta.py b/tests/functional/test_mips64_malta.py
new file mode 100755
index 00000000000..53c3e0c1221
--- /dev/null
+++ b/tests/functional/test_mips64_malta.py
@@ -0,0 +1,35 @@
+#!/usr/bin/env python3
+#
+# Functional tests for the big-endian 64-bit MIPS Malta board
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+from qemu_test import LinuxKernelTest, Asset
+from test_mips_malta import mips_check_wheezy
+
+
+class MaltaMachineConsole(LinuxKernelTest):
+
+ ASSET_WHEEZY_KERNEL = Asset(
+ ('https://people.debian.org/~aurel32/qemu/mips/'
+ 'vmlinux-3.2.0-4-5kc-malta'),
+ '3e4ec154db080b3f1839f04dde83120654a33e5e1716863de576c47cb94f68f6')
+
+ ASSET_WHEEZY_DISK = Asset(
+ ('https://people.debian.org/~aurel32/qemu/mips/'
+ 'debian_wheezy_mips_standard.qcow2'),
+ 'de03599285b8382ad309309a6c4869f6c6c42a5cfc983342bab9ec0dfa7849a2')
+
+ def test_wheezy(self):
+ kernel_path = self.ASSET_WHEEZY_KERNEL.fetch()
+ image_path = self.ASSET_WHEEZY_DISK.fetch()
+ kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
+ + 'console=ttyS0 root=/dev/sda1')
+ mips_check_wheezy(self,
+ kernel_path, image_path, kernel_command_line, cpuinfo='MIPS 20Kc',
+ dl_file='/boot/initrd.img-3.2.0-4-5kc-malta',
+ hsum='d98b953bb4a41c0fc0fd8d19bbc691c08989ac52568c1d3054d92dfd890d3f06')
+
+
+if __name__ == '__main__':
+ LinuxKernelTest.main()
--
2.49.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* Re: [PATCH 09/15] tests/functional: Convert the 64-bit big endian Wheezy mips test
2025-03-25 20:00 ` [PATCH 09/15] tests/functional: Convert the 64-bit big " Thomas Huth
@ 2025-03-26 10:00 ` Daniel P. Berrangé
0 siblings, 0 replies; 34+ messages in thread
From: Daniel P. Berrangé @ 2025-03-26 10:00 UTC (permalink / raw)
To: Thomas Huth; +Cc: qemu-devel, Philippe Mathieu-Daudé
On Tue, Mar 25, 2025 at 09:00:17PM +0100, Thomas Huth wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> Reuse the test function from the 32-bit big endian test to easily
> convert the 64-bit big endian Wheezy mips test.
>
> Since this was the last test in tests/avocado/linux_ssh_mips_malta.py,
> we can remove this avocado file now, too.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> MAINTAINERS | 1 -
> tests/avocado/linux_ssh_mips_malta.py | 181 --------------------------
> tests/functional/meson.build | 2 +
> tests/functional/test_mips64_malta.py | 35 +++++
> 4 files changed, 37 insertions(+), 182 deletions(-)
> delete mode 100644 tests/avocado/linux_ssh_mips_malta.py
> create mode 100755 tests/functional/test_mips64_malta.py
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH 10/15] tests/avocado: Remove the boot_linux.py tests
2025-03-25 20:00 [PATCH for-10.1 00/15] Convert remaining Avocado tests to functional Thomas Huth
` (8 preceding siblings ...)
2025-03-25 20:00 ` [PATCH 09/15] tests/functional: Convert the 64-bit big " Thomas Huth
@ 2025-03-25 20:00 ` Thomas Huth
2025-03-25 20:00 ` [PATCH 11/15] tests/functional: Use the tuxrun kernel for the x86 replay test Thomas Huth
` (4 subsequent siblings)
14 siblings, 0 replies; 34+ messages in thread
From: Thomas Huth @ 2025-03-25 20:00 UTC (permalink / raw)
To: qemu-devel; +Cc: Daniel P. Berrangé, Philippe Mathieu-Daudé
From: Thomas Huth <thuth@redhat.com>
These tests are based on the cloudinit functions from Avocado.
The cloudinit is very, very slow compared to our other tests,
so most of these Avocado tests have either been disabled by default
with a decorator, or have been marked to only run with KVM.
We won't include this sluggish cloudinit stuff in the functional
framework, and we've already got plenty of other tests there that
check pretty much the same things, so let's simply get rid of these
old tests now.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/avocado/boot_linux.py | 132 ------------------------------------
1 file changed, 132 deletions(-)
delete mode 100644 tests/avocado/boot_linux.py
diff --git a/tests/avocado/boot_linux.py b/tests/avocado/boot_linux.py
deleted file mode 100644
index a029ef4ad1e..00000000000
--- a/tests/avocado/boot_linux.py
+++ /dev/null
@@ -1,132 +0,0 @@
-# Functional test that boots a complete Linux system via a cloud image
-#
-# Copyright (c) 2018-2020 Red Hat, Inc.
-#
-# Author:
-# Cleber Rosa <crosa@redhat.com>
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or
-# later. See the COPYING file in the top-level directory.
-
-import os
-
-from avocado_qemu.linuxtest import LinuxTest
-from avocado_qemu import BUILD_DIR
-
-from avocado import skipUnless
-
-
-class BootLinuxX8664(LinuxTest):
- """
- :avocado: tags=arch:x86_64
- """
- timeout = 480
-
- def test_pc_i440fx_tcg(self):
- """
- :avocado: tags=machine:pc
- :avocado: tags=accel:tcg
- """
- self.require_accelerator("tcg")
- self.vm.add_args("-accel", "tcg")
- self.launch_and_wait(set_up_ssh_connection=False)
-
- def test_pc_i440fx_kvm(self):
- """
- :avocado: tags=machine:pc
- :avocado: tags=accel:kvm
- """
- self.require_accelerator("kvm")
- self.vm.add_args("-accel", "kvm")
- self.launch_and_wait(set_up_ssh_connection=False)
-
- def test_pc_q35_tcg(self):
- """
- :avocado: tags=machine:q35
- :avocado: tags=accel:tcg
- """
- self.require_accelerator("tcg")
- self.vm.add_args("-accel", "tcg")
- self.launch_and_wait(set_up_ssh_connection=False)
-
- def test_pc_q35_kvm(self):
- """
- :avocado: tags=machine:q35
- :avocado: tags=accel:kvm
- """
- self.require_accelerator("kvm")
- self.vm.add_args("-accel", "kvm")
- self.launch_and_wait(set_up_ssh_connection=False)
-
-
-# For Aarch64 we only boot KVM tests in CI as booting the current
-# Fedora OS in TCG tests is very heavyweight. There are lighter weight
-# distros which we use in the machine_aarch64_virt.py tests.
-class BootLinuxAarch64(LinuxTest):
- """
- :avocado: tags=arch:aarch64
- :avocado: tags=machine:virt
- """
- timeout = 720
-
- def test_virt_kvm(self):
- """
- :avocado: tags=accel:kvm
- :avocado: tags=cpu:host
- """
- self.require_accelerator("kvm")
- self.vm.add_args("-accel", "kvm")
- self.vm.add_args("-machine", "virt,gic-version=host")
- self.vm.add_args('-bios',
- os.path.join(BUILD_DIR, 'pc-bios',
- 'edk2-aarch64-code.fd'))
- self.vm.add_args('-device', 'virtio-rng-pci,rng=rng0')
- self.vm.add_args('-object', 'rng-random,id=rng0,filename=/dev/urandom')
- self.launch_and_wait(set_up_ssh_connection=False)
-
-
-# See the tux_baseline.py tests for almost the same coverage in a lot
-# less time.
-class BootLinuxPPC64(LinuxTest):
- """
- :avocado: tags=arch:ppc64
- """
-
- timeout = 360
-
- @skipUnless(os.getenv('SPEED') == 'slow', 'runtime limited')
- def test_pseries_tcg(self):
- """
- :avocado: tags=machine:pseries
- :avocado: tags=accel:tcg
- """
- self.require_accelerator("tcg")
- self.vm.add_args("-accel", "tcg")
- self.launch_and_wait(set_up_ssh_connection=False)
-
- def test_pseries_kvm(self):
- """
- :avocado: tags=machine:pseries
- :avocado: tags=accel:kvm
- """
- self.require_accelerator("kvm")
- self.vm.add_args("-accel", "kvm")
- self.vm.add_args("-machine", "cap-ccf-assist=off")
- self.launch_and_wait(set_up_ssh_connection=False)
-
-class BootLinuxS390X(LinuxTest):
- """
- :avocado: tags=arch:s390x
- """
-
- timeout = 240
-
- @skipUnless(os.getenv('SPEED') == 'slow', 'runtime limited')
- def test_s390_ccw_virtio_tcg(self):
- """
- :avocado: tags=machine:s390-ccw-virtio
- :avocado: tags=accel:tcg
- """
- self.require_accelerator("tcg")
- self.vm.add_args("-accel", "tcg")
- self.launch_and_wait(set_up_ssh_connection=False)
--
2.49.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 11/15] tests/functional: Use the tuxrun kernel for the x86 replay test
2025-03-25 20:00 [PATCH for-10.1 00/15] Convert remaining Avocado tests to functional Thomas Huth
` (9 preceding siblings ...)
2025-03-25 20:00 ` [PATCH 10/15] tests/avocado: Remove the boot_linux.py tests Thomas Huth
@ 2025-03-25 20:00 ` Thomas Huth
2025-03-26 9:43 ` Daniel P. Berrangé
2025-03-25 20:00 ` [PATCH 12/15] tests/functional: Use the tuxrun kernel for the aarch64 " Thomas Huth
` (3 subsequent siblings)
14 siblings, 1 reply; 34+ messages in thread
From: Thomas Huth @ 2025-03-25 20:00 UTC (permalink / raw)
To: qemu-devel; +Cc: Daniel P. Berrangé, Philippe Mathieu-Daudé
From: Thomas Huth <thuth@redhat.com>
This way we can do a full boot in record-replay mode and
should get a similar test coverage compared to the old
replay test from tests/avocado/replay_linux.py. Thus remove
the x86 avocado replay_linux test now.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/avocado/replay_linux.py | 46 --------------------------
tests/functional/test_x86_64_replay.py | 43 ++++++++++++++++++------
2 files changed, 33 insertions(+), 56 deletions(-)
diff --git a/tests/avocado/replay_linux.py b/tests/avocado/replay_linux.py
index 59169224353..6ba283d3bf6 100644
--- a/tests/avocado/replay_linux.py
+++ b/tests/avocado/replay_linux.py
@@ -118,52 +118,6 @@ def run_replay_dump(self, replay_path):
except subprocess.CalledProcessError:
self.fail('replay-dump.py failed')
-@skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
-class ReplayLinuxX8664(ReplayLinux):
- """
- :avocado: tags=arch:x86_64
- :avocado: tags=accel:tcg
- """
-
- chksum = 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0'
-
- def test_pc_i440fx(self):
- """
- :avocado: tags=machine:pc
- """
- self.run_rr(shift=1)
-
- def test_pc_q35(self):
- """
- :avocado: tags=machine:q35
- """
- self.run_rr(shift=3)
-
-@skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
-class ReplayLinuxX8664Virtio(ReplayLinux):
- """
- :avocado: tags=arch:x86_64
- :avocado: tags=virtio
- :avocado: tags=accel:tcg
- """
-
- hdd = 'virtio-blk-pci'
- cd = 'virtio-blk-pci'
- bus = None
-
- chksum = 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0'
-
- def test_pc_i440fx(self):
- """
- :avocado: tags=machine:pc
- """
- self.run_rr(shift=1)
-
- def test_pc_q35(self):
- """
- :avocado: tags=machine:q35
- """
- self.run_rr(shift=3)
@skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
class ReplayLinuxAarch64(ReplayLinux):
diff --git a/tests/functional/test_x86_64_replay.py b/tests/functional/test_x86_64_replay.py
index 180f23a60c5..27287d452dc 100755
--- a/tests/functional/test_x86_64_replay.py
+++ b/tests/functional/test_x86_64_replay.py
@@ -5,30 +5,53 @@
#
# SPDX-License-Identifier: GPL-2.0-or-later
-from qemu_test import Asset, skipFlakyTest
+from subprocess import check_call, DEVNULL
+
+from qemu_test import Asset, skipFlakyTest, get_qemu_img
from replay_kernel import ReplayKernelBase
class X86Replay(ReplayKernelBase):
ASSET_KERNEL = Asset(
- ('https://archives.fedoraproject.org/pub/archive/fedora/linux'
- '/releases/29/Everything/x86_64/os/images/pxeboot/vmlinuz'),
- '8f237d84712b1b411baf3af2aeaaee10b9aae8e345ec265b87ab3a39639eb143')
+ 'https://storage.tuxboot.com/buildroot/20241119/x86_64/bzImage',
+ 'f57bfc6553bcd6e0a54aab86095bf642b33b5571d14e3af1731b18c87ed5aef8')
+
+ ASSET_ROOTFS = Asset(
+ 'https://storage.tuxboot.com/buildroot/20241119/x86_64/rootfs.ext4.zst',
+ '4b8b2a99117519c5290e1202cb36eb6c7aaba92b357b5160f5970cf5fb78a751')
- def do_test_x86(self, machine):
+ def do_test_x86(self, machine, blkdevice, devroot):
+ self.require_netdev('user')
self.set_machine(machine)
+ self.cpu="Nehalem"
kernel_path = self.ASSET_KERNEL.fetch()
- kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
- console_pattern = 'VFS: Cannot open root device'
- self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5)
+
+ raw_disk = self.uncompress(self.ASSET_ROOTFS)
+ disk = self.scratch_file('scratch.qcow2')
+ qemu_img = get_qemu_img(self)
+ check_call([qemu_img, 'create', '-f', 'qcow2', '-b', raw_disk,
+ '-F', 'raw', disk], stdout=DEVNULL, stderr=DEVNULL)
+
+ args = ('-drive', 'file=%s,snapshot=on,id=hd0,if=none' % disk,
+ '-drive', 'driver=blkreplay,id=hd0-rr,if=none,image=hd0',
+ '-device', '%s,drive=hd0-rr' % blkdevice,
+ '-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22',
+ '-device', 'virtio-net,netdev=vnet',
+ '-object', 'filter-replay,id=replay,netdev=vnet')
+
+ kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
+ f"console=ttyS0 root=/dev/{devroot}")
+ console_pattern = 'Welcome to TuxTest'
+ self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5,
+ args=args)
@skipFlakyTest('https://gitlab.com/qemu-project/qemu/-/issues/2094')
def test_pc(self):
- self.do_test_x86('pc')
+ self.do_test_x86('pc', 'virtio-blk', 'vda')
def test_q35(self):
- self.do_test_x86('q35')
+ self.do_test_x86('q35', 'ide-hd', 'sda')
if __name__ == '__main__':
--
2.49.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* Re: [PATCH 11/15] tests/functional: Use the tuxrun kernel for the x86 replay test
2025-03-25 20:00 ` [PATCH 11/15] tests/functional: Use the tuxrun kernel for the x86 replay test Thomas Huth
@ 2025-03-26 9:43 ` Daniel P. Berrangé
2025-03-26 9:54 ` Thomas Huth
0 siblings, 1 reply; 34+ messages in thread
From: Daniel P. Berrangé @ 2025-03-26 9:43 UTC (permalink / raw)
To: Thomas Huth; +Cc: qemu-devel, Philippe Mathieu-Daudé
On Tue, Mar 25, 2025 at 09:00:19PM +0100, Thomas Huth wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> This way we can do a full boot in record-replay mode and
> should get a similar test coverage compared to the old
> replay test from tests/avocado/replay_linux.py. Thus remove
> the x86 avocado replay_linux test now.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/avocado/replay_linux.py | 46 --------------------------
> tests/functional/test_x86_64_replay.py | 43 ++++++++++++++++++------
> 2 files changed, 33 insertions(+), 56 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
> diff --git a/tests/functional/test_x86_64_replay.py b/tests/functional/test_x86_64_replay.py
> index 180f23a60c5..27287d452dc 100755
> --- a/tests/functional/test_x86_64_replay.py
> +++ b/tests/functional/test_x86_64_replay.py
> @@ -5,30 +5,53 @@
> #
> # SPDX-License-Identifier: GPL-2.0-or-later
>
> -from qemu_test import Asset, skipFlakyTest
> +from subprocess import check_call, DEVNULL
> +
> +from qemu_test import Asset, skipFlakyTest, get_qemu_img
> from replay_kernel import ReplayKernelBase
>
>
> class X86Replay(ReplayKernelBase):
>
> ASSET_KERNEL = Asset(
> - ('https://archives.fedoraproject.org/pub/archive/fedora/linux'
> - '/releases/29/Everything/x86_64/os/images/pxeboot/vmlinuz'),
> - '8f237d84712b1b411baf3af2aeaaee10b9aae8e345ec265b87ab3a39639eb143')
> + 'https://storage.tuxboot.com/buildroot/20241119/x86_64/bzImage',
> + 'f57bfc6553bcd6e0a54aab86095bf642b33b5571d14e3af1731b18c87ed5aef8')
> +
> + ASSET_ROOTFS = Asset(
> + 'https://storage.tuxboot.com/buildroot/20241119/x86_64/rootfs.ext4.zst',
> + '4b8b2a99117519c5290e1202cb36eb6c7aaba92b357b5160f5970cf5fb78a751')
As a general question, I wonder if we want to add some logic to the
pre-cache job to clean up old cached files.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 34+ messages in thread* Re: [PATCH 11/15] tests/functional: Use the tuxrun kernel for the x86 replay test
2025-03-26 9:43 ` Daniel P. Berrangé
@ 2025-03-26 9:54 ` Thomas Huth
2025-03-26 10:02 ` Daniel P. Berrangé
0 siblings, 1 reply; 34+ messages in thread
From: Thomas Huth @ 2025-03-26 9:54 UTC (permalink / raw)
To: Daniel P. Berrangé; +Cc: qemu-devel, Philippe Mathieu-Daudé
On 26/03/2025 10.43, Daniel P. Berrangé wrote:
> On Tue, Mar 25, 2025 at 09:00:19PM +0100, Thomas Huth wrote:
>> From: Thomas Huth <thuth@redhat.com>
>>
>> This way we can do a full boot in record-replay mode and
>> should get a similar test coverage compared to the old
>> replay test from tests/avocado/replay_linux.py. Thus remove
>> the x86 avocado replay_linux test now.
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>> tests/avocado/replay_linux.py | 46 --------------------------
>> tests/functional/test_x86_64_replay.py | 43 ++++++++++++++++++------
>> 2 files changed, 33 insertions(+), 56 deletions(-)
>
> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
>
>> diff --git a/tests/functional/test_x86_64_replay.py b/tests/functional/test_x86_64_replay.py
>> index 180f23a60c5..27287d452dc 100755
>> --- a/tests/functional/test_x86_64_replay.py
>> +++ b/tests/functional/test_x86_64_replay.py
>> @@ -5,30 +5,53 @@
>> #
>> # SPDX-License-Identifier: GPL-2.0-or-later
>>
>> -from qemu_test import Asset, skipFlakyTest
>> +from subprocess import check_call, DEVNULL
>> +
>> +from qemu_test import Asset, skipFlakyTest, get_qemu_img
>> from replay_kernel import ReplayKernelBase
>>
>>
>> class X86Replay(ReplayKernelBase):
>>
>> ASSET_KERNEL = Asset(
>> - ('https://archives.fedoraproject.org/pub/archive/fedora/linux'
>> - '/releases/29/Everything/x86_64/os/images/pxeboot/vmlinuz'),
>> - '8f237d84712b1b411baf3af2aeaaee10b9aae8e345ec265b87ab3a39639eb143')
>> + 'https://storage.tuxboot.com/buildroot/20241119/x86_64/bzImage',
>> + 'f57bfc6553bcd6e0a54aab86095bf642b33b5571d14e3af1731b18c87ed5aef8')
>> +
>> + ASSET_ROOTFS = Asset(
>> + 'https://storage.tuxboot.com/buildroot/20241119/x86_64/rootfs.ext4.zst',
>> + '4b8b2a99117519c5290e1202cb36eb6c7aaba92b357b5160f5970cf5fb78a751')
>
>
> As a general question, I wonder if we want to add some logic to the
> pre-cache job to clean up old cached files.
Yes, I also asked myself that question already. Being a "cache", there
should also be a way to evict old files that are not used anymore.
Maybe we could update the timestamp of the assets each time they are used,
and when doing "make clean", we also check the cache and delete the assets
with timestamps older than 6 months or so?
Thomas
^ permalink raw reply [flat|nested] 34+ messages in thread* Re: [PATCH 11/15] tests/functional: Use the tuxrun kernel for the x86 replay test
2025-03-26 9:54 ` Thomas Huth
@ 2025-03-26 10:02 ` Daniel P. Berrangé
0 siblings, 0 replies; 34+ messages in thread
From: Daniel P. Berrangé @ 2025-03-26 10:02 UTC (permalink / raw)
To: Thomas Huth; +Cc: qemu-devel, Philippe Mathieu-Daudé
On Wed, Mar 26, 2025 at 10:54:53AM +0100, Thomas Huth wrote:
> On 26/03/2025 10.43, Daniel P. Berrangé wrote:
> > On Tue, Mar 25, 2025 at 09:00:19PM +0100, Thomas Huth wrote:
> > > From: Thomas Huth <thuth@redhat.com>
> > >
> > > This way we can do a full boot in record-replay mode and
> > > should get a similar test coverage compared to the old
> > > replay test from tests/avocado/replay_linux.py. Thus remove
> > > the x86 avocado replay_linux test now.
> > >
> > > Signed-off-by: Thomas Huth <thuth@redhat.com>
> > > ---
> > > tests/avocado/replay_linux.py | 46 --------------------------
> > > tests/functional/test_x86_64_replay.py | 43 ++++++++++++++++++------
> > > 2 files changed, 33 insertions(+), 56 deletions(-)
> >
> > Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
> >
> > > diff --git a/tests/functional/test_x86_64_replay.py b/tests/functional/test_x86_64_replay.py
> > > index 180f23a60c5..27287d452dc 100755
> > > --- a/tests/functional/test_x86_64_replay.py
> > > +++ b/tests/functional/test_x86_64_replay.py
> > > @@ -5,30 +5,53 @@
> > > #
> > > # SPDX-License-Identifier: GPL-2.0-or-later
> > > -from qemu_test import Asset, skipFlakyTest
> > > +from subprocess import check_call, DEVNULL
> > > +
> > > +from qemu_test import Asset, skipFlakyTest, get_qemu_img
> > > from replay_kernel import ReplayKernelBase
> > > class X86Replay(ReplayKernelBase):
> > > ASSET_KERNEL = Asset(
> > > - ('https://archives.fedoraproject.org/pub/archive/fedora/linux'
> > > - '/releases/29/Everything/x86_64/os/images/pxeboot/vmlinuz'),
> > > - '8f237d84712b1b411baf3af2aeaaee10b9aae8e345ec265b87ab3a39639eb143')
> > > + 'https://storage.tuxboot.com/buildroot/20241119/x86_64/bzImage',
> > > + 'f57bfc6553bcd6e0a54aab86095bf642b33b5571d14e3af1731b18c87ed5aef8')
> > > +
> > > + ASSET_ROOTFS = Asset(
> > > + 'https://storage.tuxboot.com/buildroot/20241119/x86_64/rootfs.ext4.zst',
> > > + '4b8b2a99117519c5290e1202cb36eb6c7aaba92b357b5160f5970cf5fb78a751')
> >
> >
> > As a general question, I wonder if we want to add some logic to the
> > pre-cache job to clean up old cached files.
>
> Yes, I also asked myself that question already. Being a "cache", there
> should also be a way to evict old files that are not used anymore.
> Maybe we could update the timestamp of the assets each time they are used,
> and when doing "make clean", we also check the cache and delete the assets
> with timestamps older than 6 months or so?
Yeah, some time based check + "touch" would be needed, so we don't
aggressively purge files cached by a dev's use a different branch
that happens to not be checked out currently.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH 12/15] tests/functional: Use the tuxrun kernel for the aarch64 replay test
2025-03-25 20:00 [PATCH for-10.1 00/15] Convert remaining Avocado tests to functional Thomas Huth
` (10 preceding siblings ...)
2025-03-25 20:00 ` [PATCH 11/15] tests/functional: Use the tuxrun kernel for the x86 replay test Thomas Huth
@ 2025-03-25 20:00 ` Thomas Huth
2025-03-26 9:46 ` Daniel P. Berrangé
2025-03-25 20:00 ` [PATCH 13/15] tests/functional: Convert the SMMU test to the functional framework Thomas Huth
` (2 subsequent siblings)
14 siblings, 1 reply; 34+ messages in thread
From: Thomas Huth @ 2025-03-25 20:00 UTC (permalink / raw)
To: qemu-devel; +Cc: Daniel P. Berrangé, Philippe Mathieu-Daudé
From: Thomas Huth <thuth@redhat.com>
This way we can do a full boot in record-replay mode and
should get a similar test coverage compared to the old
replay test from tests/avocado/replay_linux.py.
Since the aarch64 test was the last avocado test in the
tests/avocado/replay_linux.py file, we can remove this
file now completely.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 1 -
tests/avocado/replay_linux.py | 160 ------------------------
tests/functional/test_aarch64_replay.py | 37 ++++--
3 files changed, 29 insertions(+), 169 deletions(-)
delete mode 100644 tests/avocado/replay_linux.py
diff --git a/MAINTAINERS b/MAINTAINERS
index 7b843c543ff..acdd8941168 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3667,7 +3667,6 @@ F: include/system/replay.h
F: docs/devel/replay.rst
F: docs/system/replay.rst
F: stubs/replay.c
-F: tests/avocado/replay_linux.py
F: tests/functional/*reverse_debug*.py
F: tests/functional/*replay*.py
F: qapi/replay.json
diff --git a/tests/avocado/replay_linux.py b/tests/avocado/replay_linux.py
deleted file mode 100644
index 6ba283d3bf6..00000000000
--- a/tests/avocado/replay_linux.py
+++ /dev/null
@@ -1,160 +0,0 @@
-# Record/replay test that boots a complete Linux system via a cloud image
-#
-# Copyright (c) 2020 ISP RAS
-#
-# Author:
-# Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or
-# later. See the COPYING file in the top-level directory.
-
-import os
-import logging
-import time
-
-from avocado import skipUnless
-from avocado_qemu import BUILD_DIR
-from avocado.utils import cloudinit
-from avocado.utils import network
-from avocado.utils import vmimage
-from avocado.utils import datadrainer
-from avocado.utils.path import find_command
-from avocado_qemu.linuxtest import LinuxTest
-
-class ReplayLinux(LinuxTest):
- """
- Boots a Linux system, checking for a successful initialization
- """
-
- timeout = 1800
- chksum = None
- hdd = 'ide-hd'
- cd = 'ide-cd'
- bus = 'ide'
-
- def setUp(self):
- # LinuxTest does many replay-incompatible things, but includes
- # useful methods. Do not setup LinuxTest here and just
- # call some functions.
- super(LinuxTest, self).setUp()
- self._set_distro()
- self.boot_path = self.download_boot()
- self.phone_server = cloudinit.PhoneHomeServer(('0.0.0.0', 0),
- self.name)
- ssh_pubkey, self.ssh_key = self.set_up_existing_ssh_keys()
- self.cloudinit_path = self.prepare_cloudinit(ssh_pubkey)
-
- def vm_add_disk(self, vm, path, id, device):
- bus_string = ''
- if self.bus:
- bus_string = ',bus=%s.%d' % (self.bus, id,)
- vm.add_args('-drive', 'file=%s,snapshot=on,id=disk%s,if=none' % (path, id))
- vm.add_args('-drive',
- 'driver=blkreplay,id=disk%s-rr,if=none,image=disk%s' % (id, id))
- vm.add_args('-device',
- '%s,drive=disk%s-rr%s' % (device, id, bus_string))
-
- def vm_add_cdrom(self, vm, path, id, device):
- vm.add_args('-drive', 'file=%s,id=disk%s,if=none,media=cdrom' % (path, id))
-
- def launch_and_wait(self, record, args, shift):
- self.require_netdev('user')
- vm = self.get_vm()
- vm.add_args('-smp', '1')
- vm.add_args('-m', '1024')
- vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22',
- '-device', 'virtio-net,netdev=vnet')
- vm.add_args('-object', 'filter-replay,id=replay,netdev=vnet')
- if args:
- vm.add_args(*args)
- self.vm_add_disk(vm, self.boot_path, 0, self.hdd)
- self.vm_add_cdrom(vm, self.cloudinit_path, 1, self.cd)
- logger = logging.getLogger('replay')
- if record:
- logger.info('recording the execution...')
- mode = 'record'
- else:
- logger.info('replaying the execution...')
- mode = 'replay'
- replay_path = os.path.join(self.workdir, 'replay.bin')
- vm.add_args('-icount', 'shift=%s,rr=%s,rrfile=%s' %
- (shift, mode, replay_path))
-
- start_time = time.time()
-
- vm.set_console()
- vm.launch()
- console_drainer = datadrainer.LineLogger(vm.console_socket.fileno(),
- logger=self.log.getChild('console'),
- stop_check=(lambda : not vm.is_running()))
- console_drainer.start()
- if record:
- while not self.phone_server.instance_phoned_back:
- self.phone_server.handle_request()
- vm.shutdown()
- logger.info('finished the recording with log size %s bytes'
- % os.path.getsize(replay_path))
- self.run_replay_dump(replay_path)
- logger.info('successfully tested replay-dump.py')
- else:
- vm.event_wait('SHUTDOWN', self.timeout)
- vm.wait()
- logger.info('successfully finished the replay')
- elapsed = time.time() - start_time
- logger.info('elapsed time %.2f sec' % elapsed)
- return elapsed
-
- def run_rr(self, args=None, shift=7):
- t1 = self.launch_and_wait(True, args, shift)
- t2 = self.launch_and_wait(False, args, shift)
- logger = logging.getLogger('replay')
- logger.info('replay overhead {:.2%}'.format(t2 / t1 - 1))
-
- def run_replay_dump(self, replay_path):
- try:
- subprocess.check_call(["./scripts/replay-dump.py",
- "-f", replay_path],
- stdout=subprocess.DEVNULL)
- except subprocess.CalledProcessError:
- self.fail('replay-dump.py failed')
-
-
-@skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
-class ReplayLinuxAarch64(ReplayLinux):
- """
- :avocado: tags=accel:tcg
- :avocado: tags=arch:aarch64
- :avocado: tags=machine:virt
- :avocado: tags=cpu:max
- """
-
- chksum = '1e18d9c0cf734940c4b5d5ec592facaed2af0ad0329383d5639c997fdf16fe49'
-
- hdd = 'virtio-blk-device'
- cd = 'virtio-blk-device'
- bus = None
-
- def get_common_args(self):
- return ('-bios',
- os.path.join(BUILD_DIR, 'pc-bios', 'edk2-aarch64-code.fd'),
- "-cpu", "max,lpa2=off",
- '-device', 'virtio-rng-pci,rng=rng0',
- '-object', 'rng-builtin,id=rng0')
-
- def test_virt_gicv2(self):
- """
- :avocado: tags=machine:gic-version=2
- """
-
- self.run_rr(shift=3,
- args=(*self.get_common_args(),
- "-machine", "virt,gic-version=2"))
-
- def test_virt_gicv3(self):
- """
- :avocado: tags=machine:gic-version=3
- """
-
- self.run_rr(shift=3,
- args=(*self.get_common_args(),
- "-machine", "virt,gic-version=3"))
diff --git a/tests/functional/test_aarch64_replay.py b/tests/functional/test_aarch64_replay.py
index 04cde433bcf..04dc32722c9 100755
--- a/tests/functional/test_aarch64_replay.py
+++ b/tests/functional/test_aarch64_replay.py
@@ -5,25 +5,46 @@
#
# SPDX-License-Identifier: GPL-2.0-or-later
-from qemu_test import Asset
+from subprocess import check_call, DEVNULL
+
+from qemu_test import Asset, get_qemu_img
from replay_kernel import ReplayKernelBase
class Aarch64Replay(ReplayKernelBase):
ASSET_KERNEL = Asset(
- ('https://archives.fedoraproject.org/pub/archive/fedora/linux/'
- 'releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz'),
- '7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7')
+ 'https://storage.tuxboot.com/buildroot/20241119/arm64/Image',
+ 'b74743c5e89e1cea0f73368d24ae0ae85c5204ff84be3b5e9610417417d2f235')
+
+ ASSET_ROOTFS = Asset(
+ 'https://storage.tuxboot.com/buildroot/20241119/arm64/rootfs.ext4.zst',
+ 'a1acaaae2068df4648d04ff75f532aaa8c5edcd6b936122b6f0db4848a07b465')
def test_aarch64_virt(self):
+ self.require_netdev('user')
self.set_machine('virt')
- self.cpu = 'cortex-a53'
+ self.cpu = 'cortex-a57'
kernel_path = self.ASSET_KERNEL.fetch()
+
+ raw_disk = self.uncompress(self.ASSET_ROOTFS)
+ disk = self.scratch_file('scratch.qcow2')
+ qemu_img = get_qemu_img(self)
+ check_call([qemu_img, 'create', '-f', 'qcow2', '-b', raw_disk,
+ '-F', 'raw', disk], stdout=DEVNULL, stderr=DEVNULL)
+
+ args = ('-drive', 'file=%s,snapshot=on,id=hd0,if=none' % disk,
+ '-drive', 'driver=blkreplay,id=hd0-rr,if=none,image=hd0',
+ '-device', 'virtio-blk-device,drive=hd0-rr',
+ '-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22',
+ '-device', 'virtio-net,netdev=vnet',
+ '-object', 'filter-replay,id=replay,netdev=vnet')
+
kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
- 'console=ttyAMA0')
- console_pattern = 'VFS: Cannot open root device'
- self.run_rr(kernel_path, kernel_command_line, console_pattern)
+ 'console=ttyAMA0 root=/dev/vda')
+ console_pattern = 'Welcome to TuxTest'
+ self.run_rr(kernel_path, kernel_command_line, console_pattern,
+ args=args)
if __name__ == '__main__':
--
2.49.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 13/15] tests/functional: Convert the SMMU test to the functional framework
2025-03-25 20:00 [PATCH for-10.1 00/15] Convert remaining Avocado tests to functional Thomas Huth
` (11 preceding siblings ...)
2025-03-25 20:00 ` [PATCH 12/15] tests/functional: Use the tuxrun kernel for the aarch64 " Thomas Huth
@ 2025-03-25 20:00 ` Thomas Huth
2025-03-25 20:00 ` [PATCH 14/15] gitlab-ci: Update QEMU_JOB_AVOCADO and QEMU_CI_AVOCADO_TESTING Thomas Huth
2025-03-25 20:00 ` [PATCH 15/15] Remove the remainders of the Avocado tests Thomas Huth
14 siblings, 0 replies; 34+ messages in thread
From: Thomas Huth @ 2025-03-25 20:00 UTC (permalink / raw)
To: qemu-devel; +Cc: Daniel P. Berrangé, Philippe Mathieu-Daudé
From: Thomas Huth <thuth@redhat.com>
This test was using cloudinit and a "dnf install" command in the guest
to exercise the NIC with SMMU enabled. Since we don't have the cloudinit
stuff in the functional framework and we should not rely on having access
to external networks (once our ASSETs have been cached), we rather boot
into the initrd first, manually mount the root disk and then use the
check_http_download() function from the functional framework here instead
for testing whether the network works as expected.
Unfortunately, there seems to be a small race when using the files
from Fedora 33: To enter the initrd shell, we have to send a "return"
once. But it does not seem to work if we send it too early. Using a
sleep(0.2) makes it work reliably for me, but to make it even more
unlikely to trigger this situation, let's better limit the Fedora 33
tests to only run with KVM.
Finally, while we're at it, we also add some lines for testing writes
to the hard disk, as we already do it in the test_intel_iommu test.
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Tested-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 2 +-
tests/avocado/smmu.py | 139 -----------------
tests/functional/meson.build | 2 +
tests/functional/test_aarch64_smmu.py | 205 ++++++++++++++++++++++++++
4 files changed, 208 insertions(+), 140 deletions(-)
delete mode 100644 tests/avocado/smmu.py
create mode 100755 tests/functional/test_aarch64_smmu.py
diff --git a/MAINTAINERS b/MAINTAINERS
index acdd8941168..9bd9cb836ce 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -211,7 +211,7 @@ L: qemu-arm@nongnu.org
S: Maintained
F: hw/arm/smmu*
F: include/hw/arm/smmu*
-F: tests/avocado/smmu.py
+F: tests/functional/test_aarch64_smmu.py
AVR TCG CPUs
M: Michael Rolnik <mrolnik@gmail.com>
diff --git a/tests/avocado/smmu.py b/tests/avocado/smmu.py
deleted file mode 100644
index 83fd79e922e..00000000000
--- a/tests/avocado/smmu.py
+++ /dev/null
@@ -1,139 +0,0 @@
-# SMMUv3 Functional tests
-#
-# Copyright (c) 2021 Red Hat, Inc.
-#
-# Author:
-# Eric Auger <eric.auger@redhat.com>
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or
-# later. See the COPYING file in the top-level directory.
-import os
-
-from avocado import skipUnless
-from avocado_qemu import BUILD_DIR
-from avocado_qemu.linuxtest import LinuxTest
-
-@skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab')
-class SMMU(LinuxTest):
- """
- :avocado: tags=accel:kvm
- :avocado: tags=cpu:host
- :avocado: tags=arch:aarch64
- :avocado: tags=machine:virt
- :avocado: tags=distro:fedora
- :avocado: tags=smmu
- :avocado: tags=flaky
- """
-
- IOMMU_ADDON = ',iommu_platform=on,disable-modern=off,disable-legacy=on'
- kernel_path = None
- initrd_path = None
- kernel_params = None
-
- def set_up_boot(self):
- path = self.download_boot()
- self.vm.add_args('-device', 'virtio-blk-pci,bus=pcie.0,' +
- 'drive=drv0,id=virtio-disk0,bootindex=1,'
- 'werror=stop,rerror=stop' + self.IOMMU_ADDON)
- self.vm.add_args('-drive',
- 'file=%s,if=none,cache=writethrough,id=drv0' % path)
-
- def setUp(self):
- super(SMMU, self).setUp(None, 'virtio-net-pci' + self.IOMMU_ADDON)
-
- def common_vm_setup(self, custom_kernel=False):
- self.require_accelerator("kvm")
- self.vm.add_args("-accel", "kvm")
- self.vm.add_args("-cpu", "host")
- self.vm.add_args("-machine", "iommu=smmuv3")
- self.vm.add_args("-d", "guest_errors")
- self.vm.add_args('-bios', os.path.join(BUILD_DIR, 'pc-bios',
- 'edk2-aarch64-code.fd'))
- self.vm.add_args('-device', 'virtio-rng-pci,rng=rng0')
- self.vm.add_args('-object',
- 'rng-random,id=rng0,filename=/dev/urandom')
-
- if custom_kernel is False:
- return
-
- kernel_url = self.distro.pxeboot_url + 'vmlinuz'
- initrd_url = self.distro.pxeboot_url + 'initrd.img'
- self.kernel_path = self.fetch_asset(kernel_url)
- self.initrd_path = self.fetch_asset(initrd_url)
-
- def run_and_check(self):
- if self.kernel_path:
- self.vm.add_args('-kernel', self.kernel_path,
- '-append', self.kernel_params,
- '-initrd', self.initrd_path)
- self.launch_and_wait()
- self.ssh_command('cat /proc/cmdline')
- self.ssh_command('dnf -y install numactl-devel')
-
-
- # 5.3 kernel without RIL #
-
- def test_smmu_noril(self):
- """
- :avocado: tags=smmu_noril
- :avocado: tags=smmu_noril_tests
- :avocado: tags=distro_version:31
- """
- self.common_vm_setup()
- self.run_and_check()
-
- def test_smmu_noril_passthrough(self):
- """
- :avocado: tags=smmu_noril_passthrough
- :avocado: tags=smmu_noril_tests
- :avocado: tags=distro_version:31
- """
- self.common_vm_setup(True)
- self.kernel_params = (self.distro.default_kernel_params +
- ' iommu.passthrough=on')
- self.run_and_check()
-
- def test_smmu_noril_nostrict(self):
- """
- :avocado: tags=smmu_noril_nostrict
- :avocado: tags=smmu_noril_tests
- :avocado: tags=distro_version:31
- """
- self.common_vm_setup(True)
- self.kernel_params = (self.distro.default_kernel_params +
- ' iommu.strict=0')
- self.run_and_check()
-
- # 5.8 kernel featuring range invalidation
- # >= v5.7 kernel
-
- def test_smmu_ril(self):
- """
- :avocado: tags=smmu_ril
- :avocado: tags=smmu_ril_tests
- :avocado: tags=distro_version:33
- """
- self.common_vm_setup()
- self.run_and_check()
-
- def test_smmu_ril_passthrough(self):
- """
- :avocado: tags=smmu_ril_passthrough
- :avocado: tags=smmu_ril_tests
- :avocado: tags=distro_version:33
- """
- self.common_vm_setup(True)
- self.kernel_params = (self.distro.default_kernel_params +
- ' iommu.passthrough=on')
- self.run_and_check()
-
- def test_smmu_ril_nostrict(self):
- """
- :avocado: tags=smmu_ril_nostrict
- :avocado: tags=smmu_ril_tests
- :avocado: tags=distro_version:33
- """
- self.common_vm_setup(True)
- self.kernel_params = (self.distro.default_kernel_params +
- ' iommu.strict=0')
- self.run_and_check()
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index b36fcab0896..8c1d3bb4bdb 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -18,6 +18,7 @@ test_timeouts = {
'aarch64_rme_sbsaref' : 1200,
'aarch64_sbsaref_alpine' : 1200,
'aarch64_sbsaref_freebsd' : 720,
+ 'aarch64_smmu' : 720,
'aarch64_tuxrun' : 240,
'aarch64_virt' : 360,
'aarch64_virt_gpu' : 480,
@@ -88,6 +89,7 @@ tests_aarch64_system_thorough = [
'aarch64_sbsaref',
'aarch64_sbsaref_alpine',
'aarch64_sbsaref_freebsd',
+ 'aarch64_smmu',
'aarch64_tcg_plugins',
'aarch64_tuxrun',
'aarch64_virt',
diff --git a/tests/functional/test_aarch64_smmu.py b/tests/functional/test_aarch64_smmu.py
new file mode 100755
index 00000000000..c65d0f28178
--- /dev/null
+++ b/tests/functional/test_aarch64_smmu.py
@@ -0,0 +1,205 @@
+#!/usr/bin/env python3
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# SMMUv3 Functional tests
+#
+# Copyright (c) 2021 Red Hat, Inc.
+#
+# Author:
+# Eric Auger <eric.auger@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import os
+import time
+
+from qemu_test import LinuxKernelTest, Asset, exec_command_and_wait_for_pattern
+from qemu_test import BUILD_DIR
+from qemu.utils import kvm_available
+
+
+class SMMU(LinuxKernelTest):
+
+ default_kernel_params = ('earlyprintk=pl011,0x9000000 no_timer_check '
+ 'printk.time=1 rd_NO_PLYMOUTH net.ifnames=0 '
+ 'console=ttyAMA0 rd.rescue')
+ IOMMU_ADDON = ',iommu_platform=on,disable-modern=off,disable-legacy=on'
+ kernel_path = None
+ initrd_path = None
+ kernel_params = None
+
+ GUEST_PORT = 8080
+
+ def set_up_boot(self, path):
+ self.vm.add_args('-device', 'virtio-blk-pci,bus=pcie.0,' +
+ 'drive=drv0,id=virtio-disk0,bootindex=1,'
+ 'werror=stop,rerror=stop' + self.IOMMU_ADDON)
+ self.vm.add_args('-drive',
+ f'file={path},if=none,cache=writethrough,id=drv0,snapshot=on')
+
+ self.vm.add_args('-netdev',
+ 'user,id=n1,hostfwd=tcp:127.0.0.1:0-:%d' %
+ self.GUEST_PORT)
+ self.vm.add_args('-device', 'virtio-net,netdev=n1' + self.IOMMU_ADDON)
+
+ def common_vm_setup(self, kernel, initrd, disk):
+ self.require_accelerator("kvm")
+ self.require_netdev('user')
+ self.set_machine("virt")
+ self.vm.add_args('-m', '1G')
+ self.vm.add_args("-accel", "kvm")
+ self.vm.add_args("-cpu", "host")
+ self.vm.add_args("-machine", "iommu=smmuv3")
+ self.vm.add_args("-d", "guest_errors")
+ self.vm.add_args('-bios', os.path.join(BUILD_DIR, 'pc-bios',
+ 'edk2-aarch64-code.fd'))
+ self.vm.add_args('-device', 'virtio-rng-pci,rng=rng0')
+ self.vm.add_args('-object',
+ 'rng-random,id=rng0,filename=/dev/urandom')
+
+ self.kernel_path = kernel.fetch()
+ self.initrd_path = initrd.fetch()
+ self.set_up_boot(disk.fetch())
+
+ def run_and_check(self, filename, hashsum):
+ self.vm.add_args('-initrd', self.initrd_path)
+ self.vm.add_args('-append', self.kernel_params)
+ self.launch_kernel(self.kernel_path, initrd=self.initrd_path,
+ wait_for='attach it to a bug report.')
+ prompt = '# '
+ # Fedora 33 requires 'return' to be pressed to enter the shell.
+ # There seems to be a small race between detecting the previous ':'
+ # and sending the newline, so we need to add a small delay here.
+ self.wait_for_console_pattern(':')
+ time.sleep(0.2)
+ exec_command_and_wait_for_pattern(self, '\n', prompt)
+ exec_command_and_wait_for_pattern(self, 'cat /proc/cmdline',
+ self.kernel_params)
+
+ # Checking for SMMU enablement:
+ self.log.info("Checking whether SMMU has been enabled...")
+ exec_command_and_wait_for_pattern(self, 'dmesg | grep smmu',
+ 'arm-smmu-v3')
+ self.wait_for_console_pattern(prompt)
+ exec_command_and_wait_for_pattern(self,
+ 'find /sys/kernel/iommu_groups/ -type l',
+ 'devices/0000:00:')
+ self.wait_for_console_pattern(prompt)
+
+ # Copy a file (checked later), umount afterwards to drop disk cache:
+ self.log.info("Checking hard disk...")
+ exec_command_and_wait_for_pattern(self,
+ "while ! (dmesg -c | grep vda:) ; do sleep 1 ; done",
+ "vda2")
+ exec_command_and_wait_for_pattern(self, 'mount /dev/vda2 /sysroot',
+ 'mounted filesystem')
+ exec_command_and_wait_for_pattern(self, 'cp /bin/vi /sysroot/root/vi',
+ prompt)
+ exec_command_and_wait_for_pattern(self, 'umount /sysroot', prompt)
+ # Switch from initrd to the cloud image filesystem:
+ exec_command_and_wait_for_pattern(self, 'mount /dev/vda2 /sysroot',
+ prompt)
+ exec_command_and_wait_for_pattern(self,
+ ('for d in dev proc sys run ; do '
+ 'mount -o bind /$d /sysroot/$d ; done'), prompt)
+ exec_command_and_wait_for_pattern(self, 'chroot /sysroot', prompt)
+ # Check files on the hard disk:
+ exec_command_and_wait_for_pattern(self,
+ ('if diff -q /root/vi /usr/bin/vi ; then echo "file" "ok" ; '
+ 'else echo "files differ"; fi'), 'file ok')
+ self.wait_for_console_pattern(prompt)
+ exec_command_and_wait_for_pattern(self, f'sha256sum {filename}',
+ hashsum)
+
+ # Check virtio-net via HTTP:
+ exec_command_and_wait_for_pattern(self, 'dhclient eth0', prompt)
+ self.check_http_download(filename, hashsum, self.GUEST_PORT)
+
+
+ # 5.3 kernel without RIL #
+
+ ASSET_KERNEL_F31 = Asset(
+ ('https://archives.fedoraproject.org/pub/archive/fedora/linux/'
+ 'releases/31/Server/aarch64/os/images/pxeboot/vmlinuz'),
+ '3ae07fcafbfc8e4abeb693035a74fe10698faae15e9ccd48882a9167800c1527')
+
+ ASSET_INITRD_F31 = Asset(
+ ('https://archives.fedoraproject.org/pub/archive/fedora/linux/'
+ 'releases/31/Server/aarch64/os/images/pxeboot/initrd.img'),
+ '9f3146b28bc531c689f3c5f114cb74e4bd7bd548e0ba19fa77921d8bd256755a')
+
+ ASSET_DISK_F31 = Asset(
+ ('https://archives.fedoraproject.org/pub/archive/fedora/linux/releases'
+ '/31/Cloud/aarch64/images/Fedora-Cloud-Base-31-1.9.aarch64.qcow2'),
+ '1e18d9c0cf734940c4b5d5ec592facaed2af0ad0329383d5639c997fdf16fe49')
+
+ F31_FILENAME = '/boot/initramfs-5.3.7-301.fc31.aarch64.img'
+ F31_HSUM = '1a4beec6607d94df73d9dd1b4985c9c23dd0fdcf4e6ca1351d477f190df7bef9'
+
+ def test_smmu_noril(self):
+ self.common_vm_setup(self.ASSET_KERNEL_F31, self.ASSET_INITRD_F31,
+ self.ASSET_DISK_F31)
+ self.kernel_params = self.default_kernel_params
+ self.run_and_check(self.F31_FILENAME, self.F31_HSUM)
+
+ def test_smmu_noril_passthrough(self):
+ self.common_vm_setup(self.ASSET_KERNEL_F31, self.ASSET_INITRD_F31,
+ self.ASSET_DISK_F31)
+ self.kernel_params = (self.default_kernel_params +
+ ' iommu.passthrough=on')
+ self.run_and_check(self.F31_FILENAME, self.F31_HSUM)
+
+ def test_smmu_noril_nostrict(self):
+ self.common_vm_setup(self.ASSET_KERNEL_F31, self.ASSET_INITRD_F31,
+ self.ASSET_DISK_F31)
+ self.kernel_params = (self.default_kernel_params +
+ ' iommu.strict=0')
+ self.run_and_check(self.F31_FILENAME, self.F31_HSUM)
+
+
+ # 5.8 kernel featuring range invalidation
+ # >= v5.7 kernel
+
+ ASSET_KERNEL_F33 = Asset(
+ ('https://archives.fedoraproject.org/pub/archive/fedora/linux/'
+ 'releases/33/Server/aarch64/os/images/pxeboot/vmlinuz'),
+ 'd8b1e6f7241f339d8e7609c456cf0461ffa4583ed07e0b55c7d1d8a0c154aa89')
+
+ ASSET_INITRD_F33 = Asset(
+ ('https://archives.fedoraproject.org/pub/archive/fedora/linux/'
+ 'releases/33/Server/aarch64/os/images/pxeboot/initrd.img'),
+ '92513f55295c2c16a777f7b6c35ccd70a438e9e1e40b6ba39e0e60900615b3df')
+
+ ASSET_DISK_F33 = Asset(
+ ('https://archives.fedoraproject.org/pub/archive/fedora/linux/releases'
+ '/33/Cloud/aarch64/images/Fedora-Cloud-Base-33-1.2.aarch64.qcow2'),
+ 'e7f75cdfd523fe5ac2ca9eeece68edc1a81f386a17f969c1d1c7c87031008a6b')
+
+ F33_FILENAME = '/boot/initramfs-5.8.15-301.fc33.aarch64.img'
+ F33_HSUM = '079cfad0caa82e84c8ca1fb0897a4999dd769f262216099f518619e807a550d9'
+
+ def test_smmu_ril(self):
+ self.common_vm_setup(self.ASSET_KERNEL_F33, self.ASSET_INITRD_F33,
+ self.ASSET_DISK_F33)
+ self.kernel_params = self.default_kernel_params
+ self.run_and_check(self.F33_FILENAME, self.F33_HSUM)
+
+ def test_smmu_ril_passthrough(self):
+ self.common_vm_setup(self.ASSET_KERNEL_F33, self.ASSET_INITRD_F33,
+ self.ASSET_DISK_F33)
+ self.kernel_params = (self.default_kernel_params +
+ ' iommu.passthrough=on')
+ self.run_and_check(self.F33_FILENAME, self.F33_HSUM)
+
+ def test_smmu_ril_nostrict(self):
+ self.common_vm_setup(self.ASSET_KERNEL_F33, self.ASSET_INITRD_F33,
+ self.ASSET_DISK_F33)
+ self.kernel_params = (self.default_kernel_params +
+ ' iommu.strict=0')
+ self.run_and_check(self.F33_FILENAME, self.F33_HSUM)
+
+
+if __name__ == '__main__':
+ LinuxKernelTest.main()
--
2.49.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* [PATCH 14/15] gitlab-ci: Update QEMU_JOB_AVOCADO and QEMU_CI_AVOCADO_TESTING
2025-03-25 20:00 [PATCH for-10.1 00/15] Convert remaining Avocado tests to functional Thomas Huth
` (12 preceding siblings ...)
2025-03-25 20:00 ` [PATCH 13/15] tests/functional: Convert the SMMU test to the functional framework Thomas Huth
@ 2025-03-25 20:00 ` Thomas Huth
2025-03-26 9:50 ` Daniel P. Berrangé
2025-03-25 20:00 ` [PATCH 15/15] Remove the remainders of the Avocado tests Thomas Huth
14 siblings, 1 reply; 34+ messages in thread
From: Thomas Huth @ 2025-03-25 20:00 UTC (permalink / raw)
To: qemu-devel; +Cc: Daniel P. Berrangé, Philippe Mathieu-Daudé
From: Thomas Huth <thuth@redhat.com>
Since we don't run the Avocado jobs in the CI anymore, rename
these variables to QEMU_JOB_FUNCTIONAL and QEMU_CI_FUNCTIONAL_TESTING.
Also, there was a mismatch between the documentation and the
implementation of QEMU_CI_AVOCADO_TESTING: While the documentation
said that you had to "Set this variable to have the tests using the
Avocado framework run automatically", you indeed needed to set it
to make the pipelines appear in your dashboard - but they were never
run automatically in forks and had to be triggered manually. Let's
improve this now: No need to hide these pipelines from the users
by default anymore (the functional tests should be stable enough
nowadays), and rather allow the users to run the pipelines auto-
matically with this switch now instead, as was documented.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
docs/devel/testing/ci-jobs.rst.inc | 19 +++++++++----------
.gitlab-ci.d/base.yml | 8 ++------
.gitlab-ci.d/buildtest-template.yml | 2 +-
3 files changed, 12 insertions(+), 17 deletions(-)
diff --git a/docs/devel/testing/ci-jobs.rst.inc b/docs/devel/testing/ci-jobs.rst.inc
index 3756bbe3554..4f4c96dd883 100644
--- a/docs/devel/testing/ci-jobs.rst.inc
+++ b/docs/devel/testing/ci-jobs.rst.inc
@@ -126,10 +126,10 @@ QEMU_JOB_PUBLISH
The job is for publishing content after a branch has been
merged into the upstream default branch.
-QEMU_JOB_AVOCADO
-~~~~~~~~~~~~~~~~
+QEMU_JOB_FUNCTIONAL
+~~~~~~~~~~~~~~~~~~~
-The job runs the Avocado integration test suite
+The job runs the functional test suite
Contributor controlled runtime variables
----------------------------------------
@@ -149,13 +149,12 @@ the jobs to be manually started from the UI
Set this variable to 2 to create the pipelines and run all
the jobs immediately, as was the historical behaviour
-QEMU_CI_AVOCADO_TESTING
-~~~~~~~~~~~~~~~~~~~~~~~
-By default, tests using the Avocado framework are not run automatically in
-the pipelines (because multiple artifacts have to be downloaded, and if
-these artifacts are not already cached, downloading them make the jobs
-reach the timeout limit). Set this variable to have the tests using the
-Avocado framework run automatically.
+QEMU_CI_FUNCTIONAL_TESTING
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+By default, tests using the functional framework are not run automatically
+in the pipelines (because multiple artifacts have to be downloaded, which
+might cause a lot of network traffic). Set this variable to have the tests
+using the functional framework run automatically.
Other misc variables
--------------------
diff --git a/.gitlab-ci.d/base.yml b/.gitlab-ci.d/base.yml
index 25b88aaa06a..172028e1731 100644
--- a/.gitlab-ci.d/base.yml
+++ b/.gitlab-ci.d/base.yml
@@ -69,10 +69,6 @@ variables:
- if: '$QEMU_CI != "1" && $QEMU_CI != "2" && $CI_PROJECT_NAMESPACE != $QEMU_CI_UPSTREAM'
when: never
- # Avocado jobs don't run in forks unless $QEMU_CI_AVOCADO_TESTING is set
- - if: '$QEMU_JOB_AVOCADO && $QEMU_CI_AVOCADO_TESTING != "1" && $CI_PROJECT_NAMESPACE != $QEMU_CI_UPSTREAM'
- when: never
-
#############################################################
# Stage 2: fine tune execution of jobs in specific scenarios
@@ -101,8 +97,8 @@ variables:
when: manual
allow_failure: true
- # Avocado jobs can be manually start in forks if $QEMU_CI_AVOCADO_TESTING is unset
- - if: '$QEMU_JOB_AVOCADO && $CI_PROJECT_NAMESPACE != $QEMU_CI_UPSTREAM'
+ # Functional jobs can be manually started in forks
+ - if: '$QEMU_JOB_FUNCTIONAL && $QEMU_CI_FUNCTIONAL_TESTING != "1" && $CI_PROJECT_NAMESPACE != $QEMU_CI_UPSTREAM'
when: manual
allow_failure: true
diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index 13fa4f4a4fc..d4f145fdb58 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -114,4 +114,4 @@
- cd build
- du -chs ${CI_PROJECT_DIR}/*-cache
variables:
- QEMU_JOB_AVOCADO: 1
+ QEMU_JOB_FUNCTIONAL: 1
--
2.49.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* Re: [PATCH 14/15] gitlab-ci: Update QEMU_JOB_AVOCADO and QEMU_CI_AVOCADO_TESTING
2025-03-25 20:00 ` [PATCH 14/15] gitlab-ci: Update QEMU_JOB_AVOCADO and QEMU_CI_AVOCADO_TESTING Thomas Huth
@ 2025-03-26 9:50 ` Daniel P. Berrangé
2025-04-14 7:52 ` Thomas Huth
0 siblings, 1 reply; 34+ messages in thread
From: Daniel P. Berrangé @ 2025-03-26 9:50 UTC (permalink / raw)
To: Thomas Huth; +Cc: qemu-devel, Philippe Mathieu-Daudé
On Tue, Mar 25, 2025 at 09:00:22PM +0100, Thomas Huth wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> Since we don't run the Avocado jobs in the CI anymore, rename
> these variables to QEMU_JOB_FUNCTIONAL and QEMU_CI_FUNCTIONAL_TESTING.
>
> Also, there was a mismatch between the documentation and the
> implementation of QEMU_CI_AVOCADO_TESTING: While the documentation
> said that you had to "Set this variable to have the tests using the
> Avocado framework run automatically", you indeed needed to set it
> to make the pipelines appear in your dashboard - but they were never
> run automatically in forks and had to be triggered manually. Let's
> improve this now: No need to hide these pipelines from the users
> by default anymore (the functional tests should be stable enough
> nowadays), and rather allow the users to run the pipelines auto-
> matically with this switch now instead, as was documented.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> docs/devel/testing/ci-jobs.rst.inc | 19 +++++++++----------
> .gitlab-ci.d/base.yml | 8 ++------
> .gitlab-ci.d/buildtest-template.yml | 2 +-
> 3 files changed, 12 insertions(+), 17 deletions(-)
>
> diff --git a/docs/devel/testing/ci-jobs.rst.inc b/docs/devel/testing/ci-jobs.rst.inc
> index 3756bbe3554..4f4c96dd883 100644
> --- a/docs/devel/testing/ci-jobs.rst.inc
> +++ b/docs/devel/testing/ci-jobs.rst.inc
> @@ -126,10 +126,10 @@ QEMU_JOB_PUBLISH
> The job is for publishing content after a branch has been
> merged into the upstream default branch.
>
> -QEMU_JOB_AVOCADO
> -~~~~~~~~~~~~~~~~
> +QEMU_JOB_FUNCTIONAL
> +~~~~~~~~~~~~~~~~~~~
>
> -The job runs the Avocado integration test suite
> +The job runs the functional test suite
>
> Contributor controlled runtime variables
> ----------------------------------------
> @@ -149,13 +149,12 @@ the jobs to be manually started from the UI
> Set this variable to 2 to create the pipelines and run all
> the jobs immediately, as was the historical behaviour
>
> -QEMU_CI_AVOCADO_TESTING
> -~~~~~~~~~~~~~~~~~~~~~~~
> -By default, tests using the Avocado framework are not run automatically in
> -the pipelines (because multiple artifacts have to be downloaded, and if
> -these artifacts are not already cached, downloading them make the jobs
> -reach the timeout limit). Set this variable to have the tests using the
> -Avocado framework run automatically.
> +QEMU_CI_FUNCTIONAL_TESTING
Both old and new terms are quite verbose, how about
dropping '_TESTING' suffix ?
With, or without that rename though:
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH 14/15] gitlab-ci: Update QEMU_JOB_AVOCADO and QEMU_CI_AVOCADO_TESTING
2025-03-26 9:50 ` Daniel P. Berrangé
@ 2025-04-14 7:52 ` Thomas Huth
0 siblings, 0 replies; 34+ messages in thread
From: Thomas Huth @ 2025-04-14 7:52 UTC (permalink / raw)
To: Daniel P. Berrangé; +Cc: qemu-devel, Philippe Mathieu-Daudé
On 26/03/2025 10.50, Daniel P. Berrangé wrote:
> On Tue, Mar 25, 2025 at 09:00:22PM +0100, Thomas Huth wrote:
>> From: Thomas Huth <thuth@redhat.com>
>>
>> Since we don't run the Avocado jobs in the CI anymore, rename
>> these variables to QEMU_JOB_FUNCTIONAL and QEMU_CI_FUNCTIONAL_TESTING.
...
>> -QEMU_CI_AVOCADO_TESTING
>> -~~~~~~~~~~~~~~~~~~~~~~~
>> -By default, tests using the Avocado framework are not run automatically in
>> -the pipelines (because multiple artifacts have to be downloaded, and if
>> -these artifacts are not already cached, downloading them make the jobs
>> -reach the timeout limit). Set this variable to have the tests using the
>> -Avocado framework run automatically.
>> +QEMU_CI_FUNCTIONAL_TESTING
>
> Both old and new terms are quite verbose, how about
> dropping '_TESTING' suffix ?
Sure, I'll drop it in v2!
> With, or without that rename though:
>
> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Thanks!
Thomas
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH 15/15] Remove the remainders of the Avocado tests
2025-03-25 20:00 [PATCH for-10.1 00/15] Convert remaining Avocado tests to functional Thomas Huth
` (13 preceding siblings ...)
2025-03-25 20:00 ` [PATCH 14/15] gitlab-ci: Update QEMU_JOB_AVOCADO and QEMU_CI_AVOCADO_TESTING Thomas Huth
@ 2025-03-25 20:00 ` Thomas Huth
2025-03-25 21:03 ` Philippe Mathieu-Daudé
2025-03-26 9:51 ` Daniel P. Berrangé
14 siblings, 2 replies; 34+ messages in thread
From: Thomas Huth @ 2025-03-25 20:00 UTC (permalink / raw)
To: qemu-devel; +Cc: Daniel P. Berrangé, Philippe Mathieu-Daudé
From: Thomas Huth <thuth@redhat.com>
Now that all Avocado tests have been converted to or been replaced by
other functional tests, we can delete the remainders of the Avocado
tests from the QEMU source tree.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 8 +-
docs/about/build-platforms.rst | 10 +-
docs/devel/build-system.rst | 11 +-
docs/devel/codebase.rst | 5 -
docs/devel/testing/avocado.rst | 581 ------------------------
docs/devel/testing/functional.rst | 3 -
docs/devel/testing/index.rst | 1 -
docs/devel/testing/main.rst | 15 -
tests/avocado/README.rst | 10 -
configure | 2 +-
pythondeps.toml | 8 +-
tests/Makefile.include | 60 +--
tests/avocado/avocado_qemu/__init__.py | 424 -----------------
tests/avocado/avocado_qemu/linuxtest.py | 253 -----------
14 files changed, 16 insertions(+), 1375 deletions(-)
delete mode 100644 docs/devel/testing/avocado.rst
delete mode 100644 tests/avocado/README.rst
delete mode 100644 tests/avocado/avocado_qemu/__init__.py
delete mode 100644 tests/avocado/avocado_qemu/linuxtest.py
diff --git a/MAINTAINERS b/MAINTAINERS
index 9bd9cb836ce..56408133cd6 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2072,7 +2072,7 @@ S: Supported
F: hw/acpi/viot.c
F: hw/acpi/viot.h
-ACPI/AVOCADO/BIOSBITS
+ACPI/FUNCTIONAL/BIOSBITS
M: Ani Sinha <anisinha@redhat.com>
M: Michael S. Tsirkin <mst@redhat.com>
S: Supported
@@ -4245,12 +4245,6 @@ R: Philippe Mathieu-Daudé <philmd@linaro.org>
S: Maintained
F: tests/tcg/Makefile.target
-Integration Testing with the Avocado framework
-W: https://trello.com/b/6Qi1pxVn/avocado-qemu
-R: Cleber Rosa <crosa@redhat.com>
-S: Odd Fixes
-F: tests/avocado/
-
GitLab custom runner (Works On Arm Sponsored)
M: Alex Bennée <alex.bennee@linaro.org>
M: Philippe Mathieu-Daudé <philmd@linaro.org>
diff --git a/docs/about/build-platforms.rst b/docs/about/build-platforms.rst
index 1552b1a7044..52521552c8a 100644
--- a/docs/about/build-platforms.rst
+++ b/docs/about/build-platforms.rst
@@ -123,11 +123,11 @@ Rust build dependencies
to build QEMU.
Optional build dependencies
- Build components whose absence does not affect the ability to build
- QEMU may not be available in distros, or may be too old for QEMU's
- requirements. Many of these, such as the Avocado testing framework
- or various linters, are written in Python and therefore can also
- be installed using ``pip``. Cross compilers are another example
+ Build components whose absence does not affect the ability to build QEMU
+ may not be available in distros, or may be too old for our requirements.
+ Many of these, such as additional modules for the functional testing
+ framework or various linters, are written in Python and therefore can
+ also be installed using ``pip``. Cross compilers are another example
of optional build-time dependency; in this case it is possible to
download them from repositories such as EPEL, to use container-based
cross compilation using ``docker`` or ``podman``, or to use pre-built
diff --git a/docs/devel/build-system.rst b/docs/devel/build-system.rst
index a759982f45c..258cfad3fe7 100644
--- a/docs/devel/build-system.rst
+++ b/docs/devel/build-system.rst
@@ -134,7 +134,7 @@ in how the build process runs Python code.
At this stage, ``configure`` also queries the chosen Python interpreter
about QEMU's build dependencies. Note that the build process does *not*
-look for ``meson``, ``sphinx-build`` or ``avocado`` binaries in the PATH;
+look for ``meson`` or ``sphinx-build`` binaries in the PATH;
likewise, there are no options such as ``--meson`` or ``--sphinx-build``.
This avoids a potential mismatch, where Meson and Sphinx binaries on the
PATH might operate in a different Python environment than the one chosen
@@ -151,7 +151,7 @@ virtual environment with ``pip``, either from wheels in ``python/wheels``
or by downloading the package with PyPI. Downloading can be disabled with
``--disable-download``; and anyway, it only happens when a ``configure``
option (currently, only ``--enable-docs``) is explicitly enabled but
-the dependencies are not present\ [#pip]_.
+the dependencies are not present.
.. [#distlib] The scripts are created based on the package's metadata,
specifically the ``console_script`` entry points. This is the
@@ -164,10 +164,6 @@ the dependencies are not present\ [#pip]_.
because the Python Packaging Authority provides a package
``distlib.scripts`` to perform this task.
-.. [#pip] ``pip`` might also be used when running ``make check-avocado``
- if downloading is enabled, to ensure that Avocado is
- available.
-
The required versions of the packages are stored in a configuration file
``pythondeps.toml``. The format is custom to QEMU, but it is documented
at the top of the file itself and it should be easy to understand. The
@@ -497,8 +493,7 @@ number of dynamically created files listed later.
``pyvenv/bin``, and calling ``pip`` to install dependencies.
``tests/Makefile.include``
- Rules for external test harnesses. These include the TCG tests
- and the Avocado-based integration tests.
+ Rules for external test harnesses like the TCG tests.
``tests/docker/Makefile.include``
Rules for Docker tests. Like ``tests/Makefile.include``, this file is
diff --git a/docs/devel/codebase.rst b/docs/devel/codebase.rst
index 1b09953197b..1da17f40494 100644
--- a/docs/devel/codebase.rst
+++ b/docs/devel/codebase.rst
@@ -175,11 +175,6 @@ yet, so sometimes the source code is all you have.
* `tests <https://gitlab.com/qemu-project/qemu/-/tree/master/tests>`_:
QEMU `test <testing>` suite
- - `avocado <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/avocado>`_:
- Functional tests booting full VM using `Avocado framework <checkavocado-ref>`.
- Those tests will be transformed and moved into
- `tests/functional <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/functional>`_
- in the future.
- `data <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/data>`_:
Data for various tests.
- `decode <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/decode>`_:
diff --git a/docs/devel/testing/avocado.rst b/docs/devel/testing/avocado.rst
deleted file mode 100644
index eda76fe2dbd..00000000000
--- a/docs/devel/testing/avocado.rst
+++ /dev/null
@@ -1,581 +0,0 @@
-.. _checkavocado-ref:
-
-
-Integration testing with Avocado
-================================
-
-The ``tests/avocado`` directory hosts integration tests. They're usually
-higher level tests, and may interact with external resources and with
-various guest operating systems.
-
-These tests are written using the Avocado Testing Framework (which must be
-installed separately) in conjunction with a the ``avocado_qemu.QemuSystemTest``
-class, implemented at ``tests/avocado/avocado_qemu``.
-
-Tests based on ``avocado_qemu.QemuSystemTest`` can easily:
-
- * Customize the command line arguments given to the convenience
- ``self.vm`` attribute (a QEMUMachine instance)
-
- * Interact with the QEMU monitor, send QMP commands and check
- their results
-
- * Interact with the guest OS, using the convenience console device
- (which may be useful to assert the effectiveness and correctness of
- command line arguments or QMP commands)
-
- * Interact with external data files that accompany the test itself
- (see ``self.get_data()``)
-
- * Download (and cache) remote data files, such as firmware and kernel
- images
-
- * Have access to a library of guest OS images (by means of the
- ``avocado.utils.vmimage`` library)
-
- * Make use of various other test related utilities available at the
- test class itself and at the utility library:
-
- - http://avocado-framework.readthedocs.io/en/latest/api/test/avocado.html#avocado.Test
- - http://avocado-framework.readthedocs.io/en/latest/api/utils/avocado.utils.html
-
-Running tests
--------------
-
-You can run the avocado tests simply by executing:
-
-.. code::
-
- make check-avocado
-
-This involves the automatic installation, from PyPI, of all the
-necessary avocado-framework dependencies into the QEMU venv within the
-build tree (at ``./pyvenv``). Test results are also saved within the
-build tree (at ``tests/results``).
-
-Note: the build environment must be using a Python 3 stack, and have
-the ``venv`` and ``pip`` packages installed. If necessary, make sure
-``configure`` is called with ``--python=`` and that those modules are
-available. On Debian and Ubuntu based systems, depending on the
-specific version, they may be on packages named ``python3-venv`` and
-``python3-pip``.
-
-It is also possible to run tests based on tags using the
-``make check-avocado`` command and the ``AVOCADO_TAGS`` environment
-variable:
-
-.. code::
-
- make check-avocado AVOCADO_TAGS=quick
-
-Note that tags separated with commas have an AND behavior, while tags
-separated by spaces have an OR behavior. For more information on Avocado
-tags, see:
-
- https://avocado-framework.readthedocs.io/en/latest/guides/user/chapters/tags.html
-
-To run a single test file, a couple of them, or a test within a file
-using the ``make check-avocado`` command, set the ``AVOCADO_TESTS``
-environment variable with the test files or test names. To run all
-tests from a single file, use:
-
- .. code::
-
- make check-avocado AVOCADO_TESTS=$FILEPATH
-
-The same is valid to run tests from multiple test files:
-
- .. code::
-
- make check-avocado AVOCADO_TESTS='$FILEPATH1 $FILEPATH2'
-
-To run a single test within a file, use:
-
- .. code::
-
- make check-avocado AVOCADO_TESTS=$FILEPATH:$TESTCLASS.$TESTNAME
-
-The same is valid to run single tests from multiple test files:
-
- .. code::
-
- make check-avocado AVOCADO_TESTS='$FILEPATH1:$TESTCLASS1.$TESTNAME1 $FILEPATH2:$TESTCLASS2.$TESTNAME2'
-
-The scripts installed inside the virtual environment may be used
-without an "activation". For instance, the Avocado test runner
-may be invoked by running:
-
- .. code::
-
- pyvenv/bin/avocado run $OPTION1 $OPTION2 tests/avocado/
-
-Note that if ``make check-avocado`` was not executed before, it is
-possible to create the Python virtual environment with the dependencies
-needed running:
-
- .. code::
-
- make check-venv
-
-It is also possible to run tests from a single file or a single test within
-a test file. To run tests from a single file within the build tree, use:
-
- .. code::
-
- pyvenv/bin/avocado run tests/avocado/$TESTFILE
-
-To run a single test within a test file, use:
-
- .. code::
-
- pyvenv/bin/avocado run tests/avocado/$TESTFILE:$TESTCLASS.$TESTNAME
-
-Valid test names are visible in the output from any previous execution
-of Avocado or ``make check-avocado``, and can also be queried using:
-
- .. code::
-
- pyvenv/bin/avocado list tests/avocado
-
-Manual Installation
--------------------
-
-To manually install Avocado and its dependencies, run:
-
-.. code::
-
- pip install --user avocado-framework
-
-Alternatively, follow the instructions on this link:
-
- https://avocado-framework.readthedocs.io/en/latest/guides/user/chapters/installing.html
-
-Overview
---------
-
-The ``tests/avocado/avocado_qemu`` directory provides the
-``avocado_qemu`` Python module, containing the ``avocado_qemu.QemuSystemTest``
-class. Here's a simple usage example:
-
-.. code::
-
- from avocado_qemu import QemuSystemTest
-
-
- class Version(QemuSystemTest):
- """
- :avocado: tags=quick
- """
- def test_qmp_human_info_version(self):
- self.vm.launch()
- res = self.vm.cmd('human-monitor-command',
- command_line='info version')
- self.assertRegex(res, r'^(\d+\.\d+\.\d)')
-
-To execute your test, run:
-
-.. code::
-
- avocado run version.py
-
-Tests may be classified according to a convention by using docstring
-directives such as ``:avocado: tags=TAG1,TAG2``. To run all tests
-in the current directory, tagged as "quick", run:
-
-.. code::
-
- avocado run -t quick .
-
-The ``avocado_qemu.QemuSystemTest`` base test class
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-The ``avocado_qemu.QemuSystemTest`` class has a number of characteristics
-that are worth being mentioned right away.
-
-First of all, it attempts to give each test a ready to use QEMUMachine
-instance, available at ``self.vm``. Because many tests will tweak the
-QEMU command line, launching the QEMUMachine (by using ``self.vm.launch()``)
-is left to the test writer.
-
-The base test class has also support for tests with more than one
-QEMUMachine. The way to get machines is through the ``self.get_vm()``
-method which will return a QEMUMachine instance. The ``self.get_vm()``
-method accepts arguments that will be passed to the QEMUMachine creation
-and also an optional ``name`` attribute so you can identify a specific
-machine and get it more than once through the tests methods. A simple
-and hypothetical example follows:
-
-.. code::
-
- from avocado_qemu import QemuSystemTest
-
-
- class MultipleMachines(QemuSystemTest):
- def test_multiple_machines(self):
- first_machine = self.get_vm()
- second_machine = self.get_vm()
- self.get_vm(name='third_machine').launch()
-
- first_machine.launch()
- second_machine.launch()
-
- first_res = first_machine.cmd(
- 'human-monitor-command',
- command_line='info version')
-
- second_res = second_machine.cmd(
- 'human-monitor-command',
- command_line='info version')
-
- third_res = self.get_vm(name='third_machine').cmd(
- 'human-monitor-command',
- command_line='info version')
-
- self.assertEqual(first_res, second_res, third_res)
-
-At test "tear down", ``avocado_qemu.QemuSystemTest`` handles all the
-QEMUMachines shutdown.
-
-The ``avocado_qemu.LinuxTest`` base test class
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-The ``avocado_qemu.LinuxTest`` is further specialization of the
-``avocado_qemu.QemuSystemTest`` class, so it contains all the characteristics
-of the later plus some extra features.
-
-First of all, this base class is intended for tests that need to
-interact with a fully booted and operational Linux guest. At this
-time, it uses a Fedora 31 guest image. The most basic example looks
-like this:
-
-.. code::
-
- from avocado_qemu import LinuxTest
-
-
- class SomeTest(LinuxTest):
-
- def test(self):
- self.launch_and_wait()
- self.ssh_command('some_command_to_be_run_in_the_guest')
-
-Please refer to tests that use ``avocado_qemu.LinuxTest`` under
-``tests/avocado`` for more examples.
-
-QEMUMachine
------------
-
-The QEMUMachine API is already widely used in the Python iotests,
-device-crash-test and other Python scripts. It's a wrapper around the
-execution of a QEMU binary, giving its users:
-
- * the ability to set command line arguments to be given to the QEMU
- binary
-
- * a ready to use QMP connection and interface, which can be used to
- send commands and inspect its results, as well as asynchronous
- events
-
- * convenience methods to set commonly used command line arguments in
- a more succinct and intuitive way
-
-QEMU binary selection
-^^^^^^^^^^^^^^^^^^^^^
-
-The QEMU binary used for the ``self.vm`` QEMUMachine instance will
-primarily depend on the value of the ``qemu_bin`` parameter. If it's
-not explicitly set, its default value will be the result of a dynamic
-probe in the same source tree. A suitable binary will be one that
-targets the architecture matching host machine.
-
-Based on this description, test writers will usually rely on one of
-the following approaches:
-
-1) Set ``qemu_bin``, and use the given binary
-
-2) Do not set ``qemu_bin``, and use a QEMU binary named like
- "qemu-system-${arch}", either in the current
- working directory, or in the current source tree.
-
-The resulting ``qemu_bin`` value will be preserved in the
-``avocado_qemu.QemuSystemTest`` as an attribute with the same name.
-
-Attribute reference
--------------------
-
-Test
-^^^^
-
-Besides the attributes and methods that are part of the base
-``avocado.Test`` class, the following attributes are available on any
-``avocado_qemu.QemuSystemTest`` instance.
-
-vm
-""
-
-A QEMUMachine instance, initially configured according to the given
-``qemu_bin`` parameter.
-
-arch
-""""
-
-The architecture can be used on different levels of the stack, e.g. by
-the framework or by the test itself. At the framework level, it will
-currently influence the selection of a QEMU binary (when one is not
-explicitly given).
-
-Tests are also free to use this attribute value, for their own needs.
-A test may, for instance, use the same value when selecting the
-architecture of a kernel or disk image to boot a VM with.
-
-The ``arch`` attribute will be set to the test parameter of the same
-name. If one is not given explicitly, it will either be set to
-``None``, or, if the test is tagged with one (and only one)
-``:avocado: tags=arch:VALUE`` tag, it will be set to ``VALUE``.
-
-cpu
-"""
-
-The cpu model that will be set to all QEMUMachine instances created
-by the test.
-
-The ``cpu`` attribute will be set to the test parameter of the same
-name. If one is not given explicitly, it will either be set to
-``None ``, or, if the test is tagged with one (and only one)
-``:avocado: tags=cpu:VALUE`` tag, it will be set to ``VALUE``.
-
-machine
-"""""""
-
-The machine type that will be set to all QEMUMachine instances created
-by the test.
-
-The ``machine`` attribute will be set to the test parameter of the same
-name. If one is not given explicitly, it will either be set to
-``None``, or, if the test is tagged with one (and only one)
-``:avocado: tags=machine:VALUE`` tag, it will be set to ``VALUE``.
-
-qemu_bin
-""""""""
-
-The preserved value of the ``qemu_bin`` parameter or the result of the
-dynamic probe for a QEMU binary in the current working directory or
-source tree.
-
-LinuxTest
-^^^^^^^^^
-
-Besides the attributes present on the ``avocado_qemu.QemuSystemTest`` base
-class, the ``avocado_qemu.LinuxTest`` adds the following attributes:
-
-distro
-""""""
-
-The name of the Linux distribution used as the guest image for the
-test. The name should match the **Provider** column on the list
-of images supported by the avocado.utils.vmimage library:
-
-https://avocado-framework.readthedocs.io/en/latest/guides/writer/libs/vmimage.html#supported-images
-
-distro_version
-""""""""""""""
-
-The version of the Linux distribution as the guest image for the
-test. The name should match the **Version** column on the list
-of images supported by the avocado.utils.vmimage library:
-
-https://avocado-framework.readthedocs.io/en/latest/guides/writer/libs/vmimage.html#supported-images
-
-distro_checksum
-"""""""""""""""
-
-The sha256 hash of the guest image file used for the test.
-
-If this value is not set in the code or by a test parameter (with the
-same name), no validation on the integrity of the image will be
-performed.
-
-Parameter reference
--------------------
-
-To understand how Avocado parameters are accessed by tests, and how
-they can be passed to tests, please refer to::
-
- https://avocado-framework.readthedocs.io/en/latest/guides/writer/chapters/writing.html#accessing-test-parameters
-
-Parameter values can be easily seen in the log files, and will look
-like the following:
-
-.. code::
-
- PARAMS (key=qemu_bin, path=*, default=./qemu-system-x86_64) => './qemu-system-x86_64
-
-Test
-^^^^
-
-arch
-""""
-
-The architecture that will influence the selection of a QEMU binary
-(when one is not explicitly given).
-
-Tests are also free to use this parameter value, for their own needs.
-A test may, for instance, use the same value when selecting the
-architecture of a kernel or disk image to boot a VM with.
-
-This parameter has a direct relation with the ``arch`` attribute. If
-not given, it will default to None.
-
-cpu
-"""
-
-The cpu model that will be set to all QEMUMachine instances created
-by the test.
-
-machine
-"""""""
-
-The machine type that will be set to all QEMUMachine instances created
-by the test.
-
-qemu_bin
-""""""""
-
-The exact QEMU binary to be used on QEMUMachine.
-
-LinuxTest
-^^^^^^^^^
-
-Besides the parameters present on the ``avocado_qemu.QemuSystemTest`` base
-class, the ``avocado_qemu.LinuxTest`` adds the following parameters:
-
-distro
-""""""
-
-The name of the Linux distribution used as the guest image for the
-test. The name should match the **Provider** column on the list
-of images supported by the avocado.utils.vmimage library:
-
-https://avocado-framework.readthedocs.io/en/latest/guides/writer/libs/vmimage.html#supported-images
-
-distro_version
-""""""""""""""
-
-The version of the Linux distribution as the guest image for the
-test. The name should match the **Version** column on the list
-of images supported by the avocado.utils.vmimage library:
-
-https://avocado-framework.readthedocs.io/en/latest/guides/writer/libs/vmimage.html#supported-images
-
-distro_checksum
-"""""""""""""""
-
-The sha256 hash of the guest image file used for the test.
-
-If this value is not set in the code or by this parameter no
-validation on the integrity of the image will be performed.
-
-Skipping tests
---------------
-
-The Avocado framework provides Python decorators which allow for easily skip
-tests running under certain conditions. For example, on the lack of a binary
-on the test system or when the running environment is a CI system. For further
-information about those decorators, please refer to::
-
- https://avocado-framework.readthedocs.io/en/latest/guides/writer/chapters/writing.html#skipping-tests
-
-While the conditions for skipping tests are often specifics of each one, there
-are recurring scenarios identified by the QEMU developers and the use of
-environment variables became a kind of standard way to enable/disable tests.
-
-Here is a list of the most used variables:
-
-AVOCADO_ALLOW_LARGE_STORAGE
-^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Tests which are going to fetch or produce assets considered *large* are not
-going to run unless that ``AVOCADO_ALLOW_LARGE_STORAGE=1`` is exported on
-the environment.
-
-The definition of *large* is a bit arbitrary here, but it usually means an
-asset which occupies at least 1GB of size on disk when uncompressed.
-
-SPEED
-^^^^^
-Tests which have a long runtime will not be run unless ``SPEED=slow`` is
-exported on the environment.
-
-The definition of *long* is a bit arbitrary here, and it depends on the
-usefulness of the test too. A unique test is worth spending more time on,
-small variations on existing tests perhaps less so. As a rough guide,
-a test or set of similar tests which take more than 100 seconds to
-complete.
-
-AVOCADO_ALLOW_UNTRUSTED_CODE
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-There are tests which will boot a kernel image or firmware that can be
-considered not safe to run on the developer's workstation, thus they are
-skipped by default. The definition of *not safe* is also arbitrary but
-usually it means a blob which either its source or build process aren't
-public available.
-
-You should export ``AVOCADO_ALLOW_UNTRUSTED_CODE=1`` on the environment in
-order to allow tests which make use of those kind of assets.
-
-AVOCADO_TIMEOUT_EXPECTED
-^^^^^^^^^^^^^^^^^^^^^^^^
-The Avocado framework has a timeout mechanism which interrupts tests to avoid the
-test suite of getting stuck. The timeout value can be set via test parameter or
-property defined in the test class, for further details::
-
- https://avocado-framework.readthedocs.io/en/latest/guides/writer/chapters/writing.html#setting-a-test-timeout
-
-Even though the timeout can be set by the test developer, there are some tests
-that may not have a well-defined limit of time to finish under certain
-conditions. For example, tests that take longer to execute when QEMU is
-compiled with debug flags. Therefore, the ``AVOCADO_TIMEOUT_EXPECTED`` variable
-has been used to determine whether those tests should run or not.
-
-QEMU_TEST_FLAKY_TESTS
-^^^^^^^^^^^^^^^^^^^^^
-Some tests are not working reliably and thus are disabled by default.
-This includes tests that don't run reliably on GitLab's CI which
-usually expose real issues that are rarely seen on developer machines
-due to the constraints of the CI environment. If you encounter a
-similar situation then raise a bug and then mark the test as shown on
-the code snippet below:
-
-.. code::
-
- # See https://gitlab.com/qemu-project/qemu/-/issues/nnnn
- @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab')
- def test(self):
- do_something()
-
-You can also add ``:avocado: tags=flaky`` to the test meta-data so
-only the flaky tests can be run as a group:
-
-.. code::
-
- env QEMU_TEST_FLAKY_TESTS=1 ./pyvenv/bin/avocado \
- run tests/avocado -filter-by-tags=flaky
-
-Tests should not live in this state forever and should either be fixed
-or eventually removed.
-
-
-Uninstalling Avocado
---------------------
-
-If you've followed the manual installation instructions above, you can
-easily uninstall Avocado. Start by listing the packages you have
-installed::
-
- pip list --user
-
-And remove any package you want with::
-
- pip uninstall <package_name>
-
-If you've used ``make check-avocado``, the Python virtual environment where
-Avocado is installed will be cleaned up as part of ``make check-clean``.
diff --git a/docs/devel/testing/functional.rst b/docs/devel/testing/functional.rst
index 9bc973392aa..8030cb4299f 100644
--- a/docs/devel/testing/functional.rst
+++ b/docs/devel/testing/functional.rst
@@ -6,9 +6,6 @@ Functional testing with Python
The ``tests/functional`` directory hosts functional tests written in
Python. They are usually higher level tests, and may interact with
external resources and with various guest operating systems.
-The functional tests have initially evolved from the Avocado tests, so there
-is a lot of similarity to those tests here (see :ref:`checkavocado-ref` for
-details about the Avocado tests).
The tests should be written in the style of the Python `unittest`_ framework,
using stdio for the TAP protocol. The folder ``tests/functional/qemu_test``
diff --git a/docs/devel/testing/index.rst b/docs/devel/testing/index.rst
index 1171f7db8f0..ccc2fc6cbc1 100644
--- a/docs/devel/testing/index.rst
+++ b/docs/devel/testing/index.rst
@@ -10,7 +10,6 @@ testing infrastructure.
main
qtest
functional
- avocado
acpi-bits
ci
fuzzing
diff --git a/docs/devel/testing/main.rst b/docs/devel/testing/main.rst
index 9869bcf0341..3c03d769539 100644
--- a/docs/devel/testing/main.rst
+++ b/docs/devel/testing/main.rst
@@ -894,21 +894,6 @@ Python. You can run the functional tests simply by executing:
See :ref:`checkfunctional-ref` for more details.
-Integration tests using the Avocado Framework
----------------------------------------------
-
-The ``tests/avocado`` directory hosts integration tests. They're usually
-higher level tests, and may interact with external resources and with
-various guest operating systems.
-
-You can run the avocado tests simply by executing:
-
-.. code::
-
- make check-avocado
-
-See :ref:`checkavocado-ref` for more details.
-
.. _checktcg-ref:
Testing with "make check-tcg"
diff --git a/tests/avocado/README.rst b/tests/avocado/README.rst
deleted file mode 100644
index 94488371bbe..00000000000
--- a/tests/avocado/README.rst
+++ /dev/null
@@ -1,10 +0,0 @@
-=============================================
-Integration tests using the Avocado Framework
-=============================================
-
-This directory contains integration tests. They're usually higher
-level, and may interact with external resources and with various
-guest operating systems.
-
-For more information, please refer to ``docs/devel/testing.rst``,
-section "Integration tests using the Avocado Framework".
diff --git a/configure b/configure
index 02f1dd2311f..000309cf610 100755
--- a/configure
+++ b/configure
@@ -1685,7 +1685,7 @@ LINKS="$LINKS pc-bios/optionrom/Makefile"
LINKS="$LINKS pc-bios/s390-ccw/Makefile"
LINKS="$LINKS pc-bios/vof/Makefile"
LINKS="$LINKS .gdbinit scripts" # scripts needed by relative path in .gdbinit
-LINKS="$LINKS tests/avocado tests/data"
+LINKS="$LINKS tests/data"
LINKS="$LINKS tests/qemu-iotests/check tests/qemu-iotests/Makefile"
LINKS="$LINKS python"
for f in $LINKS ; do
diff --git a/pythondeps.toml b/pythondeps.toml
index c03c9df81b5..7eaaa0fed10 100644
--- a/pythondeps.toml
+++ b/pythondeps.toml
@@ -27,9 +27,5 @@ pycotap = { accepted = ">=1.1.0", installed = "1.3.1" }
sphinx = { accepted = ">=3.4.3", installed = "5.3.0", canary = "sphinx-build" }
sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.1.1" }
-[avocado]
-# Note that qemu.git/python/ is always implicitly installed.
-# Prefer an LTS version when updating the accepted versions of
-# avocado-framework, for example right now the limit is 92.x.
-avocado-framework = { accepted = "(>=103.0, <104.0)", installed = "103.0", canary = "avocado" }
-pycdlib = { accepted = ">=1.11.0" }
+[testdeps]
+qemu.qmp = { accepted = ">=0.0.3", installed = "0.0.3" }
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 010369bd3ac..23fb722d426 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -18,7 +18,6 @@ ifneq ($(filter $(all-check-targets), check-softfloat),)
@echo " $(MAKE) check-tcg Run TCG tests"
@echo " $(MAKE) check-softfloat Run FPU emulation tests"
endif
- @echo " $(MAKE) check-avocado Run avocado (integration) tests for currently configured targets"
@echo
@echo " $(MAKE) check-report.junit.xml Generates an aggregated XML test report"
@echo " $(MAKE) check-venv Creates a Python venv for tests"
@@ -26,7 +25,6 @@ endif
@echo
@echo "The following are useful for CI builds"
@echo " $(MAKE) check-build Build most test binaries"
- @echo " $(MAKE) get-vm-images Downloads all images used by avocado tests, according to configured targets (~350 MB each, 1.5 GB max)"
@echo
@echo
@echo "The variable SPEED can be set to control the gtester speed setting."
@@ -86,26 +84,12 @@ distclean-tcg: $(DISTCLEAN_TCG_TARGET_RULES)
# Python venv for running tests
-.PHONY: check-venv check-avocado check-acceptance check-acceptance-deprecated-warning
+.PHONY: check-venv
# Build up our target list from the filtered list of ninja targets
TARGETS=$(patsubst libqemu-%.a, %, $(filter libqemu-%.a, $(ninja-targets)))
TESTS_VENV_TOKEN=$(BUILD_DIR)/pyvenv/tests.group
-TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
-ifndef AVOCADO_TESTS
- AVOCADO_TESTS=tests/avocado
-endif
-# Controls the output generated by Avocado when running tests.
-# Any number of command separated loggers are accepted. For more
-# information please refer to "avocado --help".
-AVOCADO_SHOW?=app
-ifndef AVOCADO_TAGS
- AVOCADO_CMDLINE_TAGS=$(patsubst %-softmmu,-t arch:%, \
- $(filter %-softmmu,$(TARGETS)))
-else
- AVOCADO_CMDLINE_TAGS=$(addprefix -t , $(AVOCADO_TAGS))
-endif
quiet-venv-pip = $(quiet-@)$(call quiet-command-run, \
$(PYTHON) -m pip -q --disable-pip-version-check $1, \
@@ -113,47 +97,11 @@ quiet-venv-pip = $(quiet-@)$(call quiet-command-run, \
$(TESTS_VENV_TOKEN): $(SRC_PATH)/pythondeps.toml
$(call quiet-venv-pip,install -e "$(SRC_PATH)/python/")
- $(MKVENV_ENSUREGROUP) $< avocado
+ $(MKVENV_ENSUREGROUP) $< testdeps
$(call quiet-command, touch $@)
-$(TESTS_RESULTS_DIR):
- $(call quiet-command, mkdir -p $@, \
- MKDIR, $@)
-
check-venv: $(TESTS_VENV_TOKEN)
-FEDORA_31_ARCHES_TARGETS=$(patsubst %-softmmu,%, $(filter %-softmmu,$(TARGETS)))
-FEDORA_31_ARCHES_CANDIDATES=$(patsubst ppc64,ppc64le,$(FEDORA_31_ARCHES_TARGETS))
-FEDORA_31_ARCHES := x86_64 aarch64 ppc64le s390x
-FEDORA_31_DOWNLOAD=$(filter $(FEDORA_31_ARCHES),$(FEDORA_31_ARCHES_CANDIDATES))
-
-# download one specific Fedora 31 image
-get-vm-image-fedora-31-%: check-venv
- $(call quiet-command, \
- $(PYTHON) -m avocado vmimage get \
- --distro=fedora --distro-version=31 --arch=$*, \
- "AVOCADO", "Downloading avocado tests VM image for $*")
-
-# download all vm images, according to defined targets
-get-vm-images: check-venv $(patsubst %,get-vm-image-fedora-31-%, $(FEDORA_31_DOWNLOAD))
-
-check-avocado: check-venv $(TESTS_RESULTS_DIR) get-vm-images
- $(call quiet-command, \
- $(PYTHON) -m avocado \
- --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
- $(if $(AVOCADO_TAGS),, --filter-by-tags-include-empty \
- --filter-by-tags-include-empty-key) \
- $(AVOCADO_CMDLINE_TAGS) --max-parallel-tasks=1 \
- $(if $(GITLAB_CI),,--failfast) $(AVOCADO_TESTS), \
- "AVOCADO", "tests/avocado")
-
-check-acceptance-deprecated-warning:
- @echo
- @echo "Note '$(MAKE) check-acceptance' is deprecated, use '$(MAKE) check-avocado' instead."
- @echo
-
-check-acceptance: check-acceptance-deprecated-warning | check-avocado
-
FUNCTIONAL_TARGETS=$(patsubst %-softmmu,check-functional-%, $(filter %-softmmu,$(TARGETS)))
.PHONY: $(FUNCTIONAL_TARGETS)
$(FUNCTIONAL_TARGETS):
@@ -166,13 +114,13 @@ check-functional:
# Consolidated targets
-.PHONY: check check-clean get-vm-images
+.PHONY: check check-clean
check:
check-build: run-ninja
check-clean:
- rm -rf $(TESTS_RESULTS_DIR)
+ rm -rf $(BUILD_DIR)/tests/functional
clean: check-clean clean-tcg
distclean: distclean-tcg
diff --git a/tests/avocado/avocado_qemu/__init__.py b/tests/avocado/avocado_qemu/__init__.py
deleted file mode 100644
index 93c34602428..00000000000
--- a/tests/avocado/avocado_qemu/__init__.py
+++ /dev/null
@@ -1,424 +0,0 @@
-# Test class and utilities for functional tests
-#
-# Copyright (c) 2018 Red Hat, Inc.
-#
-# Author:
-# Cleber Rosa <crosa@redhat.com>
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or
-# later. See the COPYING file in the top-level directory.
-
-import logging
-import os
-import subprocess
-import sys
-import tempfile
-import time
-import uuid
-
-import avocado
-from avocado.utils import ssh
-from avocado.utils.path import find_command
-
-from qemu.machine import QEMUMachine
-from qemu.utils import (get_info_usernet_hostfwd_port, kvm_available,
- tcg_available)
-
-
-#: The QEMU build root directory. It may also be the source directory
-#: if building from the source dir, but it's safer to use BUILD_DIR for
-#: that purpose. Be aware that if this code is moved outside of a source
-#: and build tree, it will not be accurate.
-BUILD_DIR = os.path.dirname(os.path.dirname(os.path.dirname(os.path.dirname(__file__))))
-
-
-def has_cmd(name, args=None):
- """
- This function is for use in a @avocado.skipUnless decorator, e.g.:
-
- @skipUnless(*has_cmd('sudo -n', ('sudo', '-n', 'true')))
- def test_something_that_needs_sudo(self):
- ...
- """
-
- if args is None:
- args = ('which', name)
-
- try:
- _, stderr, exitcode = run_cmd(args)
- except Exception as e:
- exitcode = -1
- stderr = str(e)
-
- if exitcode != 0:
- cmd_line = ' '.join(args)
- err = f'{name} required, but "{cmd_line}" failed: {stderr.strip()}'
- return (False, err)
- else:
- return (True, '')
-
-def has_cmds(*cmds):
- """
- This function is for use in a @avocado.skipUnless decorator and
- allows checking for the availability of multiple commands, e.g.:
-
- @skipUnless(*has_cmds(('cmd1', ('cmd1', '--some-parameter')),
- 'cmd2', 'cmd3'))
- def test_something_that_needs_cmd1_and_cmd2(self):
- ...
- """
-
- for cmd in cmds:
- if isinstance(cmd, str):
- cmd = (cmd,)
-
- ok, errstr = has_cmd(*cmd)
- if not ok:
- return (False, errstr)
-
- return (True, '')
-
-def run_cmd(args):
- subp = subprocess.Popen(args,
- stdout=subprocess.PIPE,
- stderr=subprocess.PIPE,
- universal_newlines=True)
- stdout, stderr = subp.communicate()
- ret = subp.returncode
-
- return (stdout, stderr, ret)
-
-def is_readable_executable_file(path):
- return os.path.isfile(path) and os.access(path, os.R_OK | os.X_OK)
-
-
-def pick_default_qemu_bin(bin_prefix='qemu-system-', arch=None):
- """
- Picks the path of a QEMU binary, starting either in the current working
- directory or in the source tree root directory.
-
- :param arch: the arch to use when looking for a QEMU binary (the target
- will match the arch given). If None (the default), arch
- will be the current host system arch (as given by
- :func:`os.uname`).
- :type arch: str
- :returns: the path to the default QEMU binary or None if one could not
- be found
- :rtype: str or None
- """
- if arch is None:
- arch = os.uname()[4]
- # qemu binary path does not match arch for powerpc, handle it
- if 'ppc64le' in arch:
- arch = 'ppc64'
- qemu_bin_name = bin_prefix + arch
- qemu_bin_paths = [
- os.path.join(".", qemu_bin_name),
- os.path.join(BUILD_DIR, qemu_bin_name),
- os.path.join(BUILD_DIR, "build", qemu_bin_name),
- ]
- for path in qemu_bin_paths:
- if is_readable_executable_file(path):
- return path
- return None
-
-
-def _console_interaction(test, success_message, failure_message,
- send_string, keep_sending=False, vm=None):
- assert not keep_sending or send_string
- if vm is None:
- vm = test.vm
- console = vm.console_file
- console_logger = logging.getLogger('console')
- while True:
- if send_string:
- vm.console_socket.sendall(send_string.encode())
- if not keep_sending:
- send_string = None # send only once
-
- # Only consume console output if waiting for something
- if success_message is None and failure_message is None:
- if send_string is None:
- break
- continue
-
- try:
- msg = console.readline().decode().strip()
- except UnicodeDecodeError:
- msg = None
- if not msg:
- continue
- console_logger.debug(msg)
- if success_message is None or success_message in msg:
- break
- if failure_message and failure_message in msg:
- console.close()
- fail = 'Failure message found in console: "%s". Expected: "%s"' % \
- (failure_message, success_message)
- test.fail(fail)
-
-def interrupt_interactive_console_until_pattern(test, success_message,
- failure_message=None,
- interrupt_string='\r'):
- """
- Keep sending a string to interrupt a console prompt, while logging the
- console output. Typical use case is to break a boot loader prompt, such:
-
- Press a key within 5 seconds to interrupt boot process.
- 5
- 4
- 3
- 2
- 1
- Booting default image...
-
- :param test: an Avocado test containing a VM that will have its console
- read and probed for a success or failure message
- :type test: :class:`avocado_qemu.QemuSystemTest`
- :param success_message: if this message appears, test succeeds
- :param failure_message: if this message appears, test fails
- :param interrupt_string: a string to send to the console before trying
- to read a new line
- """
- _console_interaction(test, success_message, failure_message,
- interrupt_string, True)
-
-def wait_for_console_pattern(test, success_message, failure_message=None,
- vm=None):
- """
- Waits for messages to appear on the console, while logging the content
-
- :param test: an Avocado test containing a VM that will have its console
- read and probed for a success or failure message
- :type test: :class:`avocado_qemu.QemuSystemTest`
- :param success_message: if this message appears, test succeeds
- :param failure_message: if this message appears, test fails
- """
- _console_interaction(test, success_message, failure_message, None, vm=vm)
-
-def exec_command(test, command):
- """
- Send a command to a console (appending CRLF characters), while logging
- the content.
-
- :param test: an Avocado test containing a VM.
- :type test: :class:`avocado_qemu.QemuSystemTest`
- :param command: the command to send
- :type command: str
- """
- _console_interaction(test, None, None, command + '\r')
-
-def exec_command_and_wait_for_pattern(test, command,
- success_message, failure_message=None):
- """
- Send a command to a console (appending CRLF characters), then wait
- for success_message to appear on the console, while logging the.
- content. Mark the test as failed if failure_message is found instead.
-
- :param test: an Avocado test containing a VM that will have its console
- read and probed for a success or failure message
- :type test: :class:`avocado_qemu.QemuSystemTest`
- :param command: the command to send
- :param success_message: if this message appears, test succeeds
- :param failure_message: if this message appears, test fails
- """
- _console_interaction(test, success_message, failure_message, command + '\r')
-
-class QemuBaseTest(avocado.Test):
-
- # default timeout for all tests, can be overridden
- timeout = 120
-
- def _get_unique_tag_val(self, tag_name):
- """
- Gets a tag value, if unique for a key
- """
- vals = self.tags.get(tag_name, [])
- if len(vals) == 1:
- return vals.pop()
- return None
-
- def setUp(self, bin_prefix):
- self.arch = self.params.get('arch',
- default=self._get_unique_tag_val('arch'))
-
- self.cpu = self.params.get('cpu',
- default=self._get_unique_tag_val('cpu'))
-
- default_qemu_bin = pick_default_qemu_bin(bin_prefix, arch=self.arch)
- self.qemu_bin = self.params.get('qemu_bin',
- default=default_qemu_bin)
- if self.qemu_bin is None:
- self.cancel("No QEMU binary defined or found in the build tree")
-
- def fetch_asset(self, name,
- asset_hash, algorithm=None,
- locations=None, expire=None,
- find_only=False, cancel_on_missing=True):
- return super().fetch_asset(name,
- asset_hash=asset_hash,
- algorithm=algorithm,
- locations=locations,
- expire=expire,
- find_only=find_only,
- cancel_on_missing=cancel_on_missing)
-
-
-class QemuSystemTest(QemuBaseTest):
- """Facilitates system emulation tests."""
-
- def setUp(self):
- self._vms = {}
-
- super().setUp('qemu-system-')
-
- accel_required = self._get_unique_tag_val('accel')
- if accel_required:
- self.require_accelerator(accel_required)
-
- self.machine = self.params.get('machine',
- default=self._get_unique_tag_val('machine'))
-
- def require_accelerator(self, accelerator):
- """
- Requires an accelerator to be available for the test to continue
-
- It takes into account the currently set qemu binary.
-
- If the check fails, the test is canceled. If the check itself
- for the given accelerator is not available, the test is also
- canceled.
-
- :param accelerator: name of the accelerator, such as "kvm" or "tcg"
- :type accelerator: str
- """
- checker = {'tcg': tcg_available,
- 'kvm': kvm_available}.get(accelerator)
- if checker is None:
- self.cancel("Don't know how to check for the presence "
- "of accelerator %s" % accelerator)
- if not checker(qemu_bin=self.qemu_bin):
- self.cancel("%s accelerator does not seem to be "
- "available" % accelerator)
-
- def require_netdev(self, netdevname):
- netdevhelp = run_cmd([self.qemu_bin,
- '-M', 'none', '-netdev', 'help'])[0];
- if netdevhelp.find('\n' + netdevname + '\n') < 0:
- self.cancel('no support for user networking')
-
- def _new_vm(self, name, *args):
- self._sd = tempfile.TemporaryDirectory(prefix="qemu_")
- vm = QEMUMachine(self.qemu_bin, base_temp_dir=self.workdir,
- log_dir=self.logdir)
- self.log.debug('QEMUMachine "%s" created', name)
- self.log.debug('QEMUMachine "%s" temp_dir: %s', name, vm.temp_dir)
- self.log.debug('QEMUMachine "%s" log_dir: %s', name, vm.log_dir)
- if args:
- vm.add_args(*args)
- return vm
-
- def get_qemu_img(self):
- self.log.debug('Looking for and selecting a qemu-img binary')
-
- # If qemu-img has been built, use it, otherwise the system wide one
- # will be used.
- qemu_img = os.path.join(BUILD_DIR, 'qemu-img')
- if not os.path.exists(qemu_img):
- qemu_img = find_command('qemu-img', False)
- if qemu_img is False:
- self.cancel('Could not find "qemu-img"')
-
- return qemu_img
-
- @property
- def vm(self):
- return self.get_vm(name='default')
-
- def get_vm(self, *args, name=None):
- if not name:
- name = str(uuid.uuid4())
- if self._vms.get(name) is None:
- self._vms[name] = self._new_vm(name, *args)
- if self.cpu is not None:
- self._vms[name].add_args('-cpu', self.cpu)
- if self.machine is not None:
- self._vms[name].set_machine(self.machine)
- return self._vms[name]
-
- def set_vm_arg(self, arg, value):
- """
- Set an argument to list of extra arguments to be given to the QEMU
- binary. If the argument already exists then its value is replaced.
-
- :param arg: the QEMU argument, such as "-cpu" in "-cpu host"
- :type arg: str
- :param value: the argument value, such as "host" in "-cpu host"
- :type value: str
- """
- if not arg or not value:
- return
- if arg not in self.vm.args:
- self.vm.args.extend([arg, value])
- else:
- idx = self.vm.args.index(arg) + 1
- if idx < len(self.vm.args):
- self.vm.args[idx] = value
- else:
- self.vm.args.append(value)
-
- def tearDown(self):
- for vm in self._vms.values():
- vm.shutdown()
- self._sd = None
- super().tearDown()
-
-
-class LinuxSSHMixIn:
- """Contains utility methods for interacting with a guest via SSH."""
-
- def ssh_connect(self, username, credential, credential_is_key=True):
- self.ssh_logger = logging.getLogger('ssh')
- res = self.vm.cmd('human-monitor-command',
- command_line='info usernet')
- port = get_info_usernet_hostfwd_port(res)
- self.assertIsNotNone(port)
- self.assertGreater(port, 0)
- self.log.debug('sshd listening on port: %d', port)
- if credential_is_key:
- self.ssh_session = ssh.Session('127.0.0.1', port=port,
- user=username, key=credential)
- else:
- self.ssh_session = ssh.Session('127.0.0.1', port=port,
- user=username, password=credential)
- for i in range(10):
- try:
- self.ssh_session.connect()
- return
- except:
- time.sleep(i)
- self.fail('ssh connection timeout')
-
- def ssh_command(self, command):
- self.ssh_logger.info(command)
- result = self.ssh_session.cmd(command)
- stdout_lines = [line.rstrip() for line
- in result.stdout_text.splitlines()]
- for line in stdout_lines:
- self.ssh_logger.info(line)
- stderr_lines = [line.rstrip() for line
- in result.stderr_text.splitlines()]
- for line in stderr_lines:
- self.ssh_logger.warning(line)
-
- self.assertEqual(result.exit_status, 0,
- f'Guest command failed: {command}')
- return stdout_lines, stderr_lines
-
- def ssh_command_output_contains(self, cmd, exp):
- stdout, _ = self.ssh_command(cmd)
- for line in stdout:
- if exp in line:
- break
- else:
- self.fail('"%s" output does not contain "%s"' % (cmd, exp))
diff --git a/tests/avocado/avocado_qemu/linuxtest.py b/tests/avocado/avocado_qemu/linuxtest.py
deleted file mode 100644
index 66fb9f15079..00000000000
--- a/tests/avocado/avocado_qemu/linuxtest.py
+++ /dev/null
@@ -1,253 +0,0 @@
-# Test class and utilities for functional Linux-based tests
-#
-# Copyright (c) 2018 Red Hat, Inc.
-#
-# Author:
-# Cleber Rosa <crosa@redhat.com>
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or
-# later. See the COPYING file in the top-level directory.
-
-import os
-import shutil
-
-from avocado.utils import cloudinit, datadrainer, process, vmimage
-
-from avocado_qemu import LinuxSSHMixIn
-from avocado_qemu import QemuSystemTest
-
-if os.path.islink(os.path.dirname(os.path.dirname(__file__))):
- # The link to the avocado tests dir in the source code directory
- lnk = os.path.dirname(os.path.dirname(__file__))
- #: The QEMU root source directory
- SOURCE_DIR = os.path.dirname(os.path.dirname(os.readlink(lnk)))
-else:
- SOURCE_DIR = BUILD_DIR
-
-class LinuxDistro:
- """Represents a Linux distribution
-
- Holds information of known distros.
- """
- #: A collection of known distros and their respective image checksum
- KNOWN_DISTROS = {
- 'fedora': {
- '31': {
- 'x86_64':
- {'checksum': ('e3c1b309d9203604922d6e255c2c5d09'
- '8a309c2d46215d8fc026954f3c5c27a0'),
- 'pxeboot_url': ('https://archives.fedoraproject.org/'
- 'pub/archive/fedora/linux/releases/31/'
- 'Everything/x86_64/os/images/pxeboot/'),
- 'kernel_params': ('root=UUID=b1438b9b-2cab-4065-a99a-'
- '08a96687f73c ro no_timer_check '
- 'net.ifnames=0 console=tty1 '
- 'console=ttyS0,115200n8'),
- },
- 'aarch64':
- {'checksum': ('1e18d9c0cf734940c4b5d5ec592facae'
- 'd2af0ad0329383d5639c997fdf16fe49'),
- 'pxeboot_url': 'https://archives.fedoraproject.org/'
- 'pub/archive/fedora/linux/releases/31/'
- 'Everything/aarch64/os/images/pxeboot/',
- 'kernel_params': ('root=UUID=b6950a44-9f3c-4076-a9c2-'
- '355e8475b0a7 ro earlyprintk=pl011,0x9000000'
- ' ignore_loglevel no_timer_check'
- ' printk.time=1 rd_NO_PLYMOUTH'
- ' console=ttyAMA0'),
- },
- 'ppc64':
- {'checksum': ('7c3528b85a3df4b2306e892199a9e1e4'
- '3f991c506f2cc390dc4efa2026ad2f58')},
- 's390x':
- {'checksum': ('4caaab5a434fd4d1079149a072fdc789'
- '1e354f834d355069ca982fdcaf5a122d')},
- },
- '32': {
- 'aarch64':
- {'checksum': ('b367755c664a2d7a26955bbfff985855'
- 'adfa2ca15e908baf15b4b176d68d3967'),
- 'pxeboot_url': ('http://dl.fedoraproject.org/pub/fedora/linux/'
- 'releases/32/Server/aarch64/os/images/'
- 'pxeboot/'),
- 'kernel_params': ('root=UUID=3df75b65-be8d-4db4-8655-'
- '14d95c0e90c5 ro no_timer_check net.ifnames=0'
- ' console=tty1 console=ttyS0,115200n8'),
- },
- },
- '33': {
- 'aarch64':
- {'checksum': ('e7f75cdfd523fe5ac2ca9eeece68edc1'
- 'a81f386a17f969c1d1c7c87031008a6b'),
- 'pxeboot_url': ('http://dl.fedoraproject.org/pub/fedora/linux/'
- 'releases/33/Server/aarch64/os/images/'
- 'pxeboot/'),
- 'kernel_params': ('root=UUID=d20b3ffa-6397-4a63-a734-'
- '1126a0208f8a ro no_timer_check net.ifnames=0'
- ' console=tty1 console=ttyS0,115200n8'
- ' console=tty0'),
- },
- },
- }
- }
-
- def __init__(self, name, version, arch):
- self.name = name
- self.version = version
- self.arch = arch
- try:
- info = self.KNOWN_DISTROS.get(name).get(version).get(arch)
- except AttributeError:
- # Unknown distro
- info = None
- self._info = info or {}
-
- @property
- def checksum(self):
- """Gets the cloud-image file checksum"""
- return self._info.get('checksum', None)
-
- @checksum.setter
- def checksum(self, value):
- self._info['checksum'] = value
-
- @property
- def pxeboot_url(self):
- """Gets the repository url where pxeboot files can be found"""
- return self._info.get('pxeboot_url', None)
-
- @property
- def default_kernel_params(self):
- """Gets the default kernel parameters"""
- return self._info.get('kernel_params', None)
-
-
-class LinuxTest(LinuxSSHMixIn, QemuSystemTest):
- """Facilitates having a cloud-image Linux based available.
-
- For tests that intend to interact with guests, this is a better choice
- to start with than the more vanilla `QemuSystemTest` class.
- """
-
- distro = None
- username = 'root'
- password = 'password'
- smp = '2'
- memory = '1024'
-
- def _set_distro(self):
- distro_name = self.params.get(
- 'distro',
- default=self._get_unique_tag_val('distro'))
- if not distro_name:
- distro_name = 'fedora'
-
- distro_version = self.params.get(
- 'distro_version',
- default=self._get_unique_tag_val('distro_version'))
- if not distro_version:
- distro_version = '31'
-
- self.distro = LinuxDistro(distro_name, distro_version, self.arch)
-
- # The distro checksum behaves differently than distro name and
- # version. First, it does not respect a tag with the same
- # name, given that it's not expected to be used for filtering
- # (distro name versions are the natural choice). Second, the
- # order of precedence is: parameter, attribute and then value
- # from KNOWN_DISTROS.
- distro_checksum = self.params.get('distro_checksum',
- default=None)
- if distro_checksum:
- self.distro.checksum = distro_checksum
-
- def setUp(self, ssh_pubkey=None, network_device_type='virtio-net'):
- super().setUp()
- self.require_netdev('user')
- self._set_distro()
- self.vm.add_args('-smp', self.smp)
- self.vm.add_args('-m', self.memory)
- # The following network device allows for SSH connections
- self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22',
- '-device', '%s,netdev=vnet' % network_device_type)
- self.set_up_boot()
- if ssh_pubkey is None:
- ssh_pubkey, self.ssh_key = self.set_up_existing_ssh_keys()
- self.set_up_cloudinit(ssh_pubkey)
-
- def set_up_existing_ssh_keys(self):
- ssh_public_key = os.path.join(SOURCE_DIR, 'tests', 'keys', 'id_rsa.pub')
- source_private_key = os.path.join(SOURCE_DIR, 'tests', 'keys', 'id_rsa')
- ssh_dir = os.path.join(self.workdir, '.ssh')
- os.mkdir(ssh_dir, mode=0o700)
- ssh_private_key = os.path.join(ssh_dir,
- os.path.basename(source_private_key))
- shutil.copyfile(source_private_key, ssh_private_key)
- os.chmod(ssh_private_key, 0o600)
- return (ssh_public_key, ssh_private_key)
-
- def download_boot(self):
- # Set the qemu-img binary.
- # If none is available, the test will cancel.
- vmimage.QEMU_IMG = super().get_qemu_img()
-
- self.log.info('Downloading/preparing boot image')
- # Fedora 31 only provides ppc64le images
- image_arch = self.arch
- if self.distro.name == 'fedora':
- if image_arch == 'ppc64':
- image_arch = 'ppc64le'
-
- try:
- boot = vmimage.get(
- self.distro.name, arch=image_arch, version=self.distro.version,
- checksum=self.distro.checksum,
- algorithm='sha256',
- cache_dir=self.cache_dirs[0],
- snapshot_dir=self.workdir)
- except:
- self.cancel('Failed to download/prepare boot image')
- return boot.path
-
- def prepare_cloudinit(self, ssh_pubkey=None):
- self.log.info('Preparing cloudinit image')
- try:
- cloudinit_iso = os.path.join(self.workdir, 'cloudinit.iso')
- pubkey_content = None
- if ssh_pubkey:
- with open(ssh_pubkey) as pubkey:
- pubkey_content = pubkey.read()
- cloudinit.iso(cloudinit_iso, self.name,
- username=self.username,
- password=self.password,
- # QEMU's hard coded usermode router address
- phone_home_host='10.0.2.2',
- phone_home_port=self.phone_server.server_port,
- authorized_key=pubkey_content)
- except Exception:
- self.cancel('Failed to prepare the cloudinit image')
- return cloudinit_iso
-
- def set_up_boot(self):
- path = self.download_boot()
- self.vm.add_args('-drive', 'file=%s' % path)
-
- def set_up_cloudinit(self, ssh_pubkey=None):
- self.phone_server = cloudinit.PhoneHomeServer(('0.0.0.0', 0),
- self.name)
- cloudinit_iso = self.prepare_cloudinit(ssh_pubkey)
- self.vm.add_args('-drive', 'file=%s,format=raw' % cloudinit_iso)
-
- def launch_and_wait(self, set_up_ssh_connection=True):
- self.vm.set_console()
- self.vm.launch()
- console_drainer = datadrainer.LineLogger(self.vm.console_socket.fileno(),
- logger=self.log.getChild('console'))
- console_drainer.start()
- self.log.info('VM launched, waiting for boot confirmation from guest')
- while not self.phone_server.instance_phoned_back:
- self.phone_server.handle_request()
-
- if set_up_ssh_connection:
- self.log.info('Setting up the SSH connection')
- self.ssh_connect(self.username, self.ssh_key)
--
2.49.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* Re: [PATCH 15/15] Remove the remainders of the Avocado tests
2025-03-25 20:00 ` [PATCH 15/15] Remove the remainders of the Avocado tests Thomas Huth
@ 2025-03-25 21:03 ` Philippe Mathieu-Daudé
2025-03-26 9:51 ` Daniel P. Berrangé
1 sibling, 0 replies; 34+ messages in thread
From: Philippe Mathieu-Daudé @ 2025-03-25 21:03 UTC (permalink / raw)
To: Thomas Huth, qemu-devel; +Cc: Daniel P. Berrangé
On 25/3/25 21:00, Thomas Huth wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> Now that all Avocado tests have been converted to or been replaced by
> other functional tests, we can delete the remainders of the Avocado
> tests from the QEMU source tree.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> MAINTAINERS | 8 +-
> docs/about/build-platforms.rst | 10 +-
> docs/devel/build-system.rst | 11 +-
> docs/devel/codebase.rst | 5 -
> docs/devel/testing/avocado.rst | 581 ------------------------
> docs/devel/testing/functional.rst | 3 -
> docs/devel/testing/index.rst | 1 -
> docs/devel/testing/main.rst | 15 -
> tests/avocado/README.rst | 10 -
> configure | 2 +-
> pythondeps.toml | 8 +-
> tests/Makefile.include | 60 +--
> tests/avocado/avocado_qemu/__init__.py | 424 -----------------
> tests/avocado/avocado_qemu/linuxtest.py | 253 -----------
> 14 files changed, 16 insertions(+), 1375 deletions(-)
> delete mode 100644 docs/devel/testing/avocado.rst
> delete mode 100644 tests/avocado/README.rst
> delete mode 100644 tests/avocado/avocado_qemu/__init__.py
> delete mode 100644 tests/avocado/avocado_qemu/linuxtest.py
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Thanks a lot Thomas for this huge effort of converting from Avocado
to the simpler functional framework... Sincerely appreciated!
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH 15/15] Remove the remainders of the Avocado tests
2025-03-25 20:00 ` [PATCH 15/15] Remove the remainders of the Avocado tests Thomas Huth
2025-03-25 21:03 ` Philippe Mathieu-Daudé
@ 2025-03-26 9:51 ` Daniel P. Berrangé
1 sibling, 0 replies; 34+ messages in thread
From: Daniel P. Berrangé @ 2025-03-26 9:51 UTC (permalink / raw)
To: Thomas Huth; +Cc: qemu-devel, Philippe Mathieu-Daudé
On Tue, Mar 25, 2025 at 09:00:23PM +0100, Thomas Huth wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> Now that all Avocado tests have been converted to or been replaced by
> other functional tests, we can delete the remainders of the Avocado
> tests from the QEMU source tree.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> MAINTAINERS | 8 +-
> docs/about/build-platforms.rst | 10 +-
> docs/devel/build-system.rst | 11 +-
> docs/devel/codebase.rst | 5 -
> docs/devel/testing/avocado.rst | 581 ------------------------
> docs/devel/testing/functional.rst | 3 -
> docs/devel/testing/index.rst | 1 -
> docs/devel/testing/main.rst | 15 -
> tests/avocado/README.rst | 10 -
> configure | 2 +-
> pythondeps.toml | 8 +-
> tests/Makefile.include | 60 +--
> tests/avocado/avocado_qemu/__init__.py | 424 -----------------
> tests/avocado/avocado_qemu/linuxtest.py | 253 -----------
> 14 files changed, 16 insertions(+), 1375 deletions(-)
> delete mode 100644 docs/devel/testing/avocado.rst
> delete mode 100644 tests/avocado/README.rst
> delete mode 100644 tests/avocado/avocado_qemu/__init__.py
> delete mode 100644 tests/avocado/avocado_qemu/linuxtest.py
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 34+ messages in thread