* [PATCH v2 1/5] tests/functional: Add a decorator for skipping long running tests
2025-01-28 15:28 [PATCH v2 0/5] Convert the MIPS replay tests to the functional framework Thomas Huth
@ 2025-01-28 15:28 ` Thomas Huth
2025-01-29 13:49 ` Daniel P. Berrangé
2025-01-28 15:28 ` [PATCH v2 2/5] tests/functional: Add the ReplayKernelBase class Thomas Huth
` (3 subsequent siblings)
4 siblings, 1 reply; 10+ messages in thread
From: Thomas Huth @ 2025-01-28 15:28 UTC (permalink / raw)
To: Pavel Dovgalyuk, Paolo Bonzini, Jiaxun Yang, qemu-devel
Cc: Philippe Mathieu-Daudé, Daniel P. Berrangé, Hao Wu,
Tyrone Ting, Leif Lindholm, Radoslaw Biernacki, Peter Maydell
Some tests have a very long runtime and might run into timeout issues
e.g. when QEMU has been compiled with --enable-debug. Add a decorator
for marking them more easily. Rename the corresponding environment
variable to be more in sync with the other QEMU_TEST_ALLOW_* switches
that we already have, and add a paragraph about it in the documentation.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
docs/devel/testing/functional.rst | 8 ++++++++
tests/functional/qemu_test/__init__.py | 2 +-
tests/functional/qemu_test/decorators.py | 14 ++++++++++++++
tests/functional/test_aarch64_sbsaref_alpine.py | 5 ++---
tests/functional/test_aarch64_sbsaref_freebsd.py | 9 +++------
tests/functional/test_arm_quanta_gsj.py | 6 +++---
6 files changed, 31 insertions(+), 13 deletions(-)
diff --git a/docs/devel/testing/functional.rst b/docs/devel/testing/functional.rst
index ae238ed3fc..ecc738922b 100644
--- a/docs/devel/testing/functional.rst
+++ b/docs/devel/testing/functional.rst
@@ -351,5 +351,13 @@ the code snippet below:
Tests should not live in this state forever and should either be fixed
or eventually removed.
+QEMU_TEST_ALLOW_SLOW
+^^^^^^^^^^^^^^^^^^^^
+Tests that have a very long runtime and might run into timeout issues
+e.g. if the QEMU binary has been compiled with debugging options enabled.
+To avoid these timeout issues by default and to save some precious CPU
+cycles during normal testing, such tests are disabled by default unless
+the QEMU_TEST_ALLOW_SLOW environment variable has been set.
+
.. _unittest: https://docs.python.org/3/library/unittest.html
diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu_test/__init__.py
index da1830286d..5c972843a6 100644
--- a/tests/functional/qemu_test/__init__.py
+++ b/tests/functional/qemu_test/__init__.py
@@ -14,7 +14,7 @@
from .testcase import QemuBaseTest, QemuUserTest, QemuSystemTest
from .linuxkernel import LinuxKernelTest
from .decorators import skipIfMissingCommands, skipIfNotMachine, \
- skipFlakyTest, skipUntrustedTest, skipBigDataTest, \
+ skipFlakyTest, skipUntrustedTest, skipBigDataTest, skipSlowTest, \
skipIfMissingImports
from .archive import archive_extract
from .uncompress import uncompress
diff --git a/tests/functional/qemu_test/decorators.py b/tests/functional/qemu_test/decorators.py
index 3d9c02fd59..1651eb739a 100644
--- a/tests/functional/qemu_test/decorators.py
+++ b/tests/functional/qemu_test/decorators.py
@@ -86,6 +86,20 @@ def skipBigDataTest():
return skipUnless(os.getenv('QEMU_TEST_ALLOW_LARGE_STORAGE'),
'Test requires large host storage space')
+'''
+Decorator to skip execution of tests which have a really long
+runtime (and might e.g. time out if QEMU has been compiled with
+debugging enabled) unless the $QEMU_TEST_ALLOW_SLOW
+environment variable is set
+
+Example:
+
+ @skipSlowTest()
+'''
+def skipSlowTest():
+ return skipUnless(os.getenv('QEMU_TEST_ALLOW_SLOW'),
+ 'Test has a very long runtime and might time out')
+
'''
Decorator to skip execution of a test if the list
of python imports is not available.
diff --git a/tests/functional/test_aarch64_sbsaref_alpine.py b/tests/functional/test_aarch64_sbsaref_alpine.py
index 6dbc90f30e..ce974fd7e1 100755
--- a/tests/functional/test_aarch64_sbsaref_alpine.py
+++ b/tests/functional/test_aarch64_sbsaref_alpine.py
@@ -10,7 +10,7 @@
import os
-from qemu_test import QemuSystemTest, Asset
+from qemu_test import QemuSystemTest, Asset, skipSlowTest
from qemu_test import wait_for_console_pattern
from unittest import skipUnless
from test_aarch64_sbsaref import fetch_firmware
@@ -53,8 +53,7 @@ def test_sbsaref_alpine_linux_max_pauth_off(self):
def test_sbsaref_alpine_linux_max_pauth_impdef(self):
self.boot_alpine_linux("max,pauth-impdef=on")
- @skipUnless(os.getenv('QEMU_TEST_TIMEOUT_EXPECTED'),
- 'Test might timeout due to PAuth emulation')
+ @skipSlowTest() # Test might timeout due to PAuth emulation
def test_sbsaref_alpine_linux_max(self):
self.boot_alpine_linux("max")
diff --git a/tests/functional/test_aarch64_sbsaref_freebsd.py b/tests/functional/test_aarch64_sbsaref_freebsd.py
index 77ba2ba1da..5b10bb9b64 100755
--- a/tests/functional/test_aarch64_sbsaref_freebsd.py
+++ b/tests/functional/test_aarch64_sbsaref_freebsd.py
@@ -10,9 +10,8 @@
import os
-from qemu_test import QemuSystemTest, Asset
+from qemu_test import QemuSystemTest, Asset, skipSlowTest
from qemu_test import wait_for_console_pattern
-from unittest import skipUnless
from test_aarch64_sbsaref import fetch_firmware
@@ -50,13 +49,11 @@ def test_sbsaref_freebsd14_default_cpu(self):
def test_sbsaref_freebsd14_max_pauth_off(self):
self.boot_freebsd14("max,pauth=off")
- @skipUnless(os.getenv('QEMU_TEST_TIMEOUT_EXPECTED'),
- 'Test might timeout due to PAuth emulation')
+ @skipSlowTest() # Test might timeout due to PAuth emulation
def test_sbsaref_freebsd14_max_pauth_impdef(self):
self.boot_freebsd14("max,pauth-impdef=on")
- @skipUnless(os.getenv('QEMU_TEST_TIMEOUT_EXPECTED'),
- 'Test might timeout due to PAuth emulation')
+ @skipSlowTest() # Test might timeout due to PAuth emulation
def test_sbsaref_freebsd14_max(self):
self.boot_freebsd14("max")
diff --git a/tests/functional/test_arm_quanta_gsj.py b/tests/functional/test_arm_quanta_gsj.py
index 7b82e2185c..da60aeb659 100755
--- a/tests/functional/test_arm_quanta_gsj.py
+++ b/tests/functional/test_arm_quanta_gsj.py
@@ -7,8 +7,8 @@
import os
from qemu_test import LinuxKernelTest, Asset, exec_command_and_wait_for_pattern
-from qemu_test import interrupt_interactive_console_until_pattern
-from unittest import skipUnless
+from qemu_test import interrupt_interactive_console_until_pattern, skipSlowTest
+
class EmcraftSf2Machine(LinuxKernelTest):
@@ -32,7 +32,7 @@ class EmcraftSf2Machine(LinuxKernelTest):
'20200711-gsj-qemu-0/nuvoton-npcm730-gsj.dtb'),
'3249b2da787d4b9ad4e61f315b160abfceb87b5e1895a7ce898ce7f40c8d4045')
- @skipUnless(os.getenv('QEMU_TEST_TIMEOUT_EXPECTED'), 'Test might timeout')
+ @skipSlowTest()
def test_arm_quanta_gsj(self):
self.set_machine('quanta-gsj')
image_path = self.uncompress(self.ASSET_IMAGE, format='gz')
--
2.48.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* Re: [PATCH v2 1/5] tests/functional: Add a decorator for skipping long running tests
2025-01-28 15:28 ` [PATCH v2 1/5] tests/functional: Add a decorator for skipping long running tests Thomas Huth
@ 2025-01-29 13:49 ` Daniel P. Berrangé
0 siblings, 0 replies; 10+ messages in thread
From: Daniel P. Berrangé @ 2025-01-29 13:49 UTC (permalink / raw)
To: Thomas Huth
Cc: Pavel Dovgalyuk, Paolo Bonzini, Jiaxun Yang, qemu-devel,
Philippe Mathieu-Daudé, Hao Wu, Tyrone Ting, Leif Lindholm,
Radoslaw Biernacki, Peter Maydell
On Tue, Jan 28, 2025 at 04:28:35PM +0100, Thomas Huth wrote:
> Some tests have a very long runtime and might run into timeout issues
> e.g. when QEMU has been compiled with --enable-debug. Add a decorator
> for marking them more easily. Rename the corresponding environment
> variable to be more in sync with the other QEMU_TEST_ALLOW_* switches
> that we already have, and add a paragraph about it in the documentation.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> docs/devel/testing/functional.rst | 8 ++++++++
> tests/functional/qemu_test/__init__.py | 2 +-
> tests/functional/qemu_test/decorators.py | 14 ++++++++++++++
> tests/functional/test_aarch64_sbsaref_alpine.py | 5 ++---
> tests/functional/test_aarch64_sbsaref_freebsd.py | 9 +++------
> tests/functional/test_arm_quanta_gsj.py | 6 +++---
> 6 files changed, 31 insertions(+), 13 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v2 2/5] tests/functional: Add the ReplayKernelBase class
2025-01-28 15:28 [PATCH v2 0/5] Convert the MIPS replay tests to the functional framework Thomas Huth
2025-01-28 15:28 ` [PATCH v2 1/5] tests/functional: Add a decorator for skipping long running tests Thomas Huth
@ 2025-01-28 15:28 ` Thomas Huth
2025-01-28 15:28 ` [PATCH v2 3/5] tests/functional/test_mipsel_malta: Convert the mipsel replay tests Thomas Huth
` (2 subsequent siblings)
4 siblings, 0 replies; 10+ messages in thread
From: Thomas Huth @ 2025-01-28 15:28 UTC (permalink / raw)
To: Pavel Dovgalyuk, Paolo Bonzini, Jiaxun Yang, qemu-devel
Cc: Philippe Mathieu-Daudé, Daniel P. Berrangé, Hao Wu,
Tyrone Ting, Leif Lindholm, Radoslaw Biernacki, Peter Maydell
Copy the ReplayKernelBase class from the avocado tests. We are going
to need it to convert the related replay tests in the following patches.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 1 +
tests/functional/replay_kernel.py | 84 +++++++++++++++++++++++++++++++
2 files changed, 85 insertions(+)
create mode 100644 tests/functional/replay_kernel.py
diff --git a/MAINTAINERS b/MAINTAINERS
index 7b4d84bf5f..414b6f35fe 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3632,6 +3632,7 @@ F: stubs/replay.c
F: tests/avocado/replay_kernel.py
F: tests/avocado/replay_linux.py
F: tests/avocado/reverse_debugging.py
+F: tests/functional/*replay*.py
F: qapi/replay.json
IOVA Tree
diff --git a/tests/functional/replay_kernel.py b/tests/functional/replay_kernel.py
new file mode 100644
index 0000000000..8e8ac7d052
--- /dev/null
+++ b/tests/functional/replay_kernel.py
@@ -0,0 +1,84 @@
+# Record/replay test that boots a Linux kernel
+#
+# Copyright (c) 2020 ISP RAS
+#
+# Author:
+# Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import os
+import logging
+import time
+import subprocess
+
+from qemu_test.linuxkernel import LinuxKernelTest
+
+class ReplayKernelBase(LinuxKernelTest):
+ """
+ Boots a Linux kernel in record mode and checks that the console
+ is operational and the kernel command line is properly passed
+ from QEMU to the kernel.
+ Then replays the same scenario and verifies, that QEMU correctly
+ terminates.
+ """
+
+ timeout = 180
+ REPLAY_KERNEL_COMMAND_LINE = 'printk.time=1 panic=-1 '
+
+ def run_vm(self, kernel_path, kernel_command_line, console_pattern,
+ record, shift, args, replay_path):
+ # icount requires TCG to be available
+ self.require_accelerator('tcg')
+
+ logger = logging.getLogger('replay')
+ start_time = time.time()
+ vm = self.get_vm()
+ vm.set_console()
+ if record:
+ logger.info('recording the execution...')
+ mode = 'record'
+ else:
+ logger.info('replaying the execution...')
+ mode = 'replay'
+ vm.add_args('-icount', 'shift=%s,rr=%s,rrfile=%s' %
+ (shift, mode, replay_path),
+ '-kernel', kernel_path,
+ '-append', kernel_command_line,
+ '-net', 'none',
+ '-no-reboot')
+ if args:
+ vm.add_args(*args)
+ vm.launch()
+ self.wait_for_console_pattern(console_pattern, vm)
+ if record:
+ vm.shutdown()
+ logger.info('finished the recording with log size %s bytes'
+ % os.path.getsize(replay_path))
+ self.run_replay_dump(replay_path)
+ logger.info('successfully tested replay-dump.py')
+ else:
+ vm.wait()
+ logger.info('successfully finished the replay')
+ elapsed = time.time() - start_time
+ logger.info('elapsed time %.2f sec' % elapsed)
+ return elapsed
+
+ def run_replay_dump(self, replay_path):
+ try:
+ subprocess.check_call(["./scripts/replay-dump.py",
+ "-f", replay_path],
+ stdout=subprocess.DEVNULL)
+ except subprocess.CalledProcessError:
+ self.fail('replay-dump.py failed')
+
+ def run_rr(self, kernel_path, kernel_command_line, console_pattern,
+ shift=7, args=None):
+ replay_path = os.path.join(self.workdir, 'replay.bin')
+ t1 = self.run_vm(kernel_path, kernel_command_line, console_pattern,
+ True, shift, args, replay_path)
+ t2 = self.run_vm(kernel_path, kernel_command_line, console_pattern,
+ False, shift, args, replay_path)
+ logger = logging.getLogger('replay')
+ logger.info('replay overhead {:.2%}'.format(t2 / t1 - 1))
--
2.48.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH v2 3/5] tests/functional/test_mipsel_malta: Convert the mipsel replay tests
2025-01-28 15:28 [PATCH v2 0/5] Convert the MIPS replay tests to the functional framework Thomas Huth
2025-01-28 15:28 ` [PATCH v2 1/5] tests/functional: Add a decorator for skipping long running tests Thomas Huth
2025-01-28 15:28 ` [PATCH v2 2/5] tests/functional: Add the ReplayKernelBase class Thomas Huth
@ 2025-01-28 15:28 ` Thomas Huth
2025-01-29 13:50 ` Daniel P. Berrangé
2025-01-28 15:28 ` [PATCH v2 4/5] tests/functional/test_mips64el_malta: Convert the mips64el " Thomas Huth
2025-01-28 15:28 ` [PATCH v2 5/5] tests/functional/test_mips_malta: Convert the mips big endian " Thomas Huth
4 siblings, 1 reply; 10+ messages in thread
From: Thomas Huth @ 2025-01-28 15:28 UTC (permalink / raw)
To: Pavel Dovgalyuk, Paolo Bonzini, Jiaxun Yang, qemu-devel
Cc: Philippe Mathieu-Daudé, Daniel P. Berrangé, Hao Wu,
Tyrone Ting, Leif Lindholm, Radoslaw Biernacki, Peter Maydell
Move the mipsel replay tests from tests/avocado/replay_kernel.py to
the functional framework. Since the functional tests should be run per
target, we cannot stick all replay tests in one file. Thus let's add
these tests to a new, separate file there instead.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/avocado/replay_kernel.py | 54 --------------------------
tests/functional/meson.build | 2 +
tests/functional/test_mipsel_replay.py | 54 ++++++++++++++++++++++++++
3 files changed, 56 insertions(+), 54 deletions(-)
create mode 100644 tests/functional/test_mipsel_replay.py
diff --git a/tests/avocado/replay_kernel.py b/tests/avocado/replay_kernel.py
index eae553f20e..9a3f175129 100644
--- a/tests/avocado/replay_kernel.py
+++ b/tests/avocado/replay_kernel.py
@@ -505,57 +505,3 @@ def test_mips64el_malta_5KEc_cpio(self):
console_pattern = 'Boot successful.'
self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5,
args=('-initrd', initrd_path))
-
- def do_test_mips_malta32el_nanomips(self, kernel_path_xz):
- kernel_path = self.workdir + "kernel"
- with lzma.open(kernel_path_xz, 'rb') as f_in:
- with open(kernel_path, 'wb') as f_out:
- shutil.copyfileobj(f_in, f_out)
-
- kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
- 'mem=256m@@0x0 '
- 'console=ttyS0')
- console_pattern = 'Kernel command line: %s' % kernel_command_line
- self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5)
-
- def test_mips_malta32el_nanomips_4k(self):
- """
- :avocado: tags=arch:mipsel
- :avocado: tags=machine:malta
- :avocado: tags=endian:little
- :avocado: tags=cpu:I7200
- """
- kernel_url = ('http://mipsdistros.mips.com/LinuxDistro/nanomips/'
- 'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
- 'generic_nano32r6el_page4k.xz')
- kernel_hash = '477456aafd2a0f1ddc9482727f20fe9575565dd6'
- kernel_path_xz = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
- self.do_test_mips_malta32el_nanomips(kernel_path_xz)
-
- def test_mips_malta32el_nanomips_16k_up(self):
- """
- :avocado: tags=arch:mipsel
- :avocado: tags=machine:malta
- :avocado: tags=endian:little
- :avocado: tags=cpu:I7200
- """
- kernel_url = ('http://mipsdistros.mips.com/LinuxDistro/nanomips/'
- 'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
- 'generic_nano32r6el_page16k_up.xz')
- kernel_hash = 'e882868f944c71c816e832e2303b7874d044a7bc'
- kernel_path_xz = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
- self.do_test_mips_malta32el_nanomips(kernel_path_xz)
-
- def test_mips_malta32el_nanomips_64k_dbg(self):
- """
- :avocado: tags=arch:mipsel
- :avocado: tags=machine:malta
- :avocado: tags=endian:little
- :avocado: tags=cpu:I7200
- """
- kernel_url = ('http://mipsdistros.mips.com/LinuxDistro/nanomips/'
- 'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
- 'generic_nano32r6el_page64k_dbg.xz')
- kernel_hash = '18d1c68f2e23429e266ca39ba5349ccd0aeb7180'
- kernel_path_xz = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
- self.do_test_mips_malta32el_nanomips(kernel_path_xz)
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 2b2d8953aa..13de54a1a8 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -35,6 +35,7 @@ test_timeouts = {
'arm_sx1' : 360,
'intel_iommu': 300,
'mips_malta' : 120,
+ 'mipsel_replay' : 480,
'netdev_ethtool' : 180,
'ppc_40p' : 240,
'ppc64_hv' : 1000,
@@ -161,6 +162,7 @@ tests_mips_system_thorough = [
tests_mipsel_system_thorough = [
'mipsel_malta',
+ 'mipsel_replay',
'mipsel_tuxrun',
]
diff --git a/tests/functional/test_mipsel_replay.py b/tests/functional/test_mipsel_replay.py
new file mode 100644
index 0000000000..0a330de43f
--- /dev/null
+++ b/tests/functional/test_mipsel_replay.py
@@ -0,0 +1,54 @@
+#!/usr/bin/env python3
+#
+# Replay tests for the little-endian 32-bit MIPS Malta board
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+from qemu_test import Asset, wait_for_console_pattern, skipSlowTest
+from replay_kernel import ReplayKernelBase
+
+
+class MipselReplay(ReplayKernelBase):
+
+ ASSET_KERNEL_4K = Asset(
+ ('http://mipsdistros.mips.com/LinuxDistro/nanomips/'
+ 'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
+ 'generic_nano32r6el_page4k.xz'),
+ '019e034094ac6cf3aa77df5e130fb023ce4dbc804b04bfcc560c6403e1ae6bdb')
+ ASSET_KERNEL_16K = Asset(
+ ('http://mipsdistros.mips.com/LinuxDistro/nanomips/'
+ 'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
+ 'generic_nano32r6el_page16k_up.xz'),
+ '3a54a10b3108c16a448dca9ea3db378733a27423befc2a45a5bdf990bd85e12c')
+ ASSET_KERNEL_64K = Asset(
+ ('http://mipsdistros.mips.com/LinuxDistro/nanomips/'
+ 'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
+ 'generic_nano32r6el_page64k_dbg.xz'),
+ 'ce21ff4b07a981ecb8a39db2876616f5a2473eb2ab459c6f67465b9914b0c6b6')
+
+ def do_test_replay_mips_malta32el_nanomips(self, kernel_asset):
+ self.set_machine('malta')
+ self.cpu = 'I7200'
+ kernel_path = self.uncompress(kernel_asset)
+
+ kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
+ 'mem=256m@@0x0 '
+ 'console=ttyS0')
+ console_pattern = 'Kernel command line: %s' % kernel_command_line
+ self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5)
+
+ @skipSlowTest()
+ def test_replay_mips_malta32el_nanomips_4k(self):
+ self.do_test_replay_mips_malta32el_nanomips(self.ASSET_KERNEL_4K)
+
+ @skipSlowTest()
+ def test_replay_mips_malta32el_nanomips_16k_up(self):
+ self.do_test_replay_mips_malta32el_nanomips(self.ASSET_KERNEL_16K)
+
+ @skipSlowTest()
+ def test_replay_mips_malta32el_nanomips_64k_dbg(self):
+ self.do_test_replay_mips_malta32el_nanomips(self.ASSET_KERNEL_64K)
+
+
+if __name__ == '__main__':
+ ReplayKernelBase.main()
--
2.48.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* Re: [PATCH v2 3/5] tests/functional/test_mipsel_malta: Convert the mipsel replay tests
2025-01-28 15:28 ` [PATCH v2 3/5] tests/functional/test_mipsel_malta: Convert the mipsel replay tests Thomas Huth
@ 2025-01-29 13:50 ` Daniel P. Berrangé
0 siblings, 0 replies; 10+ messages in thread
From: Daniel P. Berrangé @ 2025-01-29 13:50 UTC (permalink / raw)
To: Thomas Huth
Cc: Pavel Dovgalyuk, Paolo Bonzini, Jiaxun Yang, qemu-devel,
Philippe Mathieu-Daudé, Hao Wu, Tyrone Ting, Leif Lindholm,
Radoslaw Biernacki, Peter Maydell
On Tue, Jan 28, 2025 at 04:28:37PM +0100, Thomas Huth wrote:
> Move the mipsel replay tests from tests/avocado/replay_kernel.py to
> the functional framework. Since the functional tests should be run per
> target, we cannot stick all replay tests in one file. Thus let's add
> these tests to a new, separate file there instead.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/avocado/replay_kernel.py | 54 --------------------------
> tests/functional/meson.build | 2 +
> tests/functional/test_mipsel_replay.py | 54 ++++++++++++++++++++++++++
> 3 files changed, 56 insertions(+), 54 deletions(-)
> create mode 100644 tests/functional/test_mipsel_replay.py
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v2 4/5] tests/functional/test_mips64el_malta: Convert the mips64el replay tests
2025-01-28 15:28 [PATCH v2 0/5] Convert the MIPS replay tests to the functional framework Thomas Huth
` (2 preceding siblings ...)
2025-01-28 15:28 ` [PATCH v2 3/5] tests/functional/test_mipsel_malta: Convert the mipsel replay tests Thomas Huth
@ 2025-01-28 15:28 ` Thomas Huth
2025-01-29 13:51 ` Daniel P. Berrangé
2025-01-28 15:28 ` [PATCH v2 5/5] tests/functional/test_mips_malta: Convert the mips big endian " Thomas Huth
4 siblings, 1 reply; 10+ messages in thread
From: Thomas Huth @ 2025-01-28 15:28 UTC (permalink / raw)
To: Pavel Dovgalyuk, Paolo Bonzini, Jiaxun Yang, qemu-devel
Cc: Philippe Mathieu-Daudé, Daniel P. Berrangé, Hao Wu,
Tyrone Ting, Leif Lindholm, Radoslaw Biernacki, Peter Maydell
Move the mips64el replay tests from tests/avocado/replay_kernel.py to
the functional framework. Since the functional tests should be run per
target, we cannot stick all replay tests in one file. Thus let's add
these tests to a separate file there now.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/avocado/replay_kernel.py | 57 ----------------------
tests/functional/meson.build | 1 +
tests/functional/test_mips64el_replay.py | 60 ++++++++++++++++++++++++
3 files changed, 61 insertions(+), 57 deletions(-)
create mode 100755 tests/functional/test_mips64el_replay.py
diff --git a/tests/avocado/replay_kernel.py b/tests/avocado/replay_kernel.py
index 9a3f175129..89a0c0b997 100644
--- a/tests/avocado/replay_kernel.py
+++ b/tests/avocado/replay_kernel.py
@@ -146,33 +146,6 @@ def test_mips_malta(self):
self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5)
- def test_mips64el_malta(self):
- """
- This test requires the ar tool to extract "data.tar.gz" from
- the Debian package.
-
- The kernel can be rebuilt using this Debian kernel source [1] and
- following the instructions on [2].
-
- [1] http://snapshot.debian.org/package/linux-2.6/2.6.32-48/
- #linux-source-2.6.32_2.6.32-48
- [2] https://kernel-team.pages.debian.net/kernel-handbook/
- ch-common-tasks.html#s-common-official
-
- :avocado: tags=arch:mips64el
- :avocado: tags=machine:malta
- """
- deb_url = ('http://snapshot.debian.org/archive/debian/'
- '20130217T032700Z/pool/main/l/linux-2.6/'
- 'linux-image-2.6.32-5-5kc-malta_2.6.32-48_mipsel.deb')
- deb_hash = '1aaec92083bf22fda31e0d27fa8d9a388e5fc3d5'
- deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
- kernel_path = self.extract_from_deb(deb_path,
- '/boot/vmlinux-2.6.32-5-5kc-malta')
- kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
- console_pattern = 'Kernel command line: %s' % kernel_command_line
- self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5)
-
def test_aarch64_virt(self):
"""
:avocado: tags=arch:aarch64
@@ -475,33 +448,3 @@ def test_mips_malta_cpio(self):
console_pattern = 'Boot successful.'
self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5,
args=('-initrd', initrd_path))
-
- @skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
- def test_mips64el_malta_5KEc_cpio(self):
- """
- :avocado: tags=arch:mips64el
- :avocado: tags=machine:malta
- :avocado: tags=endian:little
- :avocado: tags=slowness:high
- :avocado: tags=cpu:5KEc
- """
- kernel_url = ('https://github.com/philmd/qemu-testing-blob/'
- 'raw/9ad2df38/mips/malta/mips64el/'
- 'vmlinux-3.19.3.mtoman.20150408')
- kernel_hash = '00d1d268fb9f7d8beda1de6bebcc46e884d71754'
- kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
- initrd_url = ('https://github.com/groeck/linux-build-test/'
- 'raw/8584a59e/rootfs/'
- 'mipsel64/rootfs.mipsel64r1.cpio.gz')
- initrd_hash = '1dbb8a396e916847325284dbe2151167'
- initrd_path_gz = self.fetch_asset(initrd_url, algorithm='md5',
- asset_hash=initrd_hash)
- initrd_path = self.workdir + "rootfs.cpio"
- archive.gzip_uncompress(initrd_path_gz, initrd_path)
-
- kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
- 'console=ttyS0 console=tty '
- 'rdinit=/sbin/init noreboot')
- console_pattern = 'Boot successful.'
- self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5,
- args=('-initrd', initrd_path))
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 13de54a1a8..0a7a382ef6 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -174,6 +174,7 @@ tests_mips64el_system_thorough = [
'mips64el_fuloong2e',
'mips64el_loongson3v',
'mips64el_malta',
+ 'mips64el_replay',
'mips64el_tuxrun',
]
diff --git a/tests/functional/test_mips64el_replay.py b/tests/functional/test_mips64el_replay.py
new file mode 100755
index 0000000000..4f63d7fb34
--- /dev/null
+++ b/tests/functional/test_mips64el_replay.py
@@ -0,0 +1,60 @@
+#!/usr/bin/env python3
+#
+# Replay tests for the little-endian 64-bit MIPS Malta board
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+import os
+import logging
+
+from qemu_test import Asset, exec_command_and_wait_for_pattern
+from qemu_test import skipIfMissingImports, skipFlakyTest, skipUntrustedTest
+from replay_kernel import ReplayKernelBase
+
+
+class Mips64elReplay(ReplayKernelBase):
+
+ ASSET_KERNEL_2_63_2 = Asset(
+ ('http://snapshot.debian.org/archive/debian/'
+ '20130217T032700Z/pool/main/l/linux-2.6/'
+ 'linux-image-2.6.32-5-5kc-malta_2.6.32-48_mipsel.deb'),
+ '35eb476f03be589824b0310358f1c447d85e645b88cbcd2ac02b97ef560f9f8d')
+
+ def test_replay_mips64el_malta(self):
+ self.set_machine('malta')
+ kernel_path = self.archive_extract(self.ASSET_KERNEL_2_63_2,
+ member='boot/vmlinux-2.6.32-5-5kc-malta')
+ kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
+ console_pattern = 'Kernel command line: %s' % kernel_command_line
+ self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5)
+
+
+ ASSET_KERNEL_3_19_3 = Asset(
+ ('https://github.com/philmd/qemu-testing-blob/'
+ 'raw/9ad2df38/mips/malta/mips64el/'
+ 'vmlinux-3.19.3.mtoman.20150408'),
+ '8d3beb003bc66051ead98e7172139017fcf9ce2172576541c57e86418dfa5ab8')
+
+ ASSET_CPIO_R1 = Asset(
+ ('https://github.com/groeck/linux-build-test/'
+ 'raw/8584a59e/rootfs/mipsel64/'
+ 'rootfs.mipsel64r1.cpio.gz'),
+ '75ba10cd35fb44e32948eeb26974f061b703c81c4ba2fab1ebcacf1d1bec3b61')
+
+ @skipUntrustedTest()
+ def test_replay_mips64el_malta_5KEc_cpio(self):
+ self.set_machine('malta')
+ self.cpu = '5KEc'
+ kernel_path = self.ASSET_KERNEL_3_19_3.fetch()
+ initrd_path = self.uncompress(self.ASSET_CPIO_R1)
+
+ kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
+ 'console=ttyS0 console=tty '
+ 'rdinit=/sbin/init noreboot')
+ console_pattern = 'Boot successful.'
+ self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5,
+ args=('-initrd', initrd_path))
+
+
+if __name__ == '__main__':
+ ReplayKernelBase.main()
--
2.48.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* Re: [PATCH v2 4/5] tests/functional/test_mips64el_malta: Convert the mips64el replay tests
2025-01-28 15:28 ` [PATCH v2 4/5] tests/functional/test_mips64el_malta: Convert the mips64el " Thomas Huth
@ 2025-01-29 13:51 ` Daniel P. Berrangé
0 siblings, 0 replies; 10+ messages in thread
From: Daniel P. Berrangé @ 2025-01-29 13:51 UTC (permalink / raw)
To: Thomas Huth
Cc: Pavel Dovgalyuk, Paolo Bonzini, Jiaxun Yang, qemu-devel,
Philippe Mathieu-Daudé, Hao Wu, Tyrone Ting, Leif Lindholm,
Radoslaw Biernacki, Peter Maydell
On Tue, Jan 28, 2025 at 04:28:38PM +0100, Thomas Huth wrote:
> Move the mips64el replay tests from tests/avocado/replay_kernel.py to
> the functional framework. Since the functional tests should be run per
> target, we cannot stick all replay tests in one file. Thus let's add
> these tests to a separate file there now.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/avocado/replay_kernel.py | 57 ----------------------
> tests/functional/meson.build | 1 +
> tests/functional/test_mips64el_replay.py | 60 ++++++++++++++++++++++++
> 3 files changed, 61 insertions(+), 57 deletions(-)
> create mode 100755 tests/functional/test_mips64el_replay.py
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v2 5/5] tests/functional/test_mips_malta: Convert the mips big endian replay tests
2025-01-28 15:28 [PATCH v2 0/5] Convert the MIPS replay tests to the functional framework Thomas Huth
` (3 preceding siblings ...)
2025-01-28 15:28 ` [PATCH v2 4/5] tests/functional/test_mips64el_malta: Convert the mips64el " Thomas Huth
@ 2025-01-28 15:28 ` Thomas Huth
2025-01-29 13:51 ` Daniel P. Berrangé
4 siblings, 1 reply; 10+ messages in thread
From: Thomas Huth @ 2025-01-28 15:28 UTC (permalink / raw)
To: Pavel Dovgalyuk, Paolo Bonzini, Jiaxun Yang, qemu-devel
Cc: Philippe Mathieu-Daudé, Daniel P. Berrangé, Hao Wu,
Tyrone Ting, Leif Lindholm, Radoslaw Biernacki, Peter Maydell
Move the mips big endian replay tests from tests/avocado/replay_kernel.py
to the functional framework. Since the functional tests should be run per
target, we cannot stick all replay tests in one file. Thus let's add
these tests to a separate file now.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/avocado/replay_kernel.py | 54 ---------------------------
tests/functional/meson.build | 1 +
tests/functional/test_mips_replay.py | 55 ++++++++++++++++++++++++++++
3 files changed, 56 insertions(+), 54 deletions(-)
create mode 100755 tests/functional/test_mips_replay.py
diff --git a/tests/avocado/replay_kernel.py b/tests/avocado/replay_kernel.py
index 89a0c0b997..39dbaf5e4d 100644
--- a/tests/avocado/replay_kernel.py
+++ b/tests/avocado/replay_kernel.py
@@ -128,24 +128,6 @@ def test_x86_64_q35(self):
self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5)
- def test_mips_malta(self):
- """
- :avocado: tags=arch:mips
- :avocado: tags=machine:malta
- :avocado: tags=endian:big
- """
- deb_url = ('http://snapshot.debian.org/archive/debian/'
- '20130217T032700Z/pool/main/l/linux-2.6/'
- 'linux-image-2.6.32-5-4kc-malta_2.6.32-48_mips.deb')
- deb_hash = 'a8cfc28ad8f45f54811fc6cf74fc43ffcfe0ba04'
- deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
- kernel_path = self.extract_from_deb(deb_path,
- '/boot/vmlinux-2.6.32-5-4kc-malta')
- kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
- console_pattern = 'Kernel command line: %s' % kernel_command_line
-
- self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5)
-
def test_aarch64_virt(self):
"""
:avocado: tags=arch:aarch64
@@ -412,39 +394,3 @@ def test_xtensa_lx60(self):
'/qac-best-of-multiarch/download/day02.tar.xz')
file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
self.do_test_advcal_2018(file_path, 'santas-sleigh-ride.elf')
-
-@skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
-class ReplayKernelSlow(ReplayKernelBase):
- # Override the timeout, because this kernel includes an inner
- # loop which is executed with TB recompilings during replay,
- # making it very slow.
- timeout = 180
-
- def test_mips_malta_cpio(self):
- """
- :avocado: tags=arch:mips
- :avocado: tags=machine:malta
- :avocado: tags=endian:big
- :avocado: tags=slowness:high
- """
- deb_url = ('http://snapshot.debian.org/archive/debian/'
- '20160601T041800Z/pool/main/l/linux/'
- 'linux-image-4.5.0-2-4kc-malta_4.5.5-1_mips.deb')
- deb_hash = 'a3c84f3e88b54e06107d65a410d1d1e8e0f340f8'
- deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
- kernel_path = self.extract_from_deb(deb_path,
- '/boot/vmlinux-4.5.0-2-4kc-malta')
- initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
- '8584a59ed9e5eb5ee7ca91f6d74bbb06619205b8/rootfs/'
- 'mips/rootfs.cpio.gz')
- initrd_hash = 'bf806e17009360a866bf537f6de66590de349a99'
- initrd_path_gz = self.fetch_asset(initrd_url, asset_hash=initrd_hash)
- initrd_path = self.workdir + "rootfs.cpio"
- archive.gzip_uncompress(initrd_path_gz, initrd_path)
-
- kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
- 'console=ttyS0 console=tty '
- 'rdinit=/sbin/init noreboot')
- console_pattern = 'Boot successful.'
- self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5,
- args=('-initrd', initrd_path))
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 0a7a382ef6..164699b6c4 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -157,6 +157,7 @@ tests_microblazeel_system_thorough = [
tests_mips_system_thorough = [
'mips_malta',
+ 'mips_replay',
'mips_tuxrun',
]
diff --git a/tests/functional/test_mips_replay.py b/tests/functional/test_mips_replay.py
new file mode 100755
index 0000000000..eda031ccad
--- /dev/null
+++ b/tests/functional/test_mips_replay.py
@@ -0,0 +1,55 @@
+#!/usr/bin/env python3
+#
+# Replay tests for the big-endian 32-bit MIPS Malta board
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+from qemu_test import Asset, skipSlowTest, exec_command_and_wait_for_pattern
+from replay_kernel import ReplayKernelBase
+
+
+class MipsReplay(ReplayKernelBase):
+
+ ASSET_KERNEL_2_63_2 = Asset(
+ ('http://snapshot.debian.org/archive/debian/'
+ '20130217T032700Z/pool/main/l/linux-2.6/'
+ 'linux-image-2.6.32-5-4kc-malta_2.6.32-48_mips.deb'),
+ '16ca524148afb0626f483163e5edf352bc1ab0e4fc7b9f9d473252762f2c7a43')
+
+ def test_replay_mips_malta(self):
+ self.set_machine('malta')
+ kernel_path = self.archive_extract(self.ASSET_KERNEL_2_63_2,
+ member='boot/vmlinux-2.6.32-5-4kc-malta')
+ kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
+ console_pattern = 'Kernel command line: %s' % kernel_command_line
+ self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5)
+
+ ASSET_KERNEL_4_5_0 = Asset(
+ ('http://snapshot.debian.org/archive/debian/'
+ '20160601T041800Z/pool/main/l/linux/'
+ 'linux-image-4.5.0-2-4kc-malta_4.5.5-1_mips.deb'),
+ '526b17d5889840888b76fc2c36a0ebde182c9b1410a3a1e68203c3b160eb2027')
+
+ ASSET_INITRD = Asset(
+ ('https://github.com/groeck/linux-build-test/raw/'
+ '8584a59ed9e5eb5ee7ca91f6d74bbb06619205b8/rootfs/'
+ 'mips/rootfs.cpio.gz'),
+ 'dcfe3a7fe3200da3a00d176b95caaa086495eb158f2bff64afc67d7e1eb2cddc')
+
+ @skipSlowTest()
+ def test_replay_mips_malta_cpio(self):
+ self.set_machine('malta')
+ kernel_path = self.archive_extract(self.ASSET_KERNEL_4_5_0,
+ member='boot/vmlinux-4.5.0-2-4kc-malta')
+ initrd_path = self.uncompress(self.ASSET_INITRD)
+
+ kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
+ 'console=ttyS0 console=tty '
+ 'rdinit=/sbin/init noreboot')
+ console_pattern = 'Boot successful.'
+ self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5,
+ args=('-initrd', initrd_path))
+
+
+if __name__ == '__main__':
+ ReplayKernelBase.main()
--
2.48.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* Re: [PATCH v2 5/5] tests/functional/test_mips_malta: Convert the mips big endian replay tests
2025-01-28 15:28 ` [PATCH v2 5/5] tests/functional/test_mips_malta: Convert the mips big endian " Thomas Huth
@ 2025-01-29 13:51 ` Daniel P. Berrangé
0 siblings, 0 replies; 10+ messages in thread
From: Daniel P. Berrangé @ 2025-01-29 13:51 UTC (permalink / raw)
To: Thomas Huth
Cc: Pavel Dovgalyuk, Paolo Bonzini, Jiaxun Yang, qemu-devel,
Philippe Mathieu-Daudé, Hao Wu, Tyrone Ting, Leif Lindholm,
Radoslaw Biernacki, Peter Maydell
On Tue, Jan 28, 2025 at 04:28:39PM +0100, Thomas Huth wrote:
> Move the mips big endian replay tests from tests/avocado/replay_kernel.py
> to the functional framework. Since the functional tests should be run per
> target, we cannot stick all replay tests in one file. Thus let's add
> these tests to a separate file now.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/avocado/replay_kernel.py | 54 ---------------------------
> tests/functional/meson.build | 1 +
> tests/functional/test_mips_replay.py | 55 ++++++++++++++++++++++++++++
> 3 files changed, 56 insertions(+), 54 deletions(-)
> create mode 100755 tests/functional/test_mips_replay.py
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 10+ messages in thread