* [PATCH v4 01/35] tests/avocado: machine aarch64: standardize location and RO access
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 9:30 ` Philippe Mathieu-Daudé
2024-08-21 8:27 ` [PATCH v4 02/35] tests/avocado/boot_xen.py: fetch kernel during test setUp() Thomas Huth
` (33 subsequent siblings)
34 siblings, 1 reply; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
From: Cleber Rosa <crosa@redhat.com>
The tests under machine_aarch64_virt.py and machine_aarch64_sbsaref.py
should not be writing to the ISO files. By adding "media=cdrom" the
"ro" is automatically set.
While at it, let's use a single code style and hash for the ISO url.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Message-ID: <20240806173119.582857-5-crosa@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/avocado/machine_aarch64_sbsaref.py | 2 +-
tests/avocado/machine_aarch64_virt.py | 14 +++++++-------
2 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/tests/avocado/machine_aarch64_sbsaref.py b/tests/avocado/machine_aarch64_sbsaref.py
index e920bbf08c..756f316ac9 100644
--- a/tests/avocado/machine_aarch64_sbsaref.py
+++ b/tests/avocado/machine_aarch64_sbsaref.py
@@ -129,7 +129,7 @@ def boot_alpine_linux(self, cpu):
"-cpu",
cpu,
"-drive",
- f"file={iso_path},format=raw",
+ f"file={iso_path},media=cdrom,format=raw",
)
self.vm.launch()
diff --git a/tests/avocado/machine_aarch64_virt.py b/tests/avocado/machine_aarch64_virt.py
index a90dc6ff4b..6831d2c0ed 100644
--- a/tests/avocado/machine_aarch64_virt.py
+++ b/tests/avocado/machine_aarch64_virt.py
@@ -37,13 +37,13 @@ def test_alpine_virt_tcg_gic_max(self):
:avocado: tags=machine:virt
:avocado: tags=accel:tcg
"""
- iso_url = ('https://dl-cdn.alpinelinux.org/'
- 'alpine/v3.17/releases/aarch64/'
- 'alpine-standard-3.17.2-aarch64.iso')
+ iso_url = (
+ "https://dl-cdn.alpinelinux.org/"
+ "alpine/v3.17/releases/aarch64/alpine-standard-3.17.2-aarch64.iso"
+ )
- # Alpine use sha256 so I recalculated this myself
- iso_sha1 = '76284fcd7b41fe899b0c2375ceb8470803eea839'
- iso_path = self.fetch_asset(iso_url, asset_hash=iso_sha1)
+ iso_hash = "5a36304ecf039292082d92b48152a9ec21009d3a62f459de623e19c4bd9dc027"
+ iso_path = self.fetch_asset(iso_url, algorithm="sha256", asset_hash=iso_hash)
self.vm.set_console()
kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
@@ -60,7 +60,7 @@ def test_alpine_virt_tcg_gic_max(self):
self.vm.add_args("-smp", "2", "-m", "1024")
self.vm.add_args('-bios', os.path.join(BUILD_DIR, 'pc-bios',
'edk2-aarch64-code.fd'))
- self.vm.add_args("-drive", f"file={iso_path},format=raw")
+ self.vm.add_args("-drive", f"file={iso_path},media=cdrom,format=raw")
self.vm.add_args('-device', 'virtio-rng-pci,rng=rng0')
self.vm.add_args('-object', 'rng-random,id=rng0,filename=/dev/urandom')
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* Re: [PATCH v4 01/35] tests/avocado: machine aarch64: standardize location and RO access
2024-08-21 8:27 ` [PATCH v4 01/35] tests/avocado: machine aarch64: standardize location and RO access Thomas Huth
@ 2024-08-21 9:30 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-21 9:30 UTC (permalink / raw)
To: Thomas Huth, Alex Bennée, qemu-devel, Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On 21/8/24 10:27, Thomas Huth wrote:
> From: Cleber Rosa <crosa@redhat.com>
>
> The tests under machine_aarch64_virt.py and machine_aarch64_sbsaref.py
> should not be writing to the ISO files. By adding "media=cdrom" the
> "ro" is automatically set.
>
> While at it, let's use a single code style and hash for the ISO url.
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> Tested-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
> Message-ID: <20240806173119.582857-5-crosa@redhat.com>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/avocado/machine_aarch64_sbsaref.py | 2 +-
> tests/avocado/machine_aarch64_virt.py | 14 +++++++-------
> 2 files changed, 8 insertions(+), 8 deletions(-)
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
^ permalink raw reply [flat|nested] 69+ messages in thread
* [PATCH v4 02/35] tests/avocado/boot_xen.py: fetch kernel during test setUp()
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
2024-08-21 8:27 ` [PATCH v4 01/35] tests/avocado: machine aarch64: standardize location and RO access Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 8:27 ` [PATCH v4 03/35] tests/avocado/machine_aarch64_sbsaref.py: allow for rw usage of image Thomas Huth
` (32 subsequent siblings)
34 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
From: Cleber Rosa <crosa@redhat.com>
The kernel is a common blob used in all tests. By moving it to the
setUp() method, the "fetch asset" plugin will recognize the kernel and
attempt to fetch it and cache it before the tests are started.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Tested-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Message-ID: <20240806173119.582857-7-crosa@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/avocado/boot_xen.py | 13 ++++++-------
1 file changed, 6 insertions(+), 7 deletions(-)
diff --git a/tests/avocado/boot_xen.py b/tests/avocado/boot_xen.py
index f29bc58b9e..490a127a3e 100644
--- a/tests/avocado/boot_xen.py
+++ b/tests/avocado/boot_xen.py
@@ -30,23 +30,22 @@ class BootXen(LinuxKernelTest):
timeout = 90
XEN_COMMON_COMMAND_LINE = 'dom0_mem=128M loglvl=all guest_loglvl=all'
- def fetch_guest_kernel(self):
+ def setUp(self):
+ super(BootXen, self).setUp()
+
# Using my own built kernel - which works
kernel_url = ('https://fileserver.linaro.org/'
's/JSsewXGZ6mqxPr5/download?path=%2F&files='
'linux-5.9.9-arm64-ajb')
kernel_sha1 = '4f92bc4b9f88d5ab792fa7a43a68555d344e1b83'
- kernel_path = self.fetch_asset(kernel_url,
- asset_hash=kernel_sha1)
-
- return kernel_path
+ self.kernel_path = self.fetch_asset(kernel_url,
+ asset_hash=kernel_sha1)
def launch_xen(self, xen_path):
"""
Launch Xen with a dom0 guest kernel
"""
self.log.info("launch with xen_path: %s", xen_path)
- kernel_path = self.fetch_guest_kernel()
self.vm.set_console()
@@ -56,7 +55,7 @@ def launch_xen(self, xen_path):
'-append', self.XEN_COMMON_COMMAND_LINE,
'-device',
'guest-loader,addr=0x47000000,kernel=%s,bootargs=console=hvc0'
- % (kernel_path))
+ % (self.kernel_path))
self.vm.launch()
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH v4 03/35] tests/avocado/machine_aarch64_sbsaref.py: allow for rw usage of image
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
2024-08-21 8:27 ` [PATCH v4 01/35] tests/avocado: machine aarch64: standardize location and RO access Thomas Huth
2024-08-21 8:27 ` [PATCH v4 02/35] tests/avocado/boot_xen.py: fetch kernel during test setUp() Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 8:27 ` [PATCH v4 04/35] Bump avocado to 103.0 Thomas Huth
` (31 subsequent siblings)
34 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
From: Cleber Rosa <crosa@redhat.com>
When the OpenBSD based tests are run in parallel, the previously
single instance of the image would become corrupt. Let's give each
test its own snapshot.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Suggested-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Message-ID: <20240806173119.582857-9-crosa@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/avocado/machine_aarch64_sbsaref.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/avocado/machine_aarch64_sbsaref.py b/tests/avocado/machine_aarch64_sbsaref.py
index 756f316ac9..f8bf40c192 100644
--- a/tests/avocado/machine_aarch64_sbsaref.py
+++ b/tests/avocado/machine_aarch64_sbsaref.py
@@ -190,7 +190,7 @@ def boot_openbsd73(self, cpu):
"-cpu",
cpu,
"-drive",
- f"file={img_path},format=raw",
+ f"file={img_path},format=raw,snapshot=on",
)
self.vm.launch()
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH v4 04/35] Bump avocado to 103.0
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (2 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 03/35] tests/avocado/machine_aarch64_sbsaref.py: allow for rw usage of image Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 10:34 ` Philippe Mathieu-Daudé
2024-08-21 8:27 ` [PATCH v4 05/35] tests/avocado/avocado_qemu: Fix the "from" statements in linuxtest.py Thomas Huth
` (30 subsequent siblings)
34 siblings, 1 reply; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
From: Cleber Rosa <crosa@redhat.com>
This bumps Avocado to latest the LTS release.
An LTS release is one that can receive bugfixes and guarantees
stability for a much longer period and has incremental minor releases
made.
Even though the 103.0 LTS release is pretty a rewrite of Avocado when
compared to 88.1, the behavior of all existing tests under
tests/avocado has been extensively tested no regression in behavior
was found.
To keep behavior of jobs as close as possible with previous version,
this version bump keeps the execution serial (maximum of one task at a
time being run).
Reference: https://avocado-framework.readthedocs.io/en/103.0/releases/lts/103_0.html
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Message-ID: <20240806173119.582857-2-crosa@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
pythondeps.toml | 2 +-
tests/Makefile.include | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/pythondeps.toml b/pythondeps.toml
index f6e590fdd8..175cf99241 100644
--- a/pythondeps.toml
+++ b/pythondeps.toml
@@ -30,5 +30,5 @@ sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.1.1" }
# Note that qemu.git/python/ is always implicitly installed.
# Prefer an LTS version when updating the accepted versions of
# avocado-framework, for example right now the limit is 92.x.
-avocado-framework = { accepted = "(>=88.1, <93.0)", installed = "88.1", canary = "avocado" }
+avocado-framework = { accepted = "(>=103.0, <104.0)", installed = "103.0", canary = "avocado" }
pycdlib = { accepted = ">=1.11.0" }
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 6618bfed70..537804d101 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -141,7 +141,7 @@ check-avocado: check-venv $(TESTS_RESULTS_DIR) get-vm-images
--show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
$(if $(AVOCADO_TAGS),, --filter-by-tags-include-empty \
--filter-by-tags-include-empty-key) \
- $(AVOCADO_CMDLINE_TAGS) \
+ $(AVOCADO_CMDLINE_TAGS) --max-parallel-tasks=1 \
$(if $(GITLAB_CI),,--failfast) $(AVOCADO_TESTS), \
"AVOCADO", "tests/avocado")
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* Re: [PATCH v4 04/35] Bump avocado to 103.0
2024-08-21 8:27 ` [PATCH v4 04/35] Bump avocado to 103.0 Thomas Huth
@ 2024-08-21 10:34 ` Philippe Mathieu-Daudé
2024-08-29 9:45 ` Daniel P. Berrangé
0 siblings, 1 reply; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-21 10:34 UTC (permalink / raw)
To: Thomas Huth, Alex Bennée, qemu-devel, Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On 21/8/24 10:27, Thomas Huth wrote:
> From: Cleber Rosa <crosa@redhat.com>
>
> This bumps Avocado to latest the LTS release.
>
> An LTS release is one that can receive bugfixes and guarantees
> stability for a much longer period and has incremental minor releases
> made.
>
> Even though the 103.0 LTS release is pretty a rewrite of Avocado when
> compared to 88.1, the behavior of all existing tests under
> tests/avocado has been extensively tested no regression in behavior
> was found.
>
> To keep behavior of jobs as close as possible with previous version,
> this version bump keeps the execution serial (maximum of one task at a
> time being run).
>
> Reference: https://avocado-framework.readthedocs.io/en/103.0/releases/lts/103_0.html
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> Tested-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
> Message-ID: <20240806173119.582857-2-crosa@redhat.com>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> pythondeps.toml | 2 +-
> tests/Makefile.include | 2 +-
> 2 files changed, 2 insertions(+), 2 deletions(-)
I suppose we now need an ultra wide monitor and set the terminal
COLUMNS to a value >= 512 to not make sense of the spaghetti console
now (my laptop terminal is 48x150 full screen, I might need new
eyes). (Compare with commit 44055caaa5 description). See for example
some lines are 267 columns wide:
$ make check-avocado AVOCADO_TAGS='machine:ppce500 machine:mpc8544ds'
AVOCADO_SHOW='app,console'
AVOCADO tests/avocado
Fetching asset from
tests/avocado/replay_kernel.py:ReplayKernelNormal.test_ppc64_e500
JOB ID : 793c892e4e11736f8ed5a7f8450bdba1c0b6187f
JOB LOG :
/Users/philmd/source/qemu/build.mac/full_clang/tests/results/job-2024-08-21T12.24-793c892/job.log
(1/3)
tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
STARTED
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,186 console __init__ L0151 DEBUG| MMU:
Supported page sizes
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,186 console __init__ L0151 DEBUG| 4 KB as direct
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,186 console __init__ L0151 DEBUG| 4096 KB as
direct
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,186 console __init__ L0151 DEBUG| 16384 KB
as direct
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,187 console __init__ L0151 DEBUG| 65536 KB
as direct
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,187 console __init__ L0151 DEBUG| 262144 KB
as direct
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,187 console __init__ L0151 DEBUG| 1048576 KB
as direct
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,187 console __init__ L0151 DEBUG| MMU:
Book3E HW tablewalk not supported
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,187 console __init__ L0151 DEBUG| Linux
version 4.16.13 (thuth@thuth.remote.csb) (gcc version 6.4.0 (Buildroot
2018.05.2)) #6 SMP Sat Dec 15 14:53:04 CET 2018
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,188 console __init__ L0151 DEBUG| Using QEMU
e500 machine description
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,188 console __init__ L0151 DEBUG|
bootconsole [udbg0] enabled
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,188 console __init__ L0151 DEBUG| CPU maps
initialized for 1 thread per core
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,188 console __init__ L0151 DEBUG|
-----------------------------------------------------
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,188 console __init__ L0151 DEBUG|
phys_mem_size = 0x8000000
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,189 console __init__ L0151 DEBUG|
dcache_bsize = 0x40
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,189 console __init__ L0151 DEBUG|
icache_bsize = 0x40
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,189 console __init__ L0151 DEBUG|
cpu_features = 0x00180400181802c0
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,189 console __init__ L0151 DEBUG| possible
= 0x00180480581802c8
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,189 console __init__ L0151 DEBUG| always
= 0x00180400581802c0
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,190 console __init__ L0151 DEBUG|
cpu_user_features = 0xcc008000 0x08000000
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,190 console __init__ L0151 DEBUG|
mmu_features = 0x000a0010
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,190 console __init__ L0151 DEBUG|
firmware_features = 0x0000000000000000
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,190 console __init__ L0151 DEBUG|
-----------------------------------------------------
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,190 console __init__ L0151 DEBUG|
qemu_e500_setup_arch()
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,191 console __init__ L0151 DEBUG| Zone ranges:
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,191 console __init__ L0151 DEBUG| DMA
[mem 0x0000000000000000-0x0000000007ffffff]
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,191 console __init__ L0151 DEBUG| DMA32 empty
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,191 console __init__ L0151 DEBUG| Normal empty
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,191 console __init__ L0151 DEBUG| Movable
zone start for each node
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,192 console __init__ L0151 DEBUG| Early
memory node ranges
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,192 console __init__ L0151 DEBUG| node 0:
[mem 0x0000000000000000-0x0000000007ffffff]
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,192 console __init__ L0151 DEBUG| Initmem
setup node 0 [mem 0x0000000000000000-0x0000000007ffffff]
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,192 console __init__ L0151 DEBUG| MMU:
Allocated 2112 bytes of context maps for 255 contexts
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,193 console __init__ L0151 DEBUG| percpu:
Embedded 22 pages/cpu @ (ptrval) s51672 r0 d38440 u1048576
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,195 console __init__ L0151 DEBUG| Built 1
zonelists, mobility grouping on. Total pages: 32320
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,195 console __init__ L0151 DEBUG| Kernel
command line:
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,196 console __init__ L0151 DEBUG| Dentry
cache hash table entries: 16384 (order: 5, 131072 bytes)
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,196 console __init__ L0151 DEBUG|
Inode-cache hash table entries: 8192 (order: 4, 65536 bytes)
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,220 console __init__ L0151 DEBUG| Memory:
41876K/131072K available (10088K kernel code, 1988K rwdata, 3292K
rodata, 4312K init, 320K bss, 89196K reserved, 0K cma-reserved)
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,223 console __init__ L0151 DEBUG| SLUB:
HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,229 console __init__ L0151 DEBUG|
Hierarchical RCU implementation.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,229 console __init__ L0151 DEBUG| RCU event
tracing is enabled.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,229 console __init__ L0151 DEBUG| RCU
restricting CPUs from NR_CPUS=24 to nr_cpu_ids=1.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,229 console __init__ L0151 DEBUG| RCU:
Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,231 console __init__ L0151 DEBUG| NR_IRQS:
512, nr_irqs: 512, preallocated irqs: 16
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,233 console __init__ L0151 DEBUG| mpic:
Setting up MPIC " OpenPIC " version 1.2 at fe0040000, max 1 CPUs
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,233 console __init__ L0151 DEBUG| mpic: ISU
size: 256, shift: 8, mask: ff
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,233 console __init__ L0151 DEBUG| mpic:
Initializing for 256 sources
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,236 console __init__ L0151 DEBUG|
clocksource: timebase: mask: 0xffffffffffffffff max_cycles:
0x5c4093a7d1, max_idle_ns: 440795210635 ns
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,237 console __init__ L0151 DEBUG|
clocksource: timebase mult[2800000] shift[24] registered
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,240 console __init__ L0151 DEBUG| Console:
colour dummy device 80x25
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,241 console __init__ L0151 DEBUG| pid_max:
default: 32768 minimum: 301
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,242 console __init__ L0151 DEBUG|
Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,242 console __init__ L0151 DEBUG|
Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes)
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,262 console __init__ L0151 DEBUG| e500
family performance monitor hardware support registered
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,263 console __init__ L0151 DEBUG|
Hierarchical SRCU implementation.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,265 console __init__ L0151 DEBUG| smp:
Bringing up secondary CPUs ...
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,266 console __init__ L0151 DEBUG| smp:
Brought up 1 node, 1 CPU
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,266 console __init__ L0151 DEBUG| Using
standard scheduler topology
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,272 console __init__ L0151 DEBUG| devtmpfs:
initialized
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,275 console __init__ L0151 DEBUG| random:
get_random_u32 called from .bucket_table_alloc+0x9c/0x234 with crng_init=0
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,277 console __init__ L0151 DEBUG|
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff,
max_idle_ns: 7645041785100000 ns
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,277 console __init__ L0151 DEBUG| futex hash
table entries: 256 (order: 2, 16384 bytes)
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,282 console __init__ L0151 DEBUG| NET:
Registered protocol family 16
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,284 console __init__ L0151 DEBUG| audit:
initializing netlink subsys (disabled)
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,297 console __init__ L0151 DEBUG| Found FSL
PCI host bridge at 0x0000000fe0008000. Firmware bus number: 0->255
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,298 console __init__ L0151 DEBUG| PCI host
bridge /pci@fe0008000 (primary) ranges:
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,298 console __init__ L0151 DEBUG| MEM
0x0000000c00000000..0x0000000c1fffffff -> 0x00000000e0000000
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,298 console __init__ L0151 DEBUG| IO
0x0000000fe1000000..0x0000000fe100ffff -> 0x0000000000000000
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,299 console __init__ L0151 DEBUG|
/pci@fe0008000: PCICSRBAR @ 0xdff00000
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,299 console __init__ L0151 DEBUG|
setup_pci_atmu: end of DRAM 8000000
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,302 console __init__ L0151 DEBUG| Machine:
QEMU ppce500
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,302 console __init__ L0151 DEBUG| SoC
family: QorIQ
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,302 console __init__ L0151 DEBUG| SoC ID:
svr:0x00000000, Revision: 0.0
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,306 console __init__ L0151 DEBUG| audit:
type=2000 audit(0.040:1): state=initialized audit_enabled=0 res=1
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,309 console __init__ L0151 DEBUG| fsl-pamu:
fsl_pamu_init: could not find a PAMU node
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,318 console __init__ L0151 DEBUG| PCI:
Probing PCI hardware
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,319 console __init__ L0151 DEBUG| fsl-pci
fe0008000.pci: PCI host bridge to bus 8000:00
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,320 console __init__ L0151 DEBUG| pci_bus
8000:00: root bus resource [io 0x8000080000010000-0x800008000001ffff]
(bus address [0x0000-0xffff])
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,320 console __init__ L0151 DEBUG| pci_bus
8000:00: root bus resource [mem 0xc00000000-0xc1fffffff] (bus address
[0xe0000000-0xffffffff])
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,320 console __init__ L0151 DEBUG| pci_bus
8000:00: root bus resource [bus 00-ff]
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,325 console __init__ L0151 DEBUG| pci
8000:00:01.0: BAR 6: assigned [mem 0xc00000000-0xc0003ffff pref]
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,326 console __init__ L0151 DEBUG| pci
8000:00:01.0: BAR 4: assigned [mem 0xc00040000-0xc00043fff 64bit pref]
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,326 console __init__ L0151 DEBUG| pci
8000:00:01.0: BAR 1: assigned [mem 0xc00044000-0xc00044fff]
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,327 console __init__ L0151 DEBUG| pci
8000:00:01.0: BAR 0: assigned [io 0x8000080000010000-0x800008000001001f]
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,349 console __init__ L0151 DEBUG| HugeTLB
registered 4.00 MiB page size, pre-allocated 0 pages
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,349 console __init__ L0151 DEBUG| HugeTLB
registered 16.0 MiB page size, pre-allocated 0 pages
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,349 console __init__ L0151 DEBUG| HugeTLB
registered 64.0 MiB page size, pre-allocated 0 pages
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,350 console __init__ L0151 DEBUG| HugeTLB
registered 256 MiB page size, pre-allocated 0 pages
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,350 console __init__ L0151 DEBUG| HugeTLB
registered 1.00 GiB page size, pre-allocated 0 pages
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,352 console __init__ L0151 DEBUG| Freescale
Elo series DMA driver
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,352 console __init__ L0151 DEBUG| vgaarb: loaded
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,353 console __init__ L0151 DEBUG| SCSI
subsystem initialized
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,355 console __init__ L0151 DEBUG| usbcore:
registered new interface driver usbfs
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,356 console __init__ L0151 DEBUG| usbcore:
registered new interface driver hub
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,356 console __init__ L0151 DEBUG| usbcore:
registered new device driver usb
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,356 console __init__ L0151 DEBUG| pps_core:
LinuxPPS API ver. 1 registered
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,356 console __init__ L0151 DEBUG| pps_core:
Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti
<giometti@linux.it>
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,356 console __init__ L0151 DEBUG| PTP clock
support registered
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,357 console __init__ L0151 DEBUG| EDAC MC:
Ver: 3.0.0
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,364 console __init__ L0151 DEBUG|
clocksource: Switched to clocksource timebase
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,419 console __init__ L0151 DEBUG| NET:
Registered protocol family 2
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,423 console __init__ L0151 DEBUG|
tcp_listen_portaddr_hash hash table entries: 256 (order: 0, 4096 bytes)
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,423 console __init__ L0151 DEBUG| TCP
established hash table entries: 1024 (order: 1, 8192 bytes)
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,423 console __init__ L0151 DEBUG| TCP bind
hash table entries: 1024 (order: 2, 16384 bytes)
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,424 console __init__ L0151 DEBUG| TCP: Hash
tables configured (established 1024 bind 1024)
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,424 console __init__ L0151 DEBUG| UDP hash
table entries: 256 (order: 1, 8192 bytes)
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,425 console __init__ L0151 DEBUG| UDP-Lite
hash table entries: 256 (order: 1, 8192 bytes)
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,426 console __init__ L0151 DEBUG| NET:
Registered protocol family 1
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,428 console __init__ L0151 DEBUG| RPC:
Registered named UNIX socket transport module.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,428 console __init__ L0151 DEBUG| RPC:
Registered udp transport module.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,429 console __init__ L0151 DEBUG| RPC:
Registered tcp transport module.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,429 console __init__ L0151 DEBUG| RPC:
Registered tcp NFSv4.1 backchannel transport module.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,582 console __init__ L0151 DEBUG| Initialise
system trusted keyrings
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,584 console __init__ L0151 DEBUG|
workingset: timestamp_bits=62 max_order=15 bucket_order=0
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,598 console __init__ L0151 DEBUG| NFS:
Registering the id_resolver key type
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,598 console __init__ L0151 DEBUG| Key type
id_resolver registered
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,598 console __init__ L0151 DEBUG| Key type
id_legacy registered
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,599 console __init__ L0151 DEBUG| Installing
knfsd (copyright (C) 1996 okir@monad.swb.de).
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,599 console __init__ L0151 DEBUG| ntfs:
driver 2.1.32 [Flags: R/O].
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,599 console __init__ L0151 DEBUG| jffs2:
version 2.2. (NAND) © 2001-2006 Red Hat, Inc.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,606 console __init__ L0151 DEBUG|
jitterentropy: Initialization failed with host not compliant with
requirements: 2
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,606 console __init__ L0151 DEBUG| Key type
asymmetric registered
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,606 console __init__ L0151 DEBUG| Asymmetric
key parser 'x509' registered
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,606 console __init__ L0151 DEBUG| io
scheduler noop registered
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,607 console __init__ L0151 DEBUG| io
scheduler deadline registered
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,607 console __init__ L0151 DEBUG| io
scheduler cfq registered (default)
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,607 console __init__ L0151 DEBUG| io
scheduler mq-deadline registered
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,607 console __init__ L0151 DEBUG| io
scheduler kyber registered
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,609 console __init__ L0151 DEBUG| virtio-pci
8000:00:01.0: enabling device (0000 -> 0003)
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,612 console __init__ L0151 DEBUG| Serial:
8250/16550 driver, 6 ports, IRQ sharing enabled
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,617 console __init__ L0151 DEBUG| console
[ttyS0] disabled
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,639 console __init__ L0151 DEBUG|
serial8250.0: ttyS0 at MMIO 0xfe0004500 (irq = 42, base_baud = 25000000)
is a 16550A
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,640 console __init__ L0151 DEBUG| console
[ttyS0] enabled
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,641 console __init__ L0151 DEBUG| console
[ttyS0] enabled
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,641 console __init__ L0151 DEBUG|
bootconsole [udbg0] disabled
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,641 console __init__ L0151 DEBUG|
bootconsole [udbg0] disabled
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,642 console __init__ L0151 DEBUG| ePAPR
hypervisor byte channel driver
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,660 console __init__ L0151 DEBUG| brd:
module loaded
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,669 console __init__ L0151 DEBUG| loop:
module loaded
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,682 console __init__ L0151 DEBUG| st:
Version 20160209, fixed bufsize 32768, s/g segs 256
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,689 console __init__ L0151 DEBUG| libphy:
Fixed MDIO Bus: probed
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,695 console __init__ L0151 DEBUG| e1000:
Intel(R) PRO/1000 Network Driver - version 7.3.21-k8-NAPI
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,695 console __init__ L0151 DEBUG| e1000:
Copyright (c) 1999-2006 Intel Corporation.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,696 console __init__ L0151 DEBUG| e1000e:
Intel(R) PRO/1000 Network Driver - 3.2.6-k
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,696 console __init__ L0151 DEBUG| e1000e:
Copyright(c) 1999 - 2015 Intel Corporation.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,696 console __init__ L0151 DEBUG| igb:
Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,696 console __init__ L0151 DEBUG| igb:
Copyright (c) 2007-2014 Intel Corporation.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,702 console __init__ L0151 DEBUG| ehci_hcd:
USB 2.0 'Enhanced' Host Controller (EHCI) Driver
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,702 console __init__ L0151 DEBUG| ehci-pci:
EHCI PCI platform driver
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,702 console __init__ L0151 DEBUG| ohci_hcd:
USB 1.1 'Open' Host Controller (OHCI) Driver
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,702 console __init__ L0151 DEBUG| ohci-pci:
OHCI PCI platform driver
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,703 console __init__ L0151 DEBUG| ehci-fsl:
Freescale EHCI Host controller driver
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,703 console __init__ L0151 DEBUG| usbcore:
registered new interface driver usb-storage
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,704 console __init__ L0151 DEBUG| mousedev:
PS/2 mouse device common for all mice
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,705 console __init__ L0151 DEBUG| i2c /dev
entries driver
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,706 console __init__ L0151 DEBUG| mpc-i2c
fe0003000.i2c: timeout 1000000 us
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,714 console __init__ L0151 DEBUG| rtc-ds1307
0-0068: registered as rtc0
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,716 console __init__ L0151 DEBUG|
Freescale(R) MPC85xx EDAC driver, (C) 2006 Montavista Software
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,717 console __init__ L0151 DEBUG| EDAC PCI0:
Giving out device to module MPC85xx_edac controller mpc85xx_pci_err: DEV
mpc85xx-pci-edac.0.auto (INTERRUPT)
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,717 console __init__ L0151 DEBUG|
MPC85xx_edac acquired irq 24 for PCI Err
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,717 console __init__ L0151 DEBUG|
MPC85xx_edac PCI err registered
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,718 console __init__ L0151 DEBUG| sdhci:
Secure Digital Host Controller Interface driver
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,718 console __init__ L0151 DEBUG| sdhci:
Copyright(c) Pierre Ossman
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,718 console __init__ L0151 DEBUG|
sdhci-pltfm: SDHCI platform and OF driver helper
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,721 console __init__ L0151 DEBUG| mmc0
bounce up to 128 segments into one, max segment size 65536 bytes
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,744 console __init__ L0151 DEBUG| mmc0:
Internal clock never stabilised.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,776 console __init__ L0151 DEBUG| mmc0:
Internal clock never stabilised.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,792 console __init__ L0151 DEBUG| mmc0:
SDHCI controller on fe002e000.sdhc [fe002e000.sdhc] using DMA
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,794 console __init__ L0151 DEBUG| usbcore:
registered new interface driver usbkbd
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,794 console __init__ L0151 DEBUG| usbcore:
registered new interface driver usbmouse
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,794 console __init__ L0151 DEBUG| Freescale
hypervisor management driver
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,794 console __init__ L0151 DEBUG| fsl-hv: no
hypervisor found
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,795 console __init__ L0151 DEBUG| ipip: IPv4
and MPLS over IPv4 tunneling driver
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,796 console __init__ L0151 DEBUG|
Initializing XFRM netlink socket
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,797 console __init__ L0151 DEBUG| NET:
Registered protocol family 10
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,806 console __init__ L0151 DEBUG| Segment
Routing with IPv6
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,807 console __init__ L0151 DEBUG| sit: IPv6,
IPv4 and MPLS over IPv4 tunneling driver
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,808 console __init__ L0151 DEBUG| NET:
Registered protocol family 17
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,808 console __init__ L0151 DEBUG| NET:
Registered protocol family 15
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,808 console __init__ L0151 DEBUG| Key type
dns_resolver registered
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,809 console __init__ L0151 DEBUG| drmem: No
dynamic reconfiguration memory found
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,810 console __init__ L0151 DEBUG| Loading
compiled-in X.509 certificates
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,831 console __init__ L0151 DEBUG| mmc0:
Internal clock never stabilised.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,856 console __init__ L0151 DEBUG| mmc0:
Internal clock never stabilised.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,857 console __init__ L0151 DEBUG| rtc-ds1307
0-0068: setting system clock to 2024-08-21 10:24:15 UTC (1724235855)
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,898 console __init__ L0151 DEBUG| Freeing
unused kernel memory: 4312K
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,899 console __init__ L0151 DEBUG| This
architecture does not have kernel memory protection.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,920 console __init__ L0151 DEBUG| mmc0:
Internal clock never stabilised.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:15,959 console __init__ L0151 DEBUG| mmc0:
Internal clock never stabilised.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:16,007 console __init__ L0151 DEBUG| mmc0:
Internal clock never stabilised.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:16,055 console __init__ L0151 DEBUG| mmc0:
Internal clock never stabilised.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:16,111 console __init__ L0151 DEBUG| mmc0:
Internal clock never stabilised.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:16,155 console __init__ L0151 DEBUG| mmc0:
Internal clock never stabilised.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:16,196 console __init__ L0151 DEBUG| mmc0:
Internal clock never stabilised.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:16,245 console __init__ L0151 DEBUG| mmc0:
Internal clock never stabilised.
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:16,568 console __init__ L0151 DEBUG| Starting
logging: OK
console:
1-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
2024-08-21 12:24:16,761 console __init__ L0151 DEBUG| QEMU
advent calendar 2018 - log in as 'root' and run 'asciijump' to play the game
(1/3)
tests/avocado/boot_linux_console.py:BootLinuxConsole.test_ppc64_e500:
PASS (1.81 s)
Failing downloads are over verbose, a pain when you get multiple of
them. A single failure produces:
Fetching asset from
tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd
Process Process-9:
Traceback (most recent call last):
File
"/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py",
line 1344, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File
"/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/http/client.py",
line 1336, in request
self._send_request(method, url, body, headers, encode_chunked)
File
"/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/http/client.py",
line 1382, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File
"/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/http/client.py",
line 1331, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File
"/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/http/client.py",
line 1091, in _send_output
self.send(msg)
File
"/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/http/client.py",
line 1035, in send
self.connect()
File
"/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/http/client.py",
line 1470, in connect
super().connect()
File
"/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/http/client.py",
line 1001, in connect
self.sock = self._create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/socket.py",
line 853, in create_connection
raise exceptions[0]
File
"/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/socket.py",
line 838, in create_connection
sock.connect(sa)
TimeoutError: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File
"/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/process.py",
line 314, in _bootstrap
self.run()
File
"/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/process.py",
line 108, in run
self._target(*self._args, **self._kwargs)
File
"/Users/philmd/source/qemu/build.mac/full_clang/pyvenv/lib/python3.12/site-packages/avocado/utils/download.py",
line 69, in _url_download
src_file = url_open(url, data=data)
^^^^^^^^^^^^^^^^^^^^^^^^
File
"/Users/philmd/source/qemu/build.mac/full_clang/pyvenv/lib/python3.12/site-packages/avocado/utils/download.py",
line 48, in url_open
result = urlopen(url, data=data, timeout=timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py",
line 215, in urlopen
return opener.open(url, data, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py",
line 515, in open
response = self._open(req, data)
^^^^^^^^^^^^^^^^^^^^^
File
"/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py",
line 532, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py",
line 492, in _call_chain
result = func(*args)
^^^^^^^^^^^
File
"/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py",
line 1392, in https_open
return self.do_open(http.client.HTTPSConnection, req,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py",
line 1347, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error timed out>
Fetching asset from
tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd
Anyway I don't mind much and don't ask/expect anyone to fix that.
On macOS:
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH v4 04/35] Bump avocado to 103.0
2024-08-21 10:34 ` Philippe Mathieu-Daudé
@ 2024-08-29 9:45 ` Daniel P. Berrangé
0 siblings, 0 replies; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-08-29 9:45 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: Thomas Huth, Alex Bennée, qemu-devel, Ani Sinha,
Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On Wed, Aug 21, 2024 at 12:34:45PM +0200, Philippe Mathieu-Daudé wrote:
> On 21/8/24 10:27, Thomas Huth wrote:
> > From: Cleber Rosa <crosa@redhat.com>
> >
> > This bumps Avocado to latest the LTS release.
> >
> > An LTS release is one that can receive bugfixes and guarantees
> > stability for a much longer period and has incremental minor releases
> > made.
> >
> > Even though the 103.0 LTS release is pretty a rewrite of Avocado when
> > compared to 88.1, the behavior of all existing tests under
> > tests/avocado has been extensively tested no regression in behavior
> > was found.
> >
> > To keep behavior of jobs as close as possible with previous version,
> > this version bump keeps the execution serial (maximum of one task at a
> > time being run).
> >
> > Reference: https://avocado-framework.readthedocs.io/en/103.0/releases/lts/103_0.html
> > Signed-off-by: Cleber Rosa <crosa@redhat.com>
> > Tested-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
> > Message-ID: <20240806173119.582857-2-crosa@redhat.com>
> > Signed-off-by: Thomas Huth <thuth@redhat.com>
> > ---
> > pythondeps.toml | 2 +-
> > tests/Makefile.include | 2 +-
> > 2 files changed, 2 insertions(+), 2 deletions(-)
>
> I suppose we now need an ultra wide monitor and set the terminal
> COLUMNS to a value >= 512 to not make sense of the spaghetti console
> now (my laptop terminal is 48x150 full screen, I might need new
> eyes). (Compare with commit 44055caaa5 description). See for example
> some lines are 267 columns wide:
>
> $ make check-avocado AVOCADO_TAGS='machine:ppce500 machine:mpc8544ds'
snip
> Anyway I don't mind much and don't ask/expect anyone to fix that.
Hopefully this will just be a temporary downside, if eventually get all
tests converted to the new functional test framework.
>
> On macOS:
> Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 69+ messages in thread
* [PATCH v4 05/35] tests/avocado/avocado_qemu: Fix the "from" statements in linuxtest.py
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (3 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 04/35] Bump avocado to 103.0 Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 9:31 ` Philippe Mathieu-Daudé
2024-08-21 8:27 ` [PATCH v4 06/35] tests/avocado/boot_linux_console: Remove the s390x subtest Thomas Huth
` (29 subsequent siblings)
34 siblings, 1 reply; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
Without this change, the new Avocado v103 fails to find the tests
that are based on the LinuxTest class.
Suggested-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/avocado/avocado_qemu/linuxtest.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tests/avocado/avocado_qemu/linuxtest.py b/tests/avocado/avocado_qemu/linuxtest.py
index e1dc838b1c..66fb9f1507 100644
--- a/tests/avocado/avocado_qemu/linuxtest.py
+++ b/tests/avocado/avocado_qemu/linuxtest.py
@@ -13,8 +13,8 @@
from avocado.utils import cloudinit, datadrainer, process, vmimage
-from . import LinuxSSHMixIn
-from . import QemuSystemTest
+from avocado_qemu import LinuxSSHMixIn
+from avocado_qemu import QemuSystemTest
if os.path.islink(os.path.dirname(os.path.dirname(__file__))):
# The link to the avocado tests dir in the source code directory
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* Re: [PATCH v4 05/35] tests/avocado/avocado_qemu: Fix the "from" statements in linuxtest.py
2024-08-21 8:27 ` [PATCH v4 05/35] tests/avocado/avocado_qemu: Fix the "from" statements in linuxtest.py Thomas Huth
@ 2024-08-21 9:31 ` Philippe Mathieu-Daudé
2024-08-21 10:07 ` Thomas Huth
0 siblings, 1 reply; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-21 9:31 UTC (permalink / raw)
To: Thomas Huth, Alex Bennée, qemu-devel, Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On 21/8/24 10:27, Thomas Huth wrote:
> Without this change, the new Avocado v103 fails to find the tests
> that are based on the LinuxTest class.
Reorder before previous patch?
> Suggested-by: Cleber Rosa <crosa@redhat.com>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/avocado/avocado_qemu/linuxtest.py | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/tests/avocado/avocado_qemu/linuxtest.py b/tests/avocado/avocado_qemu/linuxtest.py
> index e1dc838b1c..66fb9f1507 100644
> --- a/tests/avocado/avocado_qemu/linuxtest.py
> +++ b/tests/avocado/avocado_qemu/linuxtest.py
> @@ -13,8 +13,8 @@
>
> from avocado.utils import cloudinit, datadrainer, process, vmimage
>
> -from . import LinuxSSHMixIn
> -from . import QemuSystemTest
> +from avocado_qemu import LinuxSSHMixIn
> +from avocado_qemu import QemuSystemTest
>
> if os.path.islink(os.path.dirname(os.path.dirname(__file__))):
> # The link to the avocado tests dir in the source code directory
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v4 05/35] tests/avocado/avocado_qemu: Fix the "from" statements in linuxtest.py
2024-08-21 9:31 ` Philippe Mathieu-Daudé
@ 2024-08-21 10:07 ` Thomas Huth
2024-08-21 10:18 ` Philippe Mathieu-Daudé
0 siblings, 1 reply; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 10:07 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, Alex Bennée, qemu-devel,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On 21/08/2024 11.31, Philippe Mathieu-Daudé wrote:
> On 21/8/24 10:27, Thomas Huth wrote:
>> Without this change, the new Avocado v103 fails to find the tests
>> that are based on the LinuxTest class.
>
> Reorder before previous patch?
The funny thing is that Avocado v88 breaks the other way round if this patch
gets applied first. So if you are worried about bisecting, we'd need to
squash this into the patch that updates the version. OTOH, it's not a big
breakage, just some tests are not run by the test runner, so I think it's ok
to keep it separate.
Thomas
>> Suggested-by: Cleber Rosa <crosa@redhat.com>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>> tests/avocado/avocado_qemu/linuxtest.py | 4 ++--
>> 1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/tests/avocado/avocado_qemu/linuxtest.py
>> b/tests/avocado/avocado_qemu/linuxtest.py
>> index e1dc838b1c..66fb9f1507 100644
>> --- a/tests/avocado/avocado_qemu/linuxtest.py
>> +++ b/tests/avocado/avocado_qemu/linuxtest.py
>> @@ -13,8 +13,8 @@
>> from avocado.utils import cloudinit, datadrainer, process, vmimage
>> -from . import LinuxSSHMixIn
>> -from . import QemuSystemTest
>> +from avocado_qemu import LinuxSSHMixIn
>> +from avocado_qemu import QemuSystemTest
>> if os.path.islink(os.path.dirname(os.path.dirname(__file__))):
>> # The link to the avocado tests dir in the source code directory
>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v4 05/35] tests/avocado/avocado_qemu: Fix the "from" statements in linuxtest.py
2024-08-21 10:07 ` Thomas Huth
@ 2024-08-21 10:18 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-21 10:18 UTC (permalink / raw)
To: Thomas Huth, Alex Bennée, qemu-devel, Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On 21/8/24 12:07, Thomas Huth wrote:
> On 21/08/2024 11.31, Philippe Mathieu-Daudé wrote:
>> On 21/8/24 10:27, Thomas Huth wrote:
>>> Without this change, the new Avocado v103 fails to find the tests
>>> that are based on the LinuxTest class.
>>
>> Reorder before previous patch?
>
> The funny thing is that Avocado v88 breaks the other way round if this
> patch gets applied first. So if you are worried about bisecting, we'd
> need to squash this into the patch that updates the version. OTOH, it's
> not a big breakage, just some tests are not run by the test runner, so I
> think it's ok to keep it separate.
Argh OK, fine then!
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
I'm not adding my R-b because I don't understand what is
wrong in the current code and the commit description is
a justification but not an explanation; but I don't mind.
>
> Thomas
>
>
>>> Suggested-by: Cleber Rosa <crosa@redhat.com>
>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>> ---
>>> tests/avocado/avocado_qemu/linuxtest.py | 4 ++--
>>> 1 file changed, 2 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/tests/avocado/avocado_qemu/linuxtest.py
>>> b/tests/avocado/avocado_qemu/linuxtest.py
>>> index e1dc838b1c..66fb9f1507 100644
>>> --- a/tests/avocado/avocado_qemu/linuxtest.py
>>> +++ b/tests/avocado/avocado_qemu/linuxtest.py
>>> @@ -13,8 +13,8 @@
>>> from avocado.utils import cloudinit, datadrainer, process, vmimage
>>> -from . import LinuxSSHMixIn
>>> -from . import QemuSystemTest
>>> +from avocado_qemu import LinuxSSHMixIn
>>> +from avocado_qemu import QemuSystemTest
>>> if os.path.islink(os.path.dirname(os.path.dirname(__file__))):
>>> # The link to the avocado tests dir in the source code directory
>>
>
^ permalink raw reply [flat|nested] 69+ messages in thread
* [PATCH v4 06/35] tests/avocado/boot_linux_console: Remove the s390x subtest
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (4 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 05/35] tests/avocado/avocado_qemu: Fix the "from" statements in linuxtest.py Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-29 9:46 ` Daniel P. Berrangé
2024-08-21 8:27 ` [PATCH v4 07/35] python: Install pycotap in our venv if necessary Thomas Huth
` (28 subsequent siblings)
34 siblings, 1 reply; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
We've got a much more sophisticated, Fedora-based test for s390x
("test_s390x_fedora" in another file) already, so the test in
boot_linux_console.py seems to be rather a waste of precious test
cycles. Thus move the command line check and delete the s390x
test in boot_linux_console.py.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/avocado/boot_linux_console.py | 20 --------------------
tests/avocado/machine_s390_ccw_virtio.py | 2 ++
2 files changed, 2 insertions(+), 20 deletions(-)
diff --git a/tests/avocado/boot_linux_console.py b/tests/avocado/boot_linux_console.py
index cffdd6b5a2..18c69d6acc 100644
--- a/tests/avocado/boot_linux_console.py
+++ b/tests/avocado/boot_linux_console.py
@@ -1304,26 +1304,6 @@ def test_aarch64_raspi3_atf(self):
self.vm.launch()
self.wait_for_console_pattern('version UEFI Firmware v1.15')
- def test_s390x_s390_ccw_virtio(self):
- """
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
- """
- kernel_url = ('https://archives.fedoraproject.org/pub/archive'
- '/fedora-secondary/releases/29/Everything/s390x/os/images'
- '/kernel.img')
- kernel_hash = 'e8e8439103ef8053418ef062644ffd46a7919313'
- kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
-
- self.vm.set_console()
- kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=sclp0'
- self.vm.add_args('-nodefaults',
- '-kernel', kernel_path,
- '-append', kernel_command_line)
- self.vm.launch()
- console_pattern = 'Kernel command line: %s' % kernel_command_line
- self.wait_for_console_pattern(console_pattern)
-
def test_alpha_clipper(self):
"""
:avocado: tags=arch:alpha
diff --git a/tests/avocado/machine_s390_ccw_virtio.py b/tests/avocado/machine_s390_ccw_virtio.py
index 26e938c9e9..7a214110fc 100644
--- a/tests/avocado/machine_s390_ccw_virtio.py
+++ b/tests/avocado/machine_s390_ccw_virtio.py
@@ -200,6 +200,8 @@ def test_s390x_fedora(self):
'-device', 'virtio-rng-ccw,devno=fe.1.9876',
'-device', 'virtio-gpu-ccw,devno=fe.2.5432')
self.vm.launch()
+ self.wait_for_console_pattern('Kernel command line: %s'
+ % kernel_command_line)
self.wait_for_console_pattern('Entering emergency mode')
# Some tests to see whether the CLI options have been considered:
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* Re: [PATCH v4 06/35] tests/avocado/boot_linux_console: Remove the s390x subtest
2024-08-21 8:27 ` [PATCH v4 06/35] tests/avocado/boot_linux_console: Remove the s390x subtest Thomas Huth
@ 2024-08-29 9:46 ` Daniel P. Berrangé
0 siblings, 0 replies; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-08-29 9:46 UTC (permalink / raw)
To: Thomas Huth
Cc: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On Wed, Aug 21, 2024 at 10:27:07AM +0200, Thomas Huth wrote:
> We've got a much more sophisticated, Fedora-based test for s390x
> ("test_s390x_fedora" in another file) already, so the test in
> boot_linux_console.py seems to be rather a waste of precious test
> cycles. Thus move the command line check and delete the s390x
> test in boot_linux_console.py.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/avocado/boot_linux_console.py | 20 --------------------
> tests/avocado/machine_s390_ccw_virtio.py | 2 ++
> 2 files changed, 2 insertions(+), 20 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 69+ messages in thread
* [PATCH v4 07/35] python: Install pycotap in our venv if necessary
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (5 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 06/35] tests/avocado/boot_linux_console: Remove the s390x subtest Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-29 9:49 ` Daniel P. Berrangé
2024-08-21 8:27 ` [PATCH v4 08/35] tests/functional: Add base classes for the upcoming pytest-based tests Thomas Huth
` (27 subsequent siblings)
34 siblings, 1 reply; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
The upcoming functional tests will require pycotap for providing
TAP output from the python-based tests. Since we want to be able
to run some of the tests offline by default, too, let's install
it along with meson in our venv if necessary (it's size is only
5 kB, so adding the wheel here should not really be a problem).
The wheel file has been obtained with:
pip download --only-binary :all: --dest . --no-cache pycotap
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
python/wheels/pycotap-1.3.1-py3-none-any.whl | Bin 0 -> 5119 bytes
pythondeps.toml | 1 +
2 files changed, 1 insertion(+)
create mode 100644 python/wheels/pycotap-1.3.1-py3-none-any.whl
diff --git a/python/wheels/pycotap-1.3.1-py3-none-any.whl b/python/wheels/pycotap-1.3.1-py3-none-any.whl
new file mode 100644
index 0000000000000000000000000000000000000000..9c2c7d259365c8a0487228c6878e2c0b10ba6bdd
GIT binary patch
literal 5119
zcmeI0c{r49-^VX&3}X-3cO#7@+t|t4j6saOFm|%Vlq5p-b*v*b+!#WUElXPLTgIN!
zWLF9yOO}-P=zd?_J-zqc^Y8P=^F5C1I?g|?>vNoc{Jy{6`8`b$6qIZL0MG#GGI+~H
zES3$R0sv1|0AK|Gz(3g84}<oXc5v|Y^~5+hNcjgpv?lq^D>8-dSoQbO2%udjK9nR1
zX@Mw7zRZ{mFv|`)4NPytMe=eP5h(K}-@36c__dlrZxYgrbFtHmU$`UJ-juerPK0GR
zshPauoDlZTdy!Dv>PW-C`_4w>c^0|S#!PJQr&Rx&%f}eZTPHMc1X8cU%aYZ2Q)PK2
zm@9*23{tU5IZhDSn50gKRt1xkbO~0B`E<O9e<BUPhQp1f#(gmRd99?AMrO*KBFkV7
z(7ULx8)hjBwWl=YvLVrCX$xltk4av?rYp!=^KMZtM-4fJ+?f6VI127^u`p15wv3O0
zN^3C6WLa0>YJDWa1^0xIIV#bxv{|)IIpNc11RFU$I6~(T)|;Gf1>iao+q>y8Qu&AR
zX&ZGcxw#t-3T;Z$&qQX^FO{VmvdcLPNNXO?UxY^wdnFsSo`}Ojht)qBW)<Y;Ihc5E
zDnV*dO~ulY*Q8!@&Xc><Mp?ScSvfsgk8t<OD7%FsZ-2<RM_oS5RZNeAA@)X&Az_jo
zo<ic3*a_<Bi)B07<{_<8$UL&qGmga3sr#3p<8?`d+;(aXS5ahFhxTc!JLsTfEdHTW
z5q_^LuK`^r?NK4S6wTpOaSLis^E6(!U>R#y&_6pL<Zx@$pJe1DA$0*x$vO!xF6|41
zzw?zYA~5g0Hj0`)zX+pAahkhX^x8Byqgw1)i#<4dsvbl-d;}so+OdIXxfWO+1*lf4
zOI{Q&rRyqPp?r`5X|lE%QxC5pTo@dgjwji3ujncdF>sGnx#HyZ>EF1+wb6y9RGk}|
zm|OSaNVkTPnmS*=!%K8^w1vs;6Ns#%7*MoHUtf9N>mJwo_){ZBm9}~XK?Na2=L8ds
ziyM~BIXK2LGrDWYR?Js<&cYIA4VTCR^oo`(vRq1Uid<2D0I!6n9<Q<oW4d7L_4?kE
z7X|&4V)3gpj9K|RK2sZa+WGG=lsdWI*(m9KjE~hVnCqjBQI19xnC|m<*3t*@%PUk0
z$z!gMcGt(=15T7%u?vO50~YZl>V5Vs#*|i-snEN6nLM3LEgwhU2dxN<boG2XU~iSW
znD9U-3rMoq=hGjN3(fx|*!@9}?j;TFb1*bn%pog%-OcJHWvMM2S)^!rw+wM;yZkaT
zx?jP`KDjol&aU>Utb%K^JAaFZe0iU1Gur}{wa*q5c6a@yHteQ+%(2J>nTbYw0soAc
zGzp7<Z7a*TYq|TtlmLybS<dQ)ZYa9$EAyq2e(1=X;+uP4SY6JADJzI^2#cF14x6Mb
zQpwt~9^yh6F{|Aoox{f?j8)kpu9-=rZ)y0&>E(+$gN)nkBfmNxcqBpa`}?Nj(bK71
zeA%e>r%@xv#hCfu#H-)c?JC5kq@H~t-wu09ER5T4Nqp@_VX0&s$fo}4!Yz?aeMh`D
z<^;?zR(m{Wz~?cHVVN%HF*on@H1D*XMlUXr;-1eNgKNHS<PNU(@#;e@(ablh;~N;+
zzLbPw`Va#QiRoRorN*U@RNouV8V5EjGCe8RJaxo1!*Kbpc|yM}P3-W~)6Gbl*CB@B
zma2xvP?HYjmI)5Rc!-*F(u$|reuel|J*NSWFM1Qo<4LAFNKabDL!F+tq`~F=0pD@E
zHLqp8`fa`g{NUE%!iV#)3|ng@a6d9b>+_+Io!?~G%d5|vy>`kiZmn1pNZ3=@w<rCv
z2a)CrBDWB;7L?iLF#XXmXQl9#pQc5+9RUCc0e)?mK4?$hKO3gGC2B|-Dm7x#*FS1P
zlr)Ewp6cxxlzTVV*AMNHK6ws8e11yeEL19jjtwQYh!bJ=ztrlo6~HRDDIc(D4i7Xt
z?tT1VW7F(1n-+_9ad2>%o|a%K`xB<_aHhsaF%ICb4Jqcy>+>z7lj6t$07eM_{C{<D
zsH~Knl&q8s8iSVp^QB{AYID}uz~UUymn1{~^ovAXo^1=FfvkC}<P}`FKMic-nO)>8
zS*_R&5^>ee_h78y;Qml@k&aVFH|!v+ryrm9M170dm!!J8zxQ0p^xf|Bh|j(UU%He1
zN!?dLj5|ASyKJF>m|A=jBE4_72;3^vM*Tinso5R2>y}i4K(8f}HN@8Y9kdddc;)Bi
z?j$k7X0B9M7x6L<X(duK-Rj`;j3jj<=E0Gw9%uY3e6WlvFY{8}l8_=#p70ullU{K*
zmOWrq3d%5a&m~*QF<pB|#lamClrn>io2u_f6Ds8$;0?a&!}!_b0Z3lMM^wNwfFURR
zdWk;<KRG&WRBgOC>8O)`#y$R8P>MA@%A06to-;ocOykBG*IKwHv3$FkR%hI#Zs_U-
z<_~SfK?b{6N~=1}FU+Du`BCNbHo1SynT+a}GhJVcX}tikUG5q%D_qBlaM=+DzABn$
z<BC$H3SV&0VZujA77r?hl-`HBSJLTRP$kb!e+kPYkBy6MX#A3~nZd_<_+<OUlC<*J
z+cSPSa{^was6CN@yms)tPoWuY`tMzv#=~hk2~%g90I`Pk3q5L7--^s0R~De+tu3Nw
ztXWS>jyGXbWxADZydWKL50Mw5*qJQ&dTx*9c~`{snkGopR7n++t&n>S<hV>}&WMm5
zhmrO}OADL8%)vBEP?KqCo7MTH6Xj0eT~>`h>}v{h2(P#%@AW!ZTH1;7;kdG0(*jQo
zD62ScMz)JY7t}<?TY!}9stp3}{-MjR&|0~wh%hdu+~y3s#a>L{w2^q&=AIyt^-c2;
zvw}&4#JSNk?B%-GY{iWQbPel7MQ@-BEZSoe1)`>wbDstGDobMTm2|AN)-Le>Rm3bN
zc*}oN#GZi|sHd-+-+znvzeW6?Tg1coH;<zz0N^7v00{j5PW+>Yk$Na?U2T+hv;`@J
zTZuk2|AD{|4(XVBo1H2_+jMO5kB{{eF6Wx5HpbFKUl*+;LEhJ_ZpH2k?p0%RtI-tP
zTb=c~X=-X}gf&@-qIBDD(So)C;Opkk%BRZ6xYvtH2-hzwJ}H0ZIxog}6XZ;B(QrH&
zD<eiTp-hie$qEnHP}X}@$uHp8uYPDuDJ$WQlBs=C8CFb&v*(3b;L=lxUYD!No;TZ$
zR)Jt@BTG?eytzR^gysT!eZ-r)hdsoX2%gN3+`IcB#Nlk)?e<j5oy=t2@+sS7setS+
z%|6?zDFz@h#TK-n>TOWiUe&7U<mZ$6%?&+GQ!}ie7mi_Q70(v)6nM_4TDR#4WL$PK
zv-Ezoef4e7B8$pR>hTF{3|FTQLl?^a$z*zi?aB=?63OM$Yc(C*u{EWjs?QojtOmYt
zS@F$HW~Dk4gX8PvDf&*Uk)S(g85o6G-_PVNfA~Z<^pwp*HBhn0`kQx%*xu)_?Xk^4
z&rlXG28b71^-Y;R&V;DBK1LDq$m>3qarlHli^{WftFEl{e26;G&AoONT8<u$%gC)z
zh;C>6IPE@pg4^_N*mcYB7tp4T^K6pEDFtjqc2sps?vNDcR9O!DQtk!Z29<`jlH}Kz
zo9&2zue)Xg(s{i!<DF3nsUg*;Mc-SP(xL~)peKPBJD5x%gT9RpyHv+fixcVn$BaDh
zjL@Nai6cXt!k^NFCp+$|nyLl!k}I!X@l#e_yyETiHv677cJFph=AJ5F>ch?UZfpG^
z$*(={veXW|;7#=JB@B(-_VmD-$vnyDKyF>Z@6Kf;2G6)Bnk85=jIr97&%j)G&V<X7
zD_k49uN+zyUnBcU&Ip%FKVDUR+K6S==^@R<`^0#VoRwNlcT4}J#f7~SNU{L2(O2(7
z6!JO)RvN*PrB&_0&a}k10Wj_xyphm|d(%qWquWa^6fs{?#)No#5W;^oYojgrP73vt
z`_y-X57=>M=4fY2LqdAAKc5YBLk8(S22@nC?-k8WsJ5#1!XIf~vI5PLCs`abP|JJ7
z)T=BQQsP{Q(i*H(@-kbCZYS5czc$@gDGn_Sriq%%%v^QZ5Dp4`4$!Y!ppg)v{1wMT
z*Ke*KY<15HLfB%A<@PVBNIz6ZzBksB=LsH6aOaxB4zSqwT0)P{YQovO#%g&7X<WrO
z@ExkD^MvW8?ZdAT{#}pu>Hl>M6%@;^o(BNn`@{Pi2U{8H=^4=t8X}3qU<CB!K>uK^
znar5nOn?ay0fnFHm98-}g!i73ikCiCc5X;!NUnZJZg>IIHz+;4eQF#Gfx?NsMm1&-
ziId2;ptD8|W{^_(apLnq#0zBqy@fgQpN@Fnr{e`JAp_8#ksSa;f3q!&pTC2*>lIgT
zDa=)jWU0}B^x0E0zrr-`+Nrpc0f0X_0Py_=W~QfOVy63WsL8iT3kaXp7>uNq<B1XK
z%n?>(cqhYPaMMk;8^*&b*M+^C$`)@pVbSOFZ9_6lWl_C-OZK^;fpzvL%)1ya9h=~5
z_mCtLnrQEvj%hK;#R_gS4g$VjVv4tW^Ok}W4ia4g(n~xTw@6;HAHu9vo03mezljm*
zGf;hI_f>z#@&)=v9zsIHKGE#3g<yy$;VDedmmZ(d<6C#{U!S+ayhL1yU5uvkMvd(Z
z6zSxaRp5QL7`h+eC@avR%y@W=l6+q3lg=wVOsH8u9KE;Rqnbm%O6Qm!EkWCv#%9mn
zWqay0PMFK%NA;6>bBb!6>@pMwh09$sLhZa%5fsbYn!-<KJuc=-CxX_fE^F<GoiHe?
z@(4)Q;(Qtse0UwwaQUN~u5a-h(U19gqcac0mOaPWoQK!Fof3V(ZA%*bqf3~{qO{DX
zujfBd?26||uoO1zqM`$BCG0c(mmXjomwX-l-?WRHBFM<uDE|3c=17Ey@2kJLr1=@}
zcQNHiga}!xp8-EY%g=znixo#AM6~?{_@|KZGwkn<cSj;b#M1oO%@OR!oczDU@+<7u
zF?%FJgc038!;XgTue4t+e<VT#1LHr_{%iohqJFjMkq8mRzd`+rX@5ojYStswk!}12
a`M=gRMNm<H|DFb*0Z`-skj4D{@BaZhvADJX
literal 0
HcmV?d00001
diff --git a/pythondeps.toml b/pythondeps.toml
index 175cf99241..0a7f69b196 100644
--- a/pythondeps.toml
+++ b/pythondeps.toml
@@ -20,6 +20,7 @@
[meson]
# The install key should match the version in python/wheels/
meson = { accepted = ">=1.1.0", installed = "1.2.3", canary = "meson" }
+pycotap = { accepted = ">=1.1.0", installed = "1.3.1" }
[docs]
# Please keep the installed versions in sync with docs/requirements.txt
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* Re: [PATCH v4 07/35] python: Install pycotap in our venv if necessary
2024-08-21 8:27 ` [PATCH v4 07/35] python: Install pycotap in our venv if necessary Thomas Huth
@ 2024-08-29 9:49 ` Daniel P. Berrangé
0 siblings, 0 replies; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-08-29 9:49 UTC (permalink / raw)
To: Thomas Huth
Cc: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On Wed, Aug 21, 2024 at 10:27:08AM +0200, Thomas Huth wrote:
> The upcoming functional tests will require pycotap for providing
> TAP output from the python-based tests. Since we want to be able
> to run some of the tests offline by default, too, let's install
> it along with meson in our venv if necessary (it's size is only
> 5 kB, so adding the wheel here should not really be a problem).
>
> The wheel file has been obtained with:
>
> pip download --only-binary :all: --dest . --no-cache pycotap
>
> Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> python/wheels/pycotap-1.3.1-py3-none-any.whl | Bin 0 -> 5119 bytes
> pythondeps.toml | 1 +
> 2 files changed, 1 insertion(+)
> create mode 100644 python/wheels/pycotap-1.3.1-py3-none-any.whl
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 69+ messages in thread
* [PATCH v4 08/35] tests/functional: Add base classes for the upcoming pytest-based tests
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (6 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 07/35] python: Install pycotap in our venv if necessary Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 8:27 ` [PATCH v4 09/35] tests/functional: Set up logging Thomas Huth
` (26 subsequent siblings)
34 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
The files are mostly a copy of the tests/avocado/avocado_qemu/__init__.py
file with some adjustments to get rid of the Avocado dependencies (i.e.
we also have to drop the LinuxSSHMixIn and LinuxTest for now).
The emulator binary and build directory are now passed via
environment variables that will be set via meson.build later.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 5 +
tests/functional/qemu_test/__init__.py | 13 ++
tests/functional/qemu_test/cmd.py | 178 +++++++++++++++++++++++++
tests/functional/qemu_test/config.py | 36 +++++
tests/functional/qemu_test/testcase.py | 153 +++++++++++++++++++++
5 files changed, 385 insertions(+)
create mode 100644 tests/functional/qemu_test/__init__.py
create mode 100644 tests/functional/qemu_test/cmd.py
create mode 100644 tests/functional/qemu_test/config.py
create mode 100644 tests/functional/qemu_test/testcase.py
diff --git a/MAINTAINERS b/MAINTAINERS
index 3584d6a6c6..80cc84dd99 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -4146,6 +4146,11 @@ F: .gitlab-ci.d/cirrus/freebsd*
F: tests/vm/freebsd
W: https://cirrus-ci.com/github/qemu/qemu
+Functional testing framework
+M: Thomas Huth <thuth@redhat.com>
+R: Philippe Mathieu-Daudé <philmd@linaro.org>
+F: tests/functional/qemu_test/
+
Windows Hosted Continuous Integration
M: Yonggang Luo <luoyonggang@gmail.com>
S: Maintained
diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu_test/__init__.py
new file mode 100644
index 0000000000..2f1e0bc70d
--- /dev/null
+++ b/tests/functional/qemu_test/__init__.py
@@ -0,0 +1,13 @@
+# Test class and utilities for functional tests
+#
+# Copyright 2024 Red Hat, Inc.
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+
+from .config import BUILD_DIR
+from .cmd import has_cmd, has_cmds, run_cmd, is_readable_executable_file, \
+ interrupt_interactive_console_until_pattern, wait_for_console_pattern, \
+ exec_command, exec_command_and_wait_for_pattern
+from .testcase import QemuSystemTest, QemuBaseTest
diff --git a/tests/functional/qemu_test/cmd.py b/tests/functional/qemu_test/cmd.py
new file mode 100644
index 0000000000..8580ee8faf
--- /dev/null
+++ b/tests/functional/qemu_test/cmd.py
@@ -0,0 +1,178 @@
+# Test class and utilities for functional tests
+#
+# Copyright 2018, 2024 Red Hat, Inc.
+#
+# Original Author (Avocado-based tests):
+# Cleber Rosa <crosa@redhat.com>
+#
+# Adaption for standalone version:
+# Thomas Huth <thuth@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import logging
+import os
+import os.path
+import subprocess
+
+
+def has_cmd(name, args=None):
+ """
+ This function is for use in a @skipUnless decorator, e.g.:
+
+ @skipUnless(*has_cmd('sudo -n', ('sudo', '-n', 'true')))
+ def test_something_that_needs_sudo(self):
+ ...
+ """
+
+ if args is None:
+ args = ('which', name)
+
+ try:
+ _, stderr, exitcode = run_cmd(args)
+ except Exception as e:
+ exitcode = -1
+ stderr = str(e)
+
+ if exitcode != 0:
+ cmd_line = ' '.join(args)
+ err = f'{name} required, but "{cmd_line}" failed: {stderr.strip()}'
+ return (False, err)
+ else:
+ return (True, '')
+
+def has_cmds(*cmds):
+ """
+ This function is for use in a @skipUnless decorator and
+ allows checking for the availability of multiple commands, e.g.:
+
+ @skipUnless(*has_cmds(('cmd1', ('cmd1', '--some-parameter')),
+ 'cmd2', 'cmd3'))
+ def test_something_that_needs_cmd1_and_cmd2(self):
+ ...
+ """
+
+ for cmd in cmds:
+ if isinstance(cmd, str):
+ cmd = (cmd,)
+
+ ok, errstr = has_cmd(*cmd)
+ if not ok:
+ return (False, errstr)
+
+ return (True, '')
+
+def run_cmd(args):
+ subp = subprocess.Popen(args,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE,
+ universal_newlines=True)
+ stdout, stderr = subp.communicate()
+ ret = subp.returncode
+
+ return (stdout, stderr, ret)
+
+def is_readable_executable_file(path):
+ return os.path.isfile(path) and os.access(path, os.R_OK | os.X_OK)
+
+def _console_interaction(test, success_message, failure_message,
+ send_string, keep_sending=False, vm=None):
+ assert not keep_sending or send_string
+ if vm is None:
+ vm = test.vm
+ console = vm.console_file
+ console_logger = logging.getLogger('console')
+ while True:
+ if send_string:
+ vm.console_socket.sendall(send_string.encode())
+ if not keep_sending:
+ send_string = None # send only once
+
+ # Only consume console output if waiting for something
+ if success_message is None and failure_message is None:
+ if send_string is None:
+ break
+ continue
+
+ try:
+ msg = console.readline().decode().strip()
+ except UnicodeDecodeError:
+ msg = None
+ if not msg:
+ continue
+ console_logger.debug(msg)
+ if success_message is None or success_message in msg:
+ break
+ if failure_message and failure_message in msg:
+ console.close()
+ fail = 'Failure message found in console: "%s". Expected: "%s"' % \
+ (failure_message, success_message)
+ test.fail(fail)
+
+def interrupt_interactive_console_until_pattern(test, success_message,
+ failure_message=None,
+ interrupt_string='\r'):
+ """
+ Keep sending a string to interrupt a console prompt, while logging the
+ console output. Typical use case is to break a boot loader prompt, such:
+
+ Press a key within 5 seconds to interrupt boot process.
+ 5
+ 4
+ 3
+ 2
+ 1
+ Booting default image...
+
+ :param test: a test containing a VM that will have its console
+ read and probed for a success or failure message
+ :type test: :class:`qemu_test.QemuSystemTest`
+ :param success_message: if this message appears, test succeeds
+ :param failure_message: if this message appears, test fails
+ :param interrupt_string: a string to send to the console before trying
+ to read a new line
+ """
+ _console_interaction(test, success_message, failure_message,
+ interrupt_string, True)
+
+def wait_for_console_pattern(test, success_message, failure_message=None,
+ vm=None):
+ """
+ Waits for messages to appear on the console, while logging the content
+
+ :param test: a test containing a VM that will have its console
+ read and probed for a success or failure message
+ :type test: :class:`qemu_test.QemuSystemTest`
+ :param success_message: if this message appears, test succeeds
+ :param failure_message: if this message appears, test fails
+ """
+ _console_interaction(test, success_message, failure_message, None, vm=vm)
+
+def exec_command(test, command):
+ """
+ Send a command to a console (appending CRLF characters), while logging
+ the content.
+
+ :param test: a test containing a VM.
+ :type test: :class:`qemu_test.QemuSystemTest`
+ :param command: the command to send
+ :type command: str
+ """
+ _console_interaction(test, None, None, command + '\r')
+
+def exec_command_and_wait_for_pattern(test, command,
+ success_message, failure_message=None):
+ """
+ Send a command to a console (appending CRLF characters), then wait
+ for success_message to appear on the console, while logging the.
+ content. Mark the test as failed if failure_message is found instead.
+
+ :param test: a test containing a VM that will have its console
+ read and probed for a success or failure message
+ :type test: :class:`qemu_test.QemuSystemTest`
+ :param command: the command to send
+ :param success_message: if this message appears, test succeeds
+ :param failure_message: if this message appears, test fails
+ """
+ _console_interaction(test, success_message, failure_message, command + '\r')
diff --git a/tests/functional/qemu_test/config.py b/tests/functional/qemu_test/config.py
new file mode 100644
index 0000000000..edd75b7fd0
--- /dev/null
+++ b/tests/functional/qemu_test/config.py
@@ -0,0 +1,36 @@
+# Test class and utilities for functional tests
+#
+# Copyright 2018, 2024 Red Hat, Inc.
+#
+# Original Author (Avocado-based tests):
+# Cleber Rosa <crosa@redhat.com>
+#
+# Adaption for standalone version:
+# Thomas Huth <thuth@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import os
+from pathlib import Path
+
+
+def _source_dir():
+ # Determine top-level directory of the QEMU sources
+ return Path(__file__).parent.parent.parent.parent
+
+def _build_dir():
+ root = os.getenv('QEMU_BUILD_ROOT')
+ if root is not None:
+ return Path(root)
+ # Makefile.mtest only exists in build dir, so if it is available, use CWD
+ if os.path.exists('Makefile.mtest'):
+ return Path(os.getcwd())
+
+ root = os.path.join(_source_dir(), 'build')
+ if os.path.exists(root):
+ return Path(root)
+
+ raise Exception("Cannot identify build dir, set QEMU_BUILD_ROOT")
+
+BUILD_DIR = _build_dir()
diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu_test/testcase.py
new file mode 100644
index 0000000000..6905675778
--- /dev/null
+++ b/tests/functional/qemu_test/testcase.py
@@ -0,0 +1,153 @@
+# Test class and utilities for functional tests
+#
+# Copyright 2018, 2024 Red Hat, Inc.
+#
+# Original Author (Avocado-based tests):
+# Cleber Rosa <crosa@redhat.com>
+#
+# Adaption for standalone version:
+# Thomas Huth <thuth@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import logging
+import os
+import pycotap
+import sys
+import unittest
+import uuid
+
+from qemu.machine import QEMUMachine
+from qemu.utils import kvm_available, tcg_available
+
+from .cmd import run_cmd
+from .config import BUILD_DIR
+
+
+class QemuBaseTest(unittest.TestCase):
+
+ qemu_bin = os.getenv('QEMU_TEST_QEMU_BINARY')
+ arch = None
+
+ workdir = None
+ log = logging.getLogger('qemu-test')
+
+ def setUp(self, bin_prefix):
+ self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be set')
+ self.arch = self.qemu_bin.split('-')[-1]
+
+ self.workdir = os.path.join(BUILD_DIR, 'tests/functional', self.arch,
+ self.id())
+ os.makedirs(self.workdir, exist_ok=True)
+
+ def main():
+ path = os.path.basename(sys.argv[0])[:-3]
+ tr = pycotap.TAPTestRunner(message_log = pycotap.LogMode.LogToError,
+ test_output_log = pycotap.LogMode.LogToError)
+ unittest.main(module = None, testRunner = tr, argv=["__dummy__", path])
+
+
+class QemuSystemTest(QemuBaseTest):
+ """Facilitates system emulation tests."""
+
+ cpu = None
+ machine = None
+ _machinehelp = None
+
+ def setUp(self):
+ self._vms = {}
+
+ super().setUp('qemu-system-')
+
+ def set_machine(self, machinename):
+ # TODO: We should use QMP to get the list of available machines
+ if not self._machinehelp:
+ self._machinehelp = run_cmd([self.qemu_bin, '-M', 'help'])[0];
+ if self._machinehelp.find(machinename) < 0:
+ self.skipTest('no support for machine ' + machinename)
+ self.machine = machinename
+
+ def require_accelerator(self, accelerator):
+ """
+ Requires an accelerator to be available for the test to continue
+
+ It takes into account the currently set qemu binary.
+
+ If the check fails, the test is canceled. If the check itself
+ for the given accelerator is not available, the test is also
+ canceled.
+
+ :param accelerator: name of the accelerator, such as "kvm" or "tcg"
+ :type accelerator: str
+ """
+ checker = {'tcg': tcg_available,
+ 'kvm': kvm_available}.get(accelerator)
+ if checker is None:
+ self.skipTest("Don't know how to check for the presence "
+ "of accelerator %s" % accelerator)
+ if not checker(qemu_bin=self.qemu_bin):
+ self.skipTest("%s accelerator does not seem to be "
+ "available" % accelerator)
+
+ def require_netdev(self, netdevname):
+ netdevhelp = run_cmd([self.qemu_bin,
+ '-M', 'none', '-netdev', 'help'])[0];
+ if netdevhelp.find('\n' + netdevname + '\n') < 0:
+ self.skipTest('no support for " + netdevname + " networking')
+
+ def require_device(self, devicename):
+ devhelp = run_cmd([self.qemu_bin,
+ '-M', 'none', '-device', 'help'])[0];
+ if devhelp.find(devicename) < 0:
+ self.skipTest('no support for device ' + devicename)
+
+ def _new_vm(self, name, *args):
+ vm = QEMUMachine(self.qemu_bin, base_temp_dir=self.workdir)
+ self.log.debug('QEMUMachine "%s" created', name)
+ self.log.debug('QEMUMachine "%s" temp_dir: %s', name, vm.temp_dir)
+ self.log.debug('QEMUMachine "%s" log_dir: %s', name, vm.log_dir)
+ if args:
+ vm.add_args(*args)
+ return vm
+
+ @property
+ def vm(self):
+ return self.get_vm(name='default')
+
+ def get_vm(self, *args, name=None):
+ if not name:
+ name = str(uuid.uuid4())
+ if self._vms.get(name) is None:
+ self._vms[name] = self._new_vm(name, *args)
+ if self.cpu is not None:
+ self._vms[name].add_args('-cpu', self.cpu)
+ if self.machine is not None:
+ self._vms[name].set_machine(self.machine)
+ return self._vms[name]
+
+ def set_vm_arg(self, arg, value):
+ """
+ Set an argument to list of extra arguments to be given to the QEMU
+ binary. If the argument already exists then its value is replaced.
+
+ :param arg: the QEMU argument, such as "-cpu" in "-cpu host"
+ :type arg: str
+ :param value: the argument value, such as "host" in "-cpu host"
+ :type value: str
+ """
+ if not arg or not value:
+ return
+ if arg not in self.vm.args:
+ self.vm.args.extend([arg, value])
+ else:
+ idx = self.vm.args.index(arg) + 1
+ if idx < len(self.vm.args):
+ self.vm.args[idx] = value
+ else:
+ self.vm.args.append(value)
+
+ def tearDown(self):
+ for vm in self._vms.values():
+ vm.shutdown()
+ super().tearDown()
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH v4 09/35] tests/functional: Set up logging
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (7 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 08/35] tests/functional: Add base classes for the upcoming pytest-based tests Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 8:27 ` [PATCH v4 10/35] tests/Makefile.include: Increase the level of indentation in the help text Thomas Huth
` (25 subsequent siblings)
34 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
Create log files for each test separately, one file that contains
the basic logging and one that contains the console output.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/functional/qemu_test/testcase.py | 27 +++++++++++++++++++++++++-
1 file changed, 26 insertions(+), 1 deletion(-)
diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu_test/testcase.py
index 6905675778..b2dd863c6e 100644
--- a/tests/functional/qemu_test/testcase.py
+++ b/tests/functional/qemu_test/testcase.py
@@ -31,7 +31,8 @@ class QemuBaseTest(unittest.TestCase):
arch = None
workdir = None
- log = logging.getLogger('qemu-test')
+ log = None
+ logdir = None
def setUp(self, bin_prefix):
self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be set')
@@ -41,6 +42,20 @@ def setUp(self, bin_prefix):
self.id())
os.makedirs(self.workdir, exist_ok=True)
+ self.logdir = self.workdir
+ self.log = logging.getLogger('qemu-test')
+ self.log.setLevel(logging.DEBUG)
+ self._log_fh = logging.FileHandler(os.path.join(self.logdir,
+ 'base.log'), mode='w')
+ self._log_fh.setLevel(logging.DEBUG)
+ fileFormatter = logging.Formatter(
+ '%(asctime)s - %(levelname)s: %(message)s')
+ self._log_fh.setFormatter(fileFormatter)
+ self.log.addHandler(self._log_fh)
+
+ def tearDown(self):
+ self.log.removeHandler(self._log_fh)
+
def main():
path = os.path.basename(sys.argv[0])[:-3]
tr = pycotap.TAPTestRunner(message_log = pycotap.LogMode.LogToError,
@@ -60,6 +75,15 @@ def setUp(self):
super().setUp('qemu-system-')
+ console_log = logging.getLogger('console')
+ console_log.setLevel(logging.DEBUG)
+ self._console_log_fh = logging.FileHandler(os.path.join(self.workdir,
+ 'console.log'), mode='w')
+ self._console_log_fh.setLevel(logging.DEBUG)
+ fileFormatter = logging.Formatter('%(asctime)s: %(message)s')
+ self._console_log_fh.setFormatter(fileFormatter)
+ console_log.addHandler(self._console_log_fh)
+
def set_machine(self, machinename):
# TODO: We should use QMP to get the list of available machines
if not self._machinehelp:
@@ -150,4 +174,5 @@ def set_vm_arg(self, arg, value):
def tearDown(self):
for vm in self._vms.values():
vm.shutdown()
+ logging.getLogger('console').removeHandler(self._console_log_fh)
super().tearDown()
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH v4 10/35] tests/Makefile.include: Increase the level of indentation in the help text
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (8 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 09/35] tests/functional: Set up logging Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 8:27 ` [PATCH v4 11/35] tests/functional: Prepare the meson build system for the functional tests Thomas Huth
` (24 subsequent siblings)
34 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
The next patch is going to add some entries that need more space between
the command and the help text, so let's increase the indentation here
first.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/Makefile.include | 30 +++++++++++++++---------------
1 file changed, 15 insertions(+), 15 deletions(-)
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 537804d101..fbb1a4b211 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -3,28 +3,28 @@
.PHONY: check-help
check-help:
@echo "Regression testing targets:"
- @echo " $(MAKE) check Run block, qapi-schema, unit, softfloat, qtest and decodetree tests"
- @echo " $(MAKE) bench Run speed tests"
+ @echo " $(MAKE) check Run block, qapi-schema, unit, softfloat, qtest and decodetree tests"
+ @echo " $(MAKE) bench Run speed tests"
@echo
@echo "Individual test suites:"
- @echo " $(MAKE) check-qtest-TARGET Run qtest tests for given target"
- @echo " $(MAKE) check-qtest Run qtest tests"
- @echo " $(MAKE) check-unit Run qobject tests"
- @echo " $(MAKE) check-qapi-schema Run QAPI schema tests"
- @echo " $(MAKE) check-block Run block tests"
+ @echo " $(MAKE) check-qtest-TARGET Run qtest tests for given target"
+ @echo " $(MAKE) check-qtest Run qtest tests"
+ @echo " $(MAKE) check-unit Run qobject tests"
+ @echo " $(MAKE) check-qapi-schema Run QAPI schema tests"
+ @echo " $(MAKE) check-block Run block tests"
ifneq ($(filter $(all-check-targets), check-softfloat),)
- @echo " $(MAKE) check-tcg Run TCG tests"
- @echo " $(MAKE) check-softfloat Run FPU emulation tests"
+ @echo " $(MAKE) check-tcg Run TCG tests"
+ @echo " $(MAKE) check-softfloat Run FPU emulation tests"
endif
- @echo " $(MAKE) check-avocado Run avocado (integration) tests for currently configured targets"
+ @echo " $(MAKE) check-avocado Run avocado (integration) tests for currently configured targets"
@echo
- @echo " $(MAKE) check-report.junit.xml Generates an aggregated XML test report"
- @echo " $(MAKE) check-venv Creates a Python venv for tests"
- @echo " $(MAKE) check-clean Clean the tests and related data"
+ @echo " $(MAKE) check-report.junit.xml Generates an aggregated XML test report"
+ @echo " $(MAKE) check-venv Creates a Python venv for tests"
+ @echo " $(MAKE) check-clean Clean the tests and related data"
@echo
@echo "The following are useful for CI builds"
- @echo " $(MAKE) check-build Build most test binaries"
- @echo " $(MAKE) get-vm-images Downloads all images used by avocado tests, according to configured targets (~350 MB each, 1.5 GB max)"
+ @echo " $(MAKE) check-build Build most test binaries"
+ @echo " $(MAKE) get-vm-images Downloads all images used by avocado tests, according to configured targets (~350 MB each, 1.5 GB max)"
@echo
@echo
@echo "The variable SPEED can be set to control the gtester speed setting."
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH v4 11/35] tests/functional: Prepare the meson build system for the functional tests
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (9 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 10/35] tests/Makefile.include: Increase the level of indentation in the help text Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 14:30 ` Philippe Mathieu-Daudé
2024-08-23 12:54 ` Philippe Mathieu-Daudé
2024-08-21 8:27 ` [PATCH v4 12/35] tests/functional: Convert simple avocado tests into standalone python tests Thomas Huth
` (23 subsequent siblings)
34 siblings, 2 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
Provide a meson.build file for the upcoming python-based functional
tests, and add some wrapper glue targets to the tests/Makefile.include
file. We are going to use two "speed" modes for the functional tests:
The "quick" tests can be run at any time (i.e. also during "make check"),
while the "thorough" tests should only be run when running a
"make check-functional" test run (since these tests might download
additional assets from the internet).
The changes to the meson.build files are partly based on an earlier
patch by Ani Sinha.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/Makefile.include | 11 ++++++
tests/functional/meson.build | 66 ++++++++++++++++++++++++++++++++++++
tests/meson.build | 1 +
3 files changed, 78 insertions(+)
create mode 100644 tests/functional/meson.build
diff --git a/tests/Makefile.include b/tests/Makefile.include
index fbb1a4b211..66c8cc3123 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -9,6 +9,8 @@ check-help:
@echo "Individual test suites:"
@echo " $(MAKE) check-qtest-TARGET Run qtest tests for given target"
@echo " $(MAKE) check-qtest Run qtest tests"
+ @echo " $(MAKE) check-functional Run python-based functional tests"
+ @echo " $(MAKE) check-functional-TARGET Run functional tests for a given target"
@echo " $(MAKE) check-unit Run qobject tests"
@echo " $(MAKE) check-qapi-schema Run QAPI schema tests"
@echo " $(MAKE) check-block Run block tests"
@@ -152,6 +154,15 @@ check-acceptance-deprecated-warning:
check-acceptance: check-acceptance-deprecated-warning | check-avocado
+FUNCTIONAL_TARGETS=$(patsubst %-softmmu,check-functional-%, $(filter %-softmmu,$(TARGETS)))
+.PHONY: $(FUNCTIONAL_TARGETS)
+$(FUNCTIONAL_TARGETS):
+ @$(MAKE) SPEED=thorough $(subst -functional,-func,$@)
+
+.PHONY: check-functional
+check-functional:
+ @$(MAKE) SPEED=thorough check-func check-func-quick
+
# Consolidated targets
.PHONY: check check-clean get-vm-images
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
new file mode 100644
index 0000000000..7fc9c4e513
--- /dev/null
+++ b/tests/functional/meson.build
@@ -0,0 +1,66 @@
+# QEMU functional tests:
+# Tests that are put in the 'quick' category are run by default during
+# 'make check'. Everything that should not be run during 'make check'
+# (e.g. tests that fetch assets from the internet) should be put into
+# the 'thorough' category instead.
+
+# Most tests run too slow with TCI enabled, so skip the functional tests there
+if get_option('tcg_interpreter')
+ subdir_done()
+endif
+
+# Timeouts for individual tests that can be slow e.g. with debugging enabled
+test_timeouts = {
+}
+
+tests_generic = [
+]
+
+tests_x86_64_quick = [
+]
+
+tests_x86_64_thorough = [
+]
+
+foreach speed : ['quick', 'thorough']
+ foreach dir : target_dirs
+ if not dir.endswith('-softmmu')
+ continue
+ endif
+
+ target_base = dir.split('-')[0]
+ test_emulator = emulators['qemu-system-' + target_base]
+
+ if speed == 'quick'
+ suites = ['func-quick', 'func-' + target_base]
+ target_tests = get_variable('tests_' + target_base + '_quick', []) + tests_generic
+ else
+ suites = ['func-' + speed, 'func-' + target_base + '-' + speed, speed]
+ target_tests = get_variable('tests_' + target_base + '_' + speed, [])
+ endif
+
+ test_deps = roms
+ test_env = environment()
+ if have_tools
+ test_env.set('QEMU_TEST_QEMU_IMG', meson.global_build_root() / 'qemu-img')
+ test_deps += [qemu_img]
+ endif
+ test_env.set('QEMU_TEST_QEMU_BINARY',
+ meson.global_build_root() / 'qemu-system-' + target_base)
+ test_env.set('QEMU_BUILD_ROOT', meson.project_build_root())
+ test_env.set('PYTHONPATH', meson.project_source_root() / 'python:' +
+ meson.current_source_dir())
+
+ foreach test : target_tests
+ test('func-@0@/@1@'.format(target_base, test),
+ python,
+ depends: [test_deps, test_emulator, emulator_modules],
+ env: test_env,
+ args: [meson.current_source_dir() / 'test_' + test + '.py'],
+ protocol: 'tap',
+ timeout: test_timeouts.get(test, 60),
+ priority: test_timeouts.get(test, 60),
+ suite: suites)
+ endforeach
+ endforeach
+endforeach
diff --git a/tests/meson.build b/tests/meson.build
index 80dd3029cf..907a4c1c98 100644
--- a/tests/meson.build
+++ b/tests/meson.build
@@ -85,3 +85,4 @@ subdir('unit')
subdir('qapi-schema')
subdir('qtest')
subdir('migration')
+subdir('functional')
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* Re: [PATCH v4 11/35] tests/functional: Prepare the meson build system for the functional tests
2024-08-21 8:27 ` [PATCH v4 11/35] tests/functional: Prepare the meson build system for the functional tests Thomas Huth
@ 2024-08-21 14:30 ` Philippe Mathieu-Daudé
2024-08-23 12:54 ` Philippe Mathieu-Daudé
1 sibling, 0 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-21 14:30 UTC (permalink / raw)
To: Thomas Huth, Alex Bennée, qemu-devel, Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On 21/8/24 10:27, Thomas Huth wrote:
> Provide a meson.build file for the upcoming python-based functional
> tests, and add some wrapper glue targets to the tests/Makefile.include
> file. We are going to use two "speed" modes for the functional tests:
> The "quick" tests can be run at any time (i.e. also during "make check"),
> while the "thorough" tests should only be run when running a
> "make check-functional" test run (since these tests might download
> additional assets from the internet).
>
> The changes to the meson.build files are partly based on an earlier
> patch by Ani Sinha.
>
> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/Makefile.include | 11 ++++++
> tests/functional/meson.build | 66 ++++++++++++++++++++++++++++++++++++
> tests/meson.build | 1 +
> 3 files changed, 78 insertions(+)
> create mode 100644 tests/functional/meson.build
>
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index fbb1a4b211..66c8cc3123 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -9,6 +9,8 @@ check-help:
> @echo "Individual test suites:"
> @echo " $(MAKE) check-qtest-TARGET Run qtest tests for given target"
> @echo " $(MAKE) check-qtest Run qtest tests"
> + @echo " $(MAKE) check-functional Run python-based functional tests"
> + @echo " $(MAKE) check-functional-TARGET Run functional tests for a given target"
> @echo " $(MAKE) check-unit Run qobject tests"
> @echo " $(MAKE) check-qapi-schema Run QAPI schema tests"
> @echo " $(MAKE) check-block Run block tests"
> @@ -152,6 +154,15 @@ check-acceptance-deprecated-warning:
>
> check-acceptance: check-acceptance-deprecated-warning | check-avocado
>
> +FUNCTIONAL_TARGETS=$(patsubst %-softmmu,check-functional-%, $(filter %-softmmu,$(TARGETS)))
This only handles system binaries.
> +.PHONY: $(FUNCTIONAL_TARGETS)
> +$(FUNCTIONAL_TARGETS):
> + @$(MAKE) SPEED=thorough $(subst -functional,-func,$@)
> +
> +.PHONY: check-functional
> +check-functional:
> + @$(MAKE) SPEED=thorough check-func check-func-quick
> +
> # Consolidated targets
>
> .PHONY: check check-clean get-vm-images
> diff --git a/tests/functional/meson.build b/tests/functional/meson.build
> new file mode 100644
> index 0000000000..7fc9c4e513
> --- /dev/null
> +++ b/tests/functional/meson.build
> @@ -0,0 +1,66 @@
> +# QEMU functional tests:
> +# Tests that are put in the 'quick' category are run by default during
> +# 'make check'. Everything that should not be run during 'make check'
> +# (e.g. tests that fetch assets from the internet) should be put into
> +# the 'thorough' category instead.
> +
> +# Most tests run too slow with TCI enabled, so skip the functional tests there
> +if get_option('tcg_interpreter')
> + subdir_done()
> +endif
> +
> +# Timeouts for individual tests that can be slow e.g. with debugging enabled
> +test_timeouts = {
> +}
> +
> +tests_generic = [
> +]
> +
> +tests_x86_64_quick = [
Hard to distinct system/user tests.
> +]
> +
> +tests_x86_64_thorough = [
It is a bit unfortunate to have to modify meson.build to add a test:
this triggers meson to regenerate build files. Good enough for now,
hopefully we will improve.
> +]
> +
> +foreach speed : ['quick', 'thorough']
> + foreach dir : target_dirs
> + if not dir.endswith('-softmmu')
> + continue
> + endif
> +
> + target_base = dir.split('-')[0]
> + test_emulator = emulators['qemu-system-' + target_base]
> +
> + if speed == 'quick'
> + suites = ['func-quick', 'func-' + target_base]
> + target_tests = get_variable('tests_' + target_base + '_quick', []) + tests_generic
> + else
> + suites = ['func-' + speed, 'func-' + target_base + '-' + speed, speed]
> + target_tests = get_variable('tests_' + target_base + '_' + speed, [])
> + endif
> +
> + test_deps = roms
> + test_env = environment()
> + if have_tools
> + test_env.set('QEMU_TEST_QEMU_IMG', meson.global_build_root() / 'qemu-img')
> + test_deps += [qemu_img]
> + endif
> + test_env.set('QEMU_TEST_QEMU_BINARY',
> + meson.global_build_root() / 'qemu-system-' + target_base)
> + test_env.set('QEMU_BUILD_ROOT', meson.project_build_root())
> + test_env.set('PYTHONPATH', meson.project_source_root() / 'python:' +
> + meson.current_source_dir())
> +
> + foreach test : target_tests
> + test('func-@0@/@1@'.format(target_base, test),
> + python,
> + depends: [test_deps, test_emulator, emulator_modules],
> + env: test_env,
> + args: [meson.current_source_dir() / 'test_' + test + '.py'],
> + protocol: 'tap',
> + timeout: test_timeouts.get(test, 60),
> + priority: test_timeouts.get(test, 60),
> + suite: suites)
> + endforeach
> + endforeach
> +endforeach
> diff --git a/tests/meson.build b/tests/meson.build
> index 80dd3029cf..907a4c1c98 100644
> --- a/tests/meson.build
> +++ b/tests/meson.build
> @@ -85,3 +85,4 @@ subdir('unit')
> subdir('qapi-schema')
> subdir('qtest')
> subdir('migration')
> +subdir('functional')
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH v4 11/35] tests/functional: Prepare the meson build system for the functional tests
2024-08-21 8:27 ` [PATCH v4 11/35] tests/functional: Prepare the meson build system for the functional tests Thomas Huth
2024-08-21 14:30 ` Philippe Mathieu-Daudé
@ 2024-08-23 12:54 ` Philippe Mathieu-Daudé
2024-08-26 8:18 ` Thomas Huth
1 sibling, 1 reply; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-23 12:54 UTC (permalink / raw)
To: Thomas Huth, Alex Bennée, qemu-devel, Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On 21/8/24 10:27, Thomas Huth wrote:
> Provide a meson.build file for the upcoming python-based functional
> tests, and add some wrapper glue targets to the tests/Makefile.include
> file. We are going to use two "speed" modes for the functional tests:
> The "quick" tests can be run at any time (i.e. also during "make check"),
> while the "thorough" tests should only be run when running a
> "make check-functional" test run (since these tests might download
> additional assets from the internet).
>
> The changes to the meson.build files are partly based on an earlier
> patch by Ani Sinha.
>
> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/Makefile.include | 11 ++++++
> tests/functional/meson.build | 66 ++++++++++++++++++++++++++++++++++++
> tests/meson.build | 1 +
> 3 files changed, 78 insertions(+)
> create mode 100644 tests/functional/meson.build
> +# Timeouts for individual tests that can be slow e.g. with debugging enabled
> +test_timeouts = {
> +}
> + foreach test : target_tests
> + test('func-@0@/@1@'.format(target_base, test),
> + python,
> + depends: [test_deps, test_emulator, emulator_modules],
> + env: test_env,
> + args: [meson.current_source_dir() / 'test_' + test + '.py'],
> + protocol: 'tap',
> + timeout: test_timeouts.get(test, 60),
> + priority: test_timeouts.get(test, 60),
IIUC with Avocado the timeout was for each test_func in a TestClass.
Now this is only per TestClass. Hopefully I'm wrong...
> + suite: suites)
> + endforeach
> + endforeach
> +endforeach
> diff --git a/tests/meson.build b/tests/meson.build
> index 80dd3029cf..907a4c1c98 100644
> --- a/tests/meson.build
> +++ b/tests/meson.build
> @@ -85,3 +85,4 @@ subdir('unit')
> subdir('qapi-schema')
> subdir('qtest')
> subdir('migration')
> +subdir('functional')
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH v4 11/35] tests/functional: Prepare the meson build system for the functional tests
2024-08-23 12:54 ` Philippe Mathieu-Daudé
@ 2024-08-26 8:18 ` Thomas Huth
2024-08-29 9:54 ` Daniel P. Berrangé
0 siblings, 1 reply; 69+ messages in thread
From: Thomas Huth @ 2024-08-26 8:18 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, Alex Bennée, qemu-devel,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On 23/08/2024 14.54, Philippe Mathieu-Daudé wrote:
> On 21/8/24 10:27, Thomas Huth wrote:
>> Provide a meson.build file for the upcoming python-based functional
>> tests, and add some wrapper glue targets to the tests/Makefile.include
>> file. We are going to use two "speed" modes for the functional tests:
>> The "quick" tests can be run at any time (i.e. also during "make check"),
>> while the "thorough" tests should only be run when running a
>> "make check-functional" test run (since these tests might download
>> additional assets from the internet).
>>
>> The changes to the meson.build files are partly based on an earlier
>> patch by Ani Sinha.
>>
>> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
>> Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>> tests/Makefile.include | 11 ++++++
>> tests/functional/meson.build | 66 ++++++++++++++++++++++++++++++++++++
>> tests/meson.build | 1 +
>> 3 files changed, 78 insertions(+)
>> create mode 100644 tests/functional/meson.build
>
>
>> +# Timeouts for individual tests that can be slow e.g. with debugging enabled
>> +test_timeouts = {
>> +}
>
>
>> + foreach test : target_tests
>> + test('func-@0@/@1@'.format(target_base, test),
>> + python,
>> + depends: [test_deps, test_emulator, emulator_modules],
>> + env: test_env,
>> + args: [meson.current_source_dir() / 'test_' + test + '.py'],
>> + protocol: 'tap',
>> + timeout: test_timeouts.get(test, 60),
>> + priority: test_timeouts.get(test, 60),
>
> IIUC with Avocado the timeout was for each test_func in a TestClass.
> Now this is only per TestClass. Hopefully I'm wrong...
I think you're right ... we might need to adjust the meson timeouts here and
there in case they are causing problems, but that's just business as usual
(we had to do the same when enabling the meson timeouts for the qtests for
example).
Thomas
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH v4 11/35] tests/functional: Prepare the meson build system for the functional tests
2024-08-26 8:18 ` Thomas Huth
@ 2024-08-29 9:54 ` Daniel P. Berrangé
0 siblings, 0 replies; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-08-29 9:54 UTC (permalink / raw)
To: Thomas Huth
Cc: Philippe Mathieu-Daudé, Alex Bennée, qemu-devel,
Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On Mon, Aug 26, 2024 at 10:18:44AM +0200, Thomas Huth wrote:
> On 23/08/2024 14.54, Philippe Mathieu-Daudé wrote:
> > On 21/8/24 10:27, Thomas Huth wrote:
> > > Provide a meson.build file for the upcoming python-based functional
> > > tests, and add some wrapper glue targets to the tests/Makefile.include
> > > file. We are going to use two "speed" modes for the functional tests:
> > > The "quick" tests can be run at any time (i.e. also during "make check"),
> > > while the "thorough" tests should only be run when running a
> > > "make check-functional" test run (since these tests might download
> > > additional assets from the internet).
> > >
> > > The changes to the meson.build files are partly based on an earlier
> > > patch by Ani Sinha.
> > >
> > > Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> > > Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> > > Signed-off-by: Thomas Huth <thuth@redhat.com>
> > > ---
> > > tests/Makefile.include | 11 ++++++
> > > tests/functional/meson.build | 66 ++++++++++++++++++++++++++++++++++++
> > > tests/meson.build | 1 +
> > > 3 files changed, 78 insertions(+)
> > > create mode 100644 tests/functional/meson.build
> >
> >
> > > +# Timeouts for individual tests that can be slow e.g. with debugging enabled
> > > +test_timeouts = {
> > > +}
> >
> >
> > > + foreach test : target_tests
> > > + test('func-@0@/@1@'.format(target_base, test),
> > > + python,
> > > + depends: [test_deps, test_emulator, emulator_modules],
> > > + env: test_env,
> > > + args: [meson.current_source_dir() / 'test_' + test + '.py'],
> > > + protocol: 'tap',
> > > + timeout: test_timeouts.get(test, 60),
> > > + priority: test_timeouts.get(test, 60),
> >
> > IIUC with Avocado the timeout was for each test_func in a TestClass.
> > Now this is only per TestClass. Hopefully I'm wrong...
>
> I think you're right ... we might need to adjust the meson timeouts here and
> there in case they are causing problems, but that's just business as usual
> (we had to do the same when enabling the meson timeouts for the qtests for
> example).
Meson timeouts are around the lifetime of whatever child process
we tell it to execute. So, yes, the timeout is for the .py file
as a whole, not the individual TestClass within it.
This is different from Avocado, but not a big problem. Worst case
we'll adjust some meson level timeouts, or split overly long running
.py test files into multiple test .py files.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 69+ messages in thread
* [PATCH v4 12/35] tests/functional: Convert simple avocado tests into standalone python tests
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (10 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 11/35] tests/functional: Prepare the meson build system for the functional tests Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 8:27 ` [PATCH v4 13/35] tests/functional: Convert avocado tests that just need a small adjustment Thomas Huth
` (22 subsequent siblings)
34 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
These test are rather simple and don't need any modifications apart
from adjusting the "from avocado_qemu" line. To ease debugging, make
the files executable and add a shebang line and Python '__main__'
handling, too, so that these tests can now be run by executing them
directly.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 5 ++
tests/functional/meson.build | 5 ++
.../test_cpu_queries.py} | 12 +++--
.../test_empty_cpu_model.py} | 7 ++-
.../test_mem_addr_space.py} | 52 +++----------------
.../test_pc_cpu_hotplug_props.py} | 11 ++--
.../test_virtio_version.py} | 8 +--
7 files changed, 40 insertions(+), 60 deletions(-)
rename tests/{avocado/cpu_queries.py => functional/test_cpu_queries.py} (85%)
mode change 100644 => 100755
rename tests/{avocado/empty_cpu_model.py => functional/test_empty_cpu_model.py} (84%)
mode change 100644 => 100755
rename tests/{avocado/mem-addr-space-check.py => functional/test_mem_addr_space.py} (93%)
mode change 100644 => 100755
rename tests/{avocado/pc_cpu_hotplug_props.py => functional/test_pc_cpu_hotplug_props.py} (90%)
mode change 100644 => 100755
rename tests/{avocado/virtio_version.py => functional/test_virtio_version.py} (98%)
mode change 100644 => 100755
diff --git a/MAINTAINERS b/MAINTAINERS
index 80cc84dd99..b3a0ac0e8b 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1833,6 +1833,8 @@ F: hw/isa/apm.c
F: include/hw/isa/apm.h
F: tests/unit/test-x86-topo.c
F: tests/qtest/test-x86-cpuid-compat.c
+F: tests/functional/test_mem_addr_space.py
+F: tests/functional/test_pc_cpu_hotplug_props.py
PC Chipset
M: Michael S. Tsirkin <mst@redhat.com>
@@ -1899,6 +1901,8 @@ F: include/hw/boards.h
F: include/hw/core/cpu.h
F: include/hw/cpu/cluster.h
F: include/sysemu/numa.h
+F: tests/functional/test_cpu_queries.py
+F: tests/functional/test_empty_cpu_model.py
F: tests/unit/test-smp-parse.c
T: git https://gitlab.com/ehabkost/qemu.git machine-next
@@ -2238,6 +2242,7 @@ F: net/vhost-user.c
F: include/hw/virtio/
F: docs/devel/virtio*
F: docs/devel/migration/virtio.rst
+F: tests/functional/test_virtio_version.py
virtio-balloon
M: Michael S. Tsirkin <mst@redhat.com>
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 7fc9c4e513..a8cc0e6330 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -14,9 +14,14 @@ test_timeouts = {
}
tests_generic = [
+ 'empty_cpu_model',
]
tests_x86_64_quick = [
+ 'cpu_queries',
+ 'mem_addr_space',
+ 'pc_cpu_hotplug_props',
+ 'virtio_version',
]
tests_x86_64_thorough = [
diff --git a/tests/avocado/cpu_queries.py b/tests/functional/test_cpu_queries.py
old mode 100644
new mode 100755
similarity index 85%
rename from tests/avocado/cpu_queries.py
rename to tests/functional/test_cpu_queries.py
index d3faa14720..b1122a0e8f
--- a/tests/avocado/cpu_queries.py
+++ b/tests/functional/test_cpu_queries.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Sanity check of query-cpu-* results
#
# Copyright (c) 2019 Red Hat, Inc.
@@ -8,7 +10,7 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado_qemu import QemuSystemTest
+from qemu_test import QemuSystemTest
class QueryCPUModelExpansion(QemuSystemTest):
"""
@@ -16,10 +18,7 @@ class QueryCPUModelExpansion(QemuSystemTest):
"""
def test(self):
- """
- :avocado: tags=arch:x86_64
- :avocado: tags=machine:none
- """
+ self.set_machine('none')
self.vm.add_args('-S')
self.vm.launch()
@@ -33,3 +32,6 @@ def test(self):
e = self.vm.cmd('query-cpu-model-expansion', model=model,
type='full')
self.assertEqual(e['model']['name'], c['name'])
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/empty_cpu_model.py b/tests/functional/test_empty_cpu_model.py
old mode 100644
new mode 100755
similarity index 84%
rename from tests/avocado/empty_cpu_model.py
rename to tests/functional/test_empty_cpu_model.py
index d906ef3d3c..0081b06d85
--- a/tests/avocado/empty_cpu_model.py
+++ b/tests/functional/test_empty_cpu_model.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Check for crash when using empty -cpu option
#
# Copyright (c) 2019 Red Hat, Inc.
@@ -7,7 +9,7 @@
#
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado_qemu import QemuSystemTest
+from qemu_test import QemuSystemTest
class EmptyCPUModel(QemuSystemTest):
def test(self):
@@ -17,3 +19,6 @@ def test(self):
self.vm.wait()
self.assertEqual(self.vm.exitcode(), 1, "QEMU exit code should be 1")
self.assertRegex(self.vm.get_log(), r'-cpu option cannot be empty')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/mem-addr-space-check.py b/tests/functional/test_mem_addr_space.py
old mode 100644
new mode 100755
similarity index 93%
rename from tests/avocado/mem-addr-space-check.py
rename to tests/functional/test_mem_addr_space.py
index d3974599f4..bb0cf062ca
--- a/tests/avocado/mem-addr-space-check.py
+++ b/tests/functional/test_mem_addr_space.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Check for crash when using memory beyond the available guest processor
# address space.
#
@@ -8,7 +10,7 @@
#
# SPDX-License-Identifier: GPL-2.0-or-later
-from avocado_qemu import QemuSystemTest
+from qemu_test import QemuSystemTest
import time
class MemAddrCheck(QemuSystemTest):
@@ -22,9 +24,6 @@ class MemAddrCheck(QemuSystemTest):
# for all 32-bit cases, pci64_hole_size is 0.
def test_phybits_low_pse36(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
With pse36 feature ON, a processor has 36 bits of addressing. So it can
access up to a maximum of 64GiB of memory. Memory hotplug region begins
at 4 GiB boundary when "above_4g_mem_size" is 0 (this would be true when
@@ -52,9 +51,6 @@ def test_phybits_low_pse36(self):
def test_phybits_low_pae(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
With pae feature ON, a processor has 36 bits of addressing. So it can
access up to a maximum of 64GiB of memory. Rest is the same as the case
with pse36 above.
@@ -72,9 +68,6 @@ def test_phybits_low_pae(self):
def test_phybits_ok_pentium_pse36(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
Setting maxmem to 59.5G and making sure that QEMU can start with the
same options as the failing case above with pse36 cpu feature.
"""
@@ -91,9 +84,6 @@ def test_phybits_ok_pentium_pse36(self):
def test_phybits_ok_pentium_pae(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
Test is same as above but now with pae cpu feature turned on.
Setting maxmem to 59.5G and making sure that QEMU can start fine
with the same options as the case above.
@@ -111,9 +101,6 @@ def test_phybits_ok_pentium_pae(self):
def test_phybits_ok_pentium2(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
Pentium2 has 36 bits of addressing, so its same as pentium
with pse36 ON.
"""
@@ -130,9 +117,6 @@ def test_phybits_ok_pentium2(self):
def test_phybits_low_nonpse36(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
Pentium processor has 32 bits of addressing without pse36 or pae
so it can access physical address up to 4 GiB. Setting maxmem to
4 GiB should make QEMU fail to start with "phys-bits too low"
@@ -153,9 +137,6 @@ def test_phybits_low_nonpse36(self):
# now lets test some 64-bit CPU cases.
def test_phybits_low_tcg_q35_70_amd(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
For q35 7.1 machines and above, there is a HT window that starts at
1024 GiB and ends at 1 TiB - 1. If the max GPA falls in this range,
"above_4G" memory is adjusted to start at 1 TiB boundary for AMD cpus
@@ -182,9 +163,6 @@ def test_phybits_low_tcg_q35_70_amd(self):
def test_phybits_low_tcg_q35_71_amd(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
AMD_HT_START is defined to be at 1012 GiB. So for q35 machines
version > 7.0 and AMD cpus, instead of 1024 GiB limit for 40 bit
processor address space, it has to be 1012 GiB , that is 12 GiB
@@ -205,9 +183,6 @@ def test_phybits_low_tcg_q35_71_amd(self):
def test_phybits_ok_tcg_q35_70_amd(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
Same as q35-7.0 AMD case except that here we check that QEMU can
successfully start when maxmem is < 988G.
"""
@@ -224,9 +199,6 @@ def test_phybits_ok_tcg_q35_70_amd(self):
def test_phybits_ok_tcg_q35_71_amd(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
Same as q35-7.1 AMD case except that here we check that QEMU can
successfully start when maxmem is < 976G.
"""
@@ -243,9 +215,6 @@ def test_phybits_ok_tcg_q35_71_amd(self):
def test_phybits_ok_tcg_q35_71_intel(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
Same parameters as test_phybits_low_tcg_q35_71_amd() but use
Intel cpu instead. QEMU should start fine in this case as
"above_4G" memory starts at 4G.
@@ -264,9 +233,6 @@ def test_phybits_ok_tcg_q35_71_intel(self):
def test_phybits_low_tcg_q35_71_amd_41bits(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
AMD processor with 41 bits. Max cpu hw address = 2 TiB.
By setting maxram above 1012 GiB - 32 GiB - 4 GiB = 976 GiB, we can
force "above_4G" memory to start at 1 TiB for q35-7.1 machines
@@ -291,9 +257,6 @@ def test_phybits_low_tcg_q35_71_amd_41bits(self):
def test_phybits_ok_tcg_q35_71_amd_41bits(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
AMD processor with 41 bits. Max cpu hw address = 2 TiB.
Same as above but by setting maxram between 976 GiB and 992 Gib,
QEMU should start fine.
@@ -312,9 +275,6 @@ def test_phybits_ok_tcg_q35_71_amd_41bits(self):
def test_phybits_low_tcg_q35_intel_cxl(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
cxl memory window starts after memory device range. Here, we use 1 GiB
of cxl window memory. 4G_mem end aligns at 4G. pci64_hole is 32 GiB and
starts after the cxl memory window.
@@ -335,9 +295,6 @@ def test_phybits_low_tcg_q35_intel_cxl(self):
def test_phybits_ok_tcg_q35_intel_cxl(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
Same as above but here we do not reserve any cxl memory window. Hence,
with the exact same parameters as above, QEMU should start fine even
with cxl enabled.
@@ -352,3 +309,6 @@ def test_phybits_ok_tcg_q35_intel_cxl(self):
time.sleep(self.DELAY_Q35_BOOT_SEQUENCE)
self.vm.shutdown()
self.assertNotRegex(self.vm.get_log(), r'phys-bits too low')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/pc_cpu_hotplug_props.py b/tests/functional/test_pc_cpu_hotplug_props.py
old mode 100644
new mode 100755
similarity index 90%
rename from tests/avocado/pc_cpu_hotplug_props.py
rename to tests/functional/test_pc_cpu_hotplug_props.py
index 4bd3e02665..9d5a37cb17
--- a/tests/avocado/pc_cpu_hotplug_props.py
+++ b/tests/functional/test_pc_cpu_hotplug_props.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
#
# Ensure CPU die-id can be omitted on -device
#
@@ -20,16 +21,16 @@
# License along with this library; if not, see <http://www.gnu.org/licenses/>.
#
-from avocado_qemu import QemuSystemTest
+from qemu_test import QemuSystemTest
class OmittedCPUProps(QemuSystemTest):
- """
- :avocado: tags=arch:x86_64
- :avocado: tags=cpu:qemu64
- """
+
def test_no_die_id(self):
self.vm.add_args('-nodefaults', '-S')
self.vm.add_args('-smp', '1,sockets=2,cores=2,threads=2,maxcpus=8')
self.vm.add_args('-device', 'qemu64-x86_64-cpu,socket-id=1,core-id=0,thread-id=0')
self.vm.launch()
self.assertEqual(len(self.vm.cmd('query-cpus-fast')), 2)
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/virtio_version.py b/tests/functional/test_virtio_version.py
old mode 100644
new mode 100755
similarity index 98%
rename from tests/avocado/virtio_version.py
rename to tests/functional/test_virtio_version.py
index afe5e828b5..eb23060564
--- a/tests/avocado/virtio_version.py
+++ b/tests/functional/test_virtio_version.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
"""
Check compatibility of virtio device types
"""
@@ -12,7 +13,7 @@
import os
from qemu.machine import QEMUMachine
-from avocado_qemu import QemuSystemTest
+from qemu_test import QemuSystemTest
# Virtio Device IDs:
VIRTIO_NET = 1
@@ -60,8 +61,6 @@ class VirtioVersionCheck(QemuSystemTest):
Check if virtio-version-specific device types result in the
same device tree created by `disable-modern` and
`disable-legacy`.
-
- :avocado: tags=arch:x86_64
"""
# just in case there are failures, show larger diff:
@@ -173,3 +172,6 @@ def test_modern_only_devs(self):
self.check_modern_only('virtio-mouse-pci', VIRTIO_INPUT)
self.check_modern_only('virtio-tablet-pci', VIRTIO_INPUT)
self.check_modern_only('virtio-keyboard-pci', VIRTIO_INPUT)
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH v4 13/35] tests/functional: Convert avocado tests that just need a small adjustment
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (11 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 12/35] tests/functional: Convert simple avocado tests into standalone python tests Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 8:27 ` [PATCH v4 14/35] tests/functional: add a module for handling asset download & caching Thomas Huth
` (21 subsequent siblings)
34 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
These simple tests can be converted to stand-alone tests quite easily,
e.g. by just setting the machine to 'none' now manually or by adding
"-cpu" command line parameters, since we don't support the corresponding
avocado tags in the new python test framework.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 3 +
tests/functional/meson.build | 6 ++
.../test_info_usernet.py} | 11 ++-
.../test_ppc_74xx.py} | 74 ++++++++-----------
.../version.py => functional/test_version.py} | 13 ++--
5 files changed, 56 insertions(+), 51 deletions(-)
rename tests/{avocado/info_usernet.py => functional/test_info_usernet.py} (87%)
mode change 100644 => 100755
rename tests/{avocado/ppc_74xx.py => functional/test_ppc_74xx.py} (74%)
mode change 100644 => 100755
rename tests/{avocado/version.py => functional/test_version.py} (78%)
mode change 100644 => 100755
diff --git a/MAINTAINERS b/MAINTAINERS
index b3a0ac0e8b..4912f8d0da 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -318,6 +318,7 @@ F: configs/devices/ppc*
F: docs/system/ppc/embedded.rst
F: docs/system/target-ppc.rst
F: tests/tcg/ppc*/*
+F: tests/functional/test_ppc_74xx.py
RISC-V TCG CPUs
M: Palmer Dabbelt <palmer@dabbelt.com>
@@ -2107,6 +2108,7 @@ S: Odd Fixes
F: hw/net/
F: include/hw/net/
F: tests/qtest/virtio-net-test.c
+F: tests/functional/test_info_usernet.py
F: docs/virtio-net-failover.rst
T: git https://github.com/jasowang/qemu.git net
@@ -2976,6 +2978,7 @@ S: Supported
F: include/qemu/option.h
F: tests/unit/test-keyval.c
F: tests/unit/test-qemu-opts.c
+F: tests/functional/test_version.py
F: util/keyval.c
F: util/qemu-option.c
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index a8cc0e6330..8a8fa0ab99 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -15,6 +15,12 @@ test_timeouts = {
tests_generic = [
'empty_cpu_model',
+ 'info_usernet',
+ 'version',
+]
+
+tests_ppc_quick = [
+ 'ppc_74xx',
]
tests_x86_64_quick = [
diff --git a/tests/avocado/info_usernet.py b/tests/functional/test_info_usernet.py
old mode 100644
new mode 100755
similarity index 87%
rename from tests/avocado/info_usernet.py
rename to tests/functional/test_info_usernet.py
index e1aa7a6e0a..cd37524d94
--- a/tests/avocado/info_usernet.py
+++ b/tests/functional/test_info_usernet.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Test for the hmp command "info usernet"
#
# Copyright (c) 2021 Red Hat, Inc.
@@ -8,18 +10,16 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado_qemu import QemuSystemTest
+from qemu_test import QemuSystemTest
from qemu.utils import get_info_usernet_hostfwd_port
class InfoUsernet(QemuSystemTest):
- """
- :avocado: tags=machine:none
- """
def test_hostfwd(self):
self.require_netdev('user')
+ self.set_machine('none')
self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22')
self.vm.launch()
res = self.vm.cmd('human-monitor-command',
@@ -31,3 +31,6 @@ def test_hostfwd(self):
self.assertGreater(port, 0,
('Found a redirected port that is not greater than'
' zero'))
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/ppc_74xx.py b/tests/functional/test_ppc_74xx.py
old mode 100644
new mode 100755
similarity index 74%
rename from tests/avocado/ppc_74xx.py
rename to tests/functional/test_ppc_74xx.py
index f54757c243..5386016f26
--- a/tests/avocado/ppc_74xx.py
+++ b/tests/functional/test_ppc_74xx.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Smoke tests for 74xx cpus (aka G4).
#
# Copyright (c) 2021, IBM Corp.
@@ -5,132 +7,120 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
+from qemu_test import QemuSystemTest
+from qemu_test import wait_for_console_pattern
class ppc74xxCpu(QemuSystemTest):
- """
- :avocado: tags=arch:ppc
- :avocado: tags=accel:tcg
- """
+
timeout = 5
def test_ppc_7400(self):
- """
- :avocado: tags=cpu:7400
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7400')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7410(self):
- """
- :avocado: tags=cpu:7410
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7410')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,74xx')
def test_ppc_7441(self):
- """
- :avocado: tags=cpu:7441
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7441')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7445(self):
- """
- :avocado: tags=cpu:7445
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7445')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7447(self):
- """
- :avocado: tags=cpu:7447
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7447')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7447a(self):
- """
- :avocado: tags=cpu:7447a
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7447a')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7448(self):
- """
- :avocado: tags=cpu:7448
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7448')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,MPC86xx')
def test_ppc_7450(self):
- """
- :avocado: tags=cpu:7450
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7450')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7451(self):
- """
- :avocado: tags=cpu:7451
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7451')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7455(self):
- """
- :avocado: tags=cpu:7455
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7455')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7457(self):
- """
- :avocado: tags=cpu:7457
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7457')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7457a(self):
- """
- :avocado: tags=cpu:7457a
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7457a')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/version.py b/tests/functional/test_version.py
old mode 100644
new mode 100755
similarity index 78%
rename from tests/avocado/version.py
rename to tests/functional/test_version.py
index c6139568a1..3ab3b67f7e
--- a/tests/avocado/version.py
+++ b/tests/functional/test_version.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Version check example test
#
# Copyright (c) 2018 Red Hat, Inc.
@@ -9,17 +11,18 @@
# later. See the COPYING file in the top-level directory.
-from avocado_qemu import QemuSystemTest
+from qemu_test import QemuSystemTest
class Version(QemuSystemTest):
- """
- :avocado: tags=quick
- :avocado: tags=machine:none
- """
+
def test_qmp_human_info_version(self):
+ self.set_machine('none')
self.vm.add_args('-nodefaults')
self.vm.launch()
res = self.vm.cmd('human-monitor-command',
command_line='info version')
self.assertRegex(res, r'^(\d+\.\d+\.\d)')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH v4 14/35] tests/functional: add a module for handling asset download & caching
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (12 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 13/35] tests/functional: Convert avocado tests that just need a small adjustment Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 14:49 ` Philippe Mathieu-Daudé
2024-08-23 6:24 ` Philippe Mathieu-Daudé
2024-08-21 8:27 ` [PATCH v4 15/35] tests/functional: enable pre-emptive caching of assets Thomas Huth
` (20 subsequent siblings)
34 siblings, 2 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
From: Daniel P. Berrangé <berrange@redhat.com>
The 'Asset' class is a simple module that declares a downloadable
asset that can be cached locally. Downloads are stored in the user's
home dir at ~/.cache/qemu/download, using a sha256 sum of the URL.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
[thuth: Drop sha1 support, use hash on file content for naming instead of URL,
add the possibility to specify the cache dir via environment variable]
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/functional/qemu_test/__init__.py | 1 +
tests/functional/qemu_test/asset.py | 97 ++++++++++++++++++++++++++
2 files changed, 98 insertions(+)
create mode 100644 tests/functional/qemu_test/asset.py
diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu_test/__init__.py
index 2f1e0bc70d..db05c8f412 100644
--- a/tests/functional/qemu_test/__init__.py
+++ b/tests/functional/qemu_test/__init__.py
@@ -6,6 +6,7 @@
# later. See the COPYING file in the top-level directory.
+from .asset import Asset
from .config import BUILD_DIR
from .cmd import has_cmd, has_cmds, run_cmd, is_readable_executable_file, \
interrupt_interactive_console_until_pattern, wait_for_console_pattern, \
diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_test/asset.py
new file mode 100644
index 0000000000..cbeb6278af
--- /dev/null
+++ b/tests/functional/qemu_test/asset.py
@@ -0,0 +1,97 @@
+# Test utilities for fetching & caching assets
+#
+# Copyright 2024 Red Hat, Inc.
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import hashlib
+import logging
+import os
+import subprocess
+import urllib.request
+from pathlib import Path
+from shutil import copyfileobj
+
+
+# Instances of this class must be declared as class level variables
+# starting with a name "ASSET_". This enables the pre-caching logic
+# to easily find all referenced assets and download them prior to
+# execution of the tests.
+class Asset:
+
+ def __init__(self, url, hashsum):
+ self.url = url
+ self.hash = hashsum
+ cache_dir_env = os.getenv('QEMU_TEST_CACHE_DIR')
+ if cache_dir_env:
+ self.cache_dir = Path(cache_dir_env, "download")
+ else:
+ self.cache_dir = Path(Path("~").expanduser(),
+ ".cache", "qemu", "download")
+ self.cache_file = Path(self.cache_dir, hashsum)
+ self.log = logging.getLogger('qemu-test')
+
+ def __repr__(self):
+ return "Asset: url=%s hash=%s cache=%s" % (
+ self.url, self.hash, self.cache_file)
+
+ def _check(self, cache_file):
+ if self.hash is None:
+ return True
+ if len(self.hash) == 64:
+ sum_prog = 'sha256sum'
+ elif len(self.hash) == 128:
+ sum_prog = 'sha512sum'
+ else:
+ raise Exception("unknown hash type")
+
+ checksum = subprocess.check_output(
+ [sum_prog, str(cache_file)]).split()[0]
+ return self.hash == checksum.decode("utf-8")
+
+ def valid(self):
+ return self.cache_file.exists() and self._check(self.cache_file)
+
+ def fetch(self):
+ if not self.cache_dir.exists():
+ self.cache_dir.mkdir(parents=True, exist_ok=True)
+
+ if self.valid():
+ self.log.debug("Using cached asset %s for %s",
+ self.cache_file, self.url)
+ return str(self.cache_file)
+
+ self.log.info("Downloading %s to %s...", self.url, self.cache_file)
+ tmp_cache_file = self.cache_file.with_suffix(".download")
+
+ try:
+ resp = urllib.request.urlopen(self.url)
+ except Exception as e:
+ self.log.error("Unable to download %s: %s", self.url, e)
+ raise
+
+ try:
+ with tmp_cache_file.open("wb+") as dst:
+ copyfileobj(resp, dst)
+ except:
+ tmp_cache_file.unlink()
+ raise
+ try:
+ # Set these just for informational purposes
+ os.setxattr(str(tmp_cache_file), "user.qemu-asset-url",
+ self.url.encode('utf8'))
+ os.setxattr(str(tmp_cache_file), "user.qemu-asset-hash",
+ self.hash.encode('utf8'))
+ except Exception as e:
+ self.log.info("Unable to set xattr on %s: %s", tmp_cache_file, e)
+ pass
+
+ if not self._check(tmp_cache_file):
+ tmp_cache_file.unlink()
+ raise Exception("Hash of %s does not match %s" %
+ (self.url, self.hash))
+ tmp_cache_file.replace(self.cache_file)
+
+ self.log.info("Cached %s at %s" % (self.url, self.cache_file))
+ return str(self.cache_file)
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* Re: [PATCH v4 14/35] tests/functional: add a module for handling asset download & caching
2024-08-21 8:27 ` [PATCH v4 14/35] tests/functional: add a module for handling asset download & caching Thomas Huth
@ 2024-08-21 14:49 ` Philippe Mathieu-Daudé
2024-08-29 9:57 ` Daniel P. Berrangé
2024-08-23 6:24 ` Philippe Mathieu-Daudé
1 sibling, 1 reply; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-21 14:49 UTC (permalink / raw)
To: Thomas Huth, Alex Bennée, qemu-devel, Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On 21/8/24 10:27, Thomas Huth wrote:
> From: Daniel P. Berrangé <berrange@redhat.com>
>
> The 'Asset' class is a simple module that declares a downloadable
> asset that can be cached locally. Downloads are stored in the user's
> home dir at ~/.cache/qemu/download, using a sha256 sum of the URL.
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> [thuth: Drop sha1 support, use hash on file content for naming instead of URL,
> add the possibility to specify the cache dir via environment variable]
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/functional/qemu_test/__init__.py | 1 +
> tests/functional/qemu_test/asset.py | 97 ++++++++++++++++++++++++++
> 2 files changed, 98 insertions(+)
> create mode 100644 tests/functional/qemu_test/asset.py
>
> diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu_test/__init__.py
> index 2f1e0bc70d..db05c8f412 100644
> --- a/tests/functional/qemu_test/__init__.py
> +++ b/tests/functional/qemu_test/__init__.py
> @@ -6,6 +6,7 @@
> # later. See the COPYING file in the top-level directory.
>
>
> +from .asset import Asset
> from .config import BUILD_DIR
> from .cmd import has_cmd, has_cmds, run_cmd, is_readable_executable_file, \
> interrupt_interactive_console_until_pattern, wait_for_console_pattern, \
> diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_test/asset.py
> new file mode 100644
> index 0000000000..cbeb6278af
> --- /dev/null
> +++ b/tests/functional/qemu_test/asset.py
> @@ -0,0 +1,97 @@
> +# Test utilities for fetching & caching assets
> +#
> +# Copyright 2024 Red Hat, Inc.
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or
> +# later. See the COPYING file in the top-level directory.
> +
> +import hashlib
> +import logging
> +import os
> +import subprocess
> +import urllib.request
> +from pathlib import Path
> +from shutil import copyfileobj
> +
> +
> +# Instances of this class must be declared as class level variables
> +# starting with a name "ASSET_". This enables the pre-caching logic
> +# to easily find all referenced assets and download them prior to
> +# execution of the tests.
> +class Asset:
> +
> + def __init__(self, url, hashsum):
> + self.url = url
> + self.hash = hashsum
> + cache_dir_env = os.getenv('QEMU_TEST_CACHE_DIR')
> + if cache_dir_env:
> + self.cache_dir = Path(cache_dir_env, "download")
> + else:
> + self.cache_dir = Path(Path("~").expanduser(),
> + ".cache", "qemu", "download")
> + self.cache_file = Path(self.cache_dir, hashsum)
> + self.log = logging.getLogger('qemu-test')
> +
> + def __repr__(self):
> + return "Asset: url=%s hash=%s cache=%s" % (
> + self.url, self.hash, self.cache_file)
> +
> + def _check(self, cache_file):
> + if self.hash is None:
> + return True
> + if len(self.hash) == 64:
> + sum_prog = 'sha256sum'
> + elif len(self.hash) == 128:
> + sum_prog = 'sha512sum'
> + else:
> + raise Exception("unknown hash type")
> +
> + checksum = subprocess.check_output(
> + [sum_prog, str(cache_file)]).split()[0]
> + return self.hash == checksum.decode("utf-8")
> +
> + def valid(self):
> + return self.cache_file.exists() and self._check(self.cache_file)
> +
> + def fetch(self):
> + if not self.cache_dir.exists():
> + self.cache_dir.mkdir(parents=True, exist_ok=True)
This doesn't work with QEMU_TEST_CACHE_DIR set to someone else:
File
"/home/philippe.mathieu-daude/qemu/tests/functional/qemu_test/asset.py",
line 60, in fetch
self.cache_dir.mkdir(parents=True, exist_ok=True)
File "/usr/lib/python3.10/pathlib.py", line 1175, in mkdir
self._accessor.mkdir(self, mode)
PermissionError: [Errno 13] Permission denied:
'/home/alex.bennee/.cache/qemu/download'
ninja: build stopped: subcommand failed.
Maybe use a getter which falls back to Path("~").expanduser() when
no access on QEMU_TEST_CACHE_DIR? This happens when downloading a
new file (the recent MIPS tests I converted) which isn't in Alex's cache:
2024-08-21 15:45:48,896 - qemu-test - INFO - Attempting to cache 'Asset:
url=https://s3-eu-west-1.amazonaws.com/downloads-mips/mips-downloads/YAMON/yamon-bin-02.22.zip
hash=eef86f0eed0ef554f041dcd47b87eebea0e6f9f1184ed31f7e9e8b4a803860ab
cache=/home/alex.bennee/.cache/download/eef86f0eed0ef554f041dcd47b87eebea0e6f9f1184ed31f7e9e8b4a803860ab'
> + if self.valid():
> + self.log.debug("Using cached asset %s for %s",
> + self.cache_file, self.url)
> + return str(self.cache_file)
> +
> + self.log.info("Downloading %s to %s...", self.url, self.cache_file)
> + tmp_cache_file = self.cache_file.with_suffix(".download")
> +
> + try:
> + resp = urllib.request.urlopen(self.url)
> + except Exception as e:
> + self.log.error("Unable to download %s: %s", self.url, e)
> + raise
> +
> + try:
> + with tmp_cache_file.open("wb+") as dst:
> + copyfileobj(resp, dst)
> + except:
> + tmp_cache_file.unlink()
> + raise
> + try:
> + # Set these just for informational purposes
> + os.setxattr(str(tmp_cache_file), "user.qemu-asset-url",
> + self.url.encode('utf8'))
> + os.setxattr(str(tmp_cache_file), "user.qemu-asset-hash",
> + self.hash.encode('utf8'))
> + except Exception as e:
> + self.log.info("Unable to set xattr on %s: %s", tmp_cache_file, e)
> + pass
> +
> + if not self._check(tmp_cache_file):
> + tmp_cache_file.unlink()
> + raise Exception("Hash of %s does not match %s" %
> + (self.url, self.hash))
> + tmp_cache_file.replace(self.cache_file)
> +
> + self.log.info("Cached %s at %s" % (self.url, self.cache_file))
> + return str(self.cache_file)
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH v4 14/35] tests/functional: add a module for handling asset download & caching
2024-08-21 14:49 ` Philippe Mathieu-Daudé
@ 2024-08-29 9:57 ` Daniel P. Berrangé
0 siblings, 0 replies; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-08-29 9:57 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: Thomas Huth, Alex Bennée, qemu-devel, Ani Sinha,
Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On Wed, Aug 21, 2024 at 04:49:42PM +0200, Philippe Mathieu-Daudé wrote:
> On 21/8/24 10:27, Thomas Huth wrote:
> > From: Daniel P. Berrangé <berrange@redhat.com>
> >
> > The 'Asset' class is a simple module that declares a downloadable
> > asset that can be cached locally. Downloads are stored in the user's
> > home dir at ~/.cache/qemu/download, using a sha256 sum of the URL.
> >
> > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> > Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> > Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> > [thuth: Drop sha1 support, use hash on file content for naming instead of URL,
> > add the possibility to specify the cache dir via environment variable]
> > Signed-off-by: Thomas Huth <thuth@redhat.com>
> > ---
> > tests/functional/qemu_test/__init__.py | 1 +
> > tests/functional/qemu_test/asset.py | 97 ++++++++++++++++++++++++++
> > 2 files changed, 98 insertions(+)
> > create mode 100644 tests/functional/qemu_test/asset.py
> >
> > diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu_test/__init__.py
> > index 2f1e0bc70d..db05c8f412 100644
> > --- a/tests/functional/qemu_test/__init__.py
> > +++ b/tests/functional/qemu_test/__init__.py
> > @@ -6,6 +6,7 @@
> > # later. See the COPYING file in the top-level directory.
> > +from .asset import Asset
> > from .config import BUILD_DIR
> > from .cmd import has_cmd, has_cmds, run_cmd, is_readable_executable_file, \
> > interrupt_interactive_console_until_pattern, wait_for_console_pattern, \
> > diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_test/asset.py
> > new file mode 100644
> > index 0000000000..cbeb6278af
> > --- /dev/null
> > +++ b/tests/functional/qemu_test/asset.py
> > @@ -0,0 +1,97 @@
> > +# Test utilities for fetching & caching assets
> > +#
> > +# Copyright 2024 Red Hat, Inc.
> > +#
> > +# This work is licensed under the terms of the GNU GPL, version 2 or
> > +# later. See the COPYING file in the top-level directory.
> > +
> > +import hashlib
> > +import logging
> > +import os
> > +import subprocess
> > +import urllib.request
> > +from pathlib import Path
> > +from shutil import copyfileobj
> > +
> > +
> > +# Instances of this class must be declared as class level variables
> > +# starting with a name "ASSET_". This enables the pre-caching logic
> > +# to easily find all referenced assets and download them prior to
> > +# execution of the tests.
> > +class Asset:
> > +
> > + def __init__(self, url, hashsum):
> > + self.url = url
> > + self.hash = hashsum
> > + cache_dir_env = os.getenv('QEMU_TEST_CACHE_DIR')
> > + if cache_dir_env:
> > + self.cache_dir = Path(cache_dir_env, "download")
> > + else:
> > + self.cache_dir = Path(Path("~").expanduser(),
> > + ".cache", "qemu", "download")
> > + self.cache_file = Path(self.cache_dir, hashsum)
> > + self.log = logging.getLogger('qemu-test')
> > +
> > + def __repr__(self):
> > + return "Asset: url=%s hash=%s cache=%s" % (
> > + self.url, self.hash, self.cache_file)
> > +
> > + def _check(self, cache_file):
> > + if self.hash is None:
> > + return True
> > + if len(self.hash) == 64:
> > + sum_prog = 'sha256sum'
> > + elif len(self.hash) == 128:
> > + sum_prog = 'sha512sum'
> > + else:
> > + raise Exception("unknown hash type")
> > +
> > + checksum = subprocess.check_output(
> > + [sum_prog, str(cache_file)]).split()[0]
> > + return self.hash == checksum.decode("utf-8")
> > +
> > + def valid(self):
> > + return self.cache_file.exists() and self._check(self.cache_file)
> > +
> > + def fetch(self):
> > + if not self.cache_dir.exists():
> > + self.cache_dir.mkdir(parents=True, exist_ok=True)
>
> This doesn't work with QEMU_TEST_CACHE_DIR set to someone else:
>
> File
> "/home/philippe.mathieu-daude/qemu/tests/functional/qemu_test/asset.py",
> line 60, in fetch
> self.cache_dir.mkdir(parents=True, exist_ok=True)
> File "/usr/lib/python3.10/pathlib.py", line 1175, in mkdir
> self._accessor.mkdir(self, mode)
> PermissionError: [Errno 13] Permission denied:
> '/home/alex.bennee/.cache/qemu/download'
> ninja: build stopped: subcommand failed.
>
> Maybe use a getter which falls back to Path("~").expanduser() when
> no access on QEMU_TEST_CACHE_DIR? This happens when downloading a
> new file (the recent MIPS tests I converted) which isn't in Alex's cache:
Is it really valid to point QEMU_TEST_CACHE_DIR to a directory
that you don't have permission to access ? This feels like it
could be classed as invalid usage to me, rather than needing a
workaround.
>
> 2024-08-21 15:45:48,896 - qemu-test - INFO - Attempting to cache 'Asset: url=https://s3-eu-west-1.amazonaws.com/downloads-mips/mips-downloads/YAMON/yamon-bin-02.22.zip
> hash=eef86f0eed0ef554f041dcd47b87eebea0e6f9f1184ed31f7e9e8b4a803860ab cache=/home/alex.bennee/.cache/download/eef86f0eed0ef554f041dcd47b87eebea0e6f9f1184ed31f7e9e8b4a803860ab'
>
> > + if self.valid():
> > + self.log.debug("Using cached asset %s for %s",
> > + self.cache_file, self.url)
> > + return str(self.cache_file)
> > +
> > + self.log.info("Downloading %s to %s...", self.url, self.cache_file)
> > + tmp_cache_file = self.cache_file.with_suffix(".download")
> > +
> > + try:
> > + resp = urllib.request.urlopen(self.url)
> > + except Exception as e:
> > + self.log.error("Unable to download %s: %s", self.url, e)
> > + raise
> > +
> > + try:
> > + with tmp_cache_file.open("wb+") as dst:
> > + copyfileobj(resp, dst)
> > + except:
> > + tmp_cache_file.unlink()
> > + raise
> > + try:
> > + # Set these just for informational purposes
> > + os.setxattr(str(tmp_cache_file), "user.qemu-asset-url",
> > + self.url.encode('utf8'))
> > + os.setxattr(str(tmp_cache_file), "user.qemu-asset-hash",
> > + self.hash.encode('utf8'))
> > + except Exception as e:
> > + self.log.info("Unable to set xattr on %s: %s", tmp_cache_file, e)
> > + pass
> > +
> > + if not self._check(tmp_cache_file):
> > + tmp_cache_file.unlink()
> > + raise Exception("Hash of %s does not match %s" %
> > + (self.url, self.hash))
> > + tmp_cache_file.replace(self.cache_file)
> > +
> > + self.log.info("Cached %s at %s" % (self.url, self.cache_file))
> > + return str(self.cache_file)
>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v4 14/35] tests/functional: add a module for handling asset download & caching
2024-08-21 8:27 ` [PATCH v4 14/35] tests/functional: add a module for handling asset download & caching Thomas Huth
2024-08-21 14:49 ` Philippe Mathieu-Daudé
@ 2024-08-23 6:24 ` Philippe Mathieu-Daudé
2024-08-29 10:00 ` Daniel P. Berrangé
1 sibling, 1 reply; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-23 6:24 UTC (permalink / raw)
To: Thomas Huth, Alex Bennée, qemu-devel, Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
Hi,
On 21/8/24 10:27, Thomas Huth wrote:
> From: Daniel P. Berrangé <berrange@redhat.com>
>
> The 'Asset' class is a simple module that declares a downloadable
> asset that can be cached locally. Downloads are stored in the user's
> home dir at ~/.cache/qemu/download, using a sha256 sum of the URL.
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> [thuth: Drop sha1 support, use hash on file content for naming instead of URL,
> add the possibility to specify the cache dir via environment variable]
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/functional/qemu_test/__init__.py | 1 +
> tests/functional/qemu_test/asset.py | 97 ++++++++++++++++++++++++++
> 2 files changed, 98 insertions(+)
> create mode 100644 tests/functional/qemu_test/asset.py
> + def fetch(self):
> + if not self.cache_dir.exists():
> + self.cache_dir.mkdir(parents=True, exist_ok=True)
> +
> + if self.valid():
> + self.log.debug("Using cached asset %s for %s",
> + self.cache_file, self.url)
> + return str(self.cache_file)
> +
> + self.log.info("Downloading %s to %s...", self.url, self.cache_file)
> + tmp_cache_file = self.cache_file.with_suffix(".download")
> +
> + try:
> + resp = urllib.request.urlopen(self.url)
> + except Exception as e:
> + self.log.error("Unable to download %s: %s", self.url, e)
> + raise
> +
> + try:
> + with tmp_cache_file.open("wb+") as dst:
> + copyfileobj(resp, dst)
> + except:
> + tmp_cache_file.unlink()
> + raise
> + try:
> + # Set these just for informational purposes
> + os.setxattr(str(tmp_cache_file), "user.qemu-asset-url",
> + self.url.encode('utf8'))
> + os.setxattr(str(tmp_cache_file), "user.qemu-asset-hash",
> + self.hash.encode('utf8'))
> + except Exception as e:
> + self.log.info("Unable to set xattr on %s: %s", tmp_cache_file, e)
This line is annoying on macOS as it is logged for each file downloaded.
Is it really useful? Can we demote to DEBUG level or log it just once,
given all tmp_cache_files will always be on the same cache_dir thus
filesystem?
> + pass
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH v4 14/35] tests/functional: add a module for handling asset download & caching
2024-08-23 6:24 ` Philippe Mathieu-Daudé
@ 2024-08-29 10:00 ` Daniel P. Berrangé
0 siblings, 0 replies; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-08-29 10:00 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: Thomas Huth, Alex Bennée, qemu-devel, Ani Sinha,
Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On Fri, Aug 23, 2024 at 08:24:45AM +0200, Philippe Mathieu-Daudé wrote:
> Hi,
>
> On 21/8/24 10:27, Thomas Huth wrote:
> > From: Daniel P. Berrangé <berrange@redhat.com>
> >
> > The 'Asset' class is a simple module that declares a downloadable
> > asset that can be cached locally. Downloads are stored in the user's
> > home dir at ~/.cache/qemu/download, using a sha256 sum of the URL.
> >
> > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> > Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> > Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> > [thuth: Drop sha1 support, use hash on file content for naming instead of URL,
> > add the possibility to specify the cache dir via environment variable]
> > Signed-off-by: Thomas Huth <thuth@redhat.com>
> > ---
> > tests/functional/qemu_test/__init__.py | 1 +
> > tests/functional/qemu_test/asset.py | 97 ++++++++++++++++++++++++++
> > 2 files changed, 98 insertions(+)
> > create mode 100644 tests/functional/qemu_test/asset.py
>
>
> > + def fetch(self):
> > + if not self.cache_dir.exists():
> > + self.cache_dir.mkdir(parents=True, exist_ok=True)
> > +
> > + if self.valid():
> > + self.log.debug("Using cached asset %s for %s",
> > + self.cache_file, self.url)
> > + return str(self.cache_file)
> > +
> > + self.log.info("Downloading %s to %s...", self.url, self.cache_file)
> > + tmp_cache_file = self.cache_file.with_suffix(".download")
> > +
> > + try:
> > + resp = urllib.request.urlopen(self.url)
> > + except Exception as e:
> > + self.log.error("Unable to download %s: %s", self.url, e)
> > + raise
> > +
> > + try:
> > + with tmp_cache_file.open("wb+") as dst:
> > + copyfileobj(resp, dst)
> > + except:
> > + tmp_cache_file.unlink()
> > + raise
> > + try:
> > + # Set these just for informational purposes
> > + os.setxattr(str(tmp_cache_file), "user.qemu-asset-url",
> > + self.url.encode('utf8'))
> > + os.setxattr(str(tmp_cache_file), "user.qemu-asset-hash",
> > + self.hash.encode('utf8'))
> > + except Exception as e:
> > + self.log.info("Unable to set xattr on %s: %s", tmp_cache_file, e)
>
> This line is annoying on macOS as it is logged for each file downloaded.
> Is it really useful? Can we demote to DEBUG level or log it just once,
> given all tmp_cache_files will always be on the same cache_dir thus
> filesystem?
Yeah, DEBUG would be fine.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 69+ messages in thread
* [PATCH v4 15/35] tests/functional: enable pre-emptive caching of assets
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (13 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 14/35] tests/functional: add a module for handling asset download & caching Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-23 7:28 ` Philippe Mathieu-Daudé
2024-08-21 8:27 ` [PATCH v4 16/35] tests/functional: Convert some tests that download files via fetch_asset() Thomas Huth
` (19 subsequent siblings)
34 siblings, 1 reply; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
From: Daniel P. Berrangé <berrange@redhat.com>
Many tests need to access assets stored on remote sites. We don't want
to download these during test execution when run by meson, since this
risks hitting test timeouts when data transfers are slow.
Add support for pre-emptive caching of assets by setting the env var
QEMU_TEST_PRECACHE to point to a timestamp file. When this is set,
instead of running the test, the assets will be downloaded and saved
to the cache, then the timestamp file created.
A meson custom target is created as a dependency of each test suite
to trigger the pre-emptive caching logic before the test runs.
When run in caching mode, it will locate assets by looking for class
level variables with a name prefix "ASSET_", and type "Asset".
At the ninja level
ninja test --suite functional
will speculatively download any assets that are not already cached,
so it is advisable to set a timeout multiplier.
QEMU_TEST_NO_DOWNLOAD=1 ninja test --suite functional
will fail the test if a required asset is not already cached
ninja precache-functional
will download and cache all assets required by the functional
tests
At the make level, precaching is always done by
make check-functional
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Tested-by: Richard Henderson <richard.henderson@linaro.org>
[thuth: Remove the duplicated "path = os.path.basename(...)" line]
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/Makefile.include | 3 ++-
tests/functional/meson.build | 33 +++++++++++++++++++++++--
tests/functional/qemu_test/asset.py | 34 ++++++++++++++++++++++++++
tests/functional/qemu_test/testcase.py | 7 ++++++
4 files changed, 74 insertions(+), 3 deletions(-)
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 66c8cc3123..010369bd3a 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -161,7 +161,8 @@ $(FUNCTIONAL_TARGETS):
.PHONY: check-functional
check-functional:
- @$(MAKE) SPEED=thorough check-func check-func-quick
+ @$(NINJA) precache-functional
+ @QEMU_TEST_NO_DOWNLOAD=1 $(MAKE) SPEED=thorough check-func check-func-quick
# Consolidated targets
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 8a8fa0ab99..cef74b82a9 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -33,6 +33,7 @@ tests_x86_64_quick = [
tests_x86_64_thorough = [
]
+precache_all = []
foreach speed : ['quick', 'thorough']
foreach dir : target_dirs
if not dir.endswith('-softmmu')
@@ -63,11 +64,35 @@ foreach speed : ['quick', 'thorough']
meson.current_source_dir())
foreach test : target_tests
- test('func-@0@/@1@'.format(target_base, test),
+ testname = '@0@-@1@'.format(target_base, test)
+ testfile = 'test_' + test + '.py'
+ testpath = meson.current_source_dir() / testfile
+ teststamp = testname + '.tstamp'
+ test_precache_env = environment()
+ test_precache_env.set('QEMU_TEST_PRECACHE', meson.current_build_dir() / teststamp)
+ test_precache_env.set('PYTHONPATH', meson.project_source_root() / 'python:' +
+ meson.current_source_dir())
+ precache = custom_target('func-precache-' + testname,
+ output: teststamp,
+ command: [python, testpath],
+ depend_files: files(testpath),
+ build_by_default: false,
+ env: test_precache_env)
+ precache_all += precache
+
+ # Ideally we would add 'precache' to 'depends' here, such that
+ # 'build_by_default: false' lets the pre-caching automatically
+ # run immediately before the test runs. In practice this is
+ # broken in meson, with it running the pre-caching in the normal
+ # compile phase https://github.com/mesonbuild/meson/issues/2518
+ # If the above bug ever gets fixed, when QEMU changes the min
+ # meson version, add the 'depends' and remove the custom
+ # 'run_target' logic below & in Makefile.include
+ test('func-' + testname,
python,
depends: [test_deps, test_emulator, emulator_modules],
env: test_env,
- args: [meson.current_source_dir() / 'test_' + test + '.py'],
+ args: [testpath],
protocol: 'tap',
timeout: test_timeouts.get(test, 60),
priority: test_timeouts.get(test, 60),
@@ -75,3 +100,7 @@ foreach speed : ['quick', 'thorough']
endforeach
endforeach
endforeach
+
+run_target('precache-functional',
+ depends: precache_all,
+ command: ['true'])
diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_test/asset.py
index cbeb6278af..9250ff2b06 100644
--- a/tests/functional/qemu_test/asset.py
+++ b/tests/functional/qemu_test/asset.py
@@ -9,6 +9,8 @@
import logging
import os
import subprocess
+import sys
+import unittest
import urllib.request
from pathlib import Path
from shutil import copyfileobj
@@ -62,6 +64,9 @@ def fetch(self):
self.cache_file, self.url)
return str(self.cache_file)
+ if os.environ.get("QEMU_TEST_NO_DOWNLOAD", False):
+ raise Exception("Asset cache is invalid and downloads disabled")
+
self.log.info("Downloading %s to %s...", self.url, self.cache_file)
tmp_cache_file = self.cache_file.with_suffix(".download")
@@ -95,3 +100,32 @@ def fetch(self):
self.log.info("Cached %s at %s" % (self.url, self.cache_file))
return str(self.cache_file)
+
+ def precache_test(test):
+ log = logging.getLogger('qemu-test')
+ log.setLevel(logging.DEBUG)
+ handler = logging.StreamHandler(sys.stdout)
+ handler.setLevel(logging.DEBUG)
+ formatter = logging.Formatter(
+ '%(asctime)s - %(name)s - %(levelname)s - %(message)s')
+ handler.setFormatter(formatter)
+ log.addHandler(handler)
+ for name, asset in vars(test.__class__).items():
+ if name.startswith("ASSET_") and type(asset) == Asset:
+ log.info("Attempting to cache '%s'" % asset)
+ asset.fetch()
+ log.removeHandler(handler)
+
+ def precache_suite(suite):
+ for test in suite:
+ if isinstance(test, unittest.TestSuite):
+ Asset.precache_suite(test)
+ elif isinstance(test, unittest.TestCase):
+ Asset.precache_test(test)
+
+ def precache_suites(path, cacheTstamp):
+ loader = unittest.loader.defaultTestLoader
+ tests = loader.loadTestsFromNames([path], None)
+
+ with open(cacheTstamp, "w") as fh:
+ Asset.precache_suite(tests)
diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu_test/testcase.py
index b2dd863c6e..18314be9d1 100644
--- a/tests/functional/qemu_test/testcase.py
+++ b/tests/functional/qemu_test/testcase.py
@@ -21,6 +21,7 @@
from qemu.machine import QEMUMachine
from qemu.utils import kvm_available, tcg_available
+from .asset import Asset
from .cmd import run_cmd
from .config import BUILD_DIR
@@ -58,6 +59,12 @@ def tearDown(self):
def main():
path = os.path.basename(sys.argv[0])[:-3]
+
+ cache = os.environ.get("QEMU_TEST_PRECACHE", None)
+ if cache is not None:
+ Asset.precache_suites(path, cache)
+ return
+
tr = pycotap.TAPTestRunner(message_log = pycotap.LogMode.LogToError,
test_output_log = pycotap.LogMode.LogToError)
unittest.main(module = None, testRunner = tr, argv=["__dummy__", path])
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* Re: [PATCH v4 15/35] tests/functional: enable pre-emptive caching of assets
2024-08-21 8:27 ` [PATCH v4 15/35] tests/functional: enable pre-emptive caching of assets Thomas Huth
@ 2024-08-23 7:28 ` Philippe Mathieu-Daudé
2024-08-27 13:16 ` Thomas Huth
0 siblings, 1 reply; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-23 7:28 UTC (permalink / raw)
To: Thomas Huth, Alex Bennée, qemu-devel, Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
Hi,
On 21/8/24 10:27, Thomas Huth wrote:
> From: Daniel P. Berrangé <berrange@redhat.com>
>
> Many tests need to access assets stored on remote sites. We don't want
> to download these during test execution when run by meson, since this
> risks hitting test timeouts when data transfers are slow.
>
> Add support for pre-emptive caching of assets by setting the env var
> QEMU_TEST_PRECACHE to point to a timestamp file. When this is set,
> instead of running the test, the assets will be downloaded and saved
> to the cache, then the timestamp file created.
>
> A meson custom target is created as a dependency of each test suite
> to trigger the pre-emptive caching logic before the test runs.
>
> When run in caching mode, it will locate assets by looking for class
> level variables with a name prefix "ASSET_", and type "Asset".
>
> At the ninja level
>
> ninja test --suite functional
>
> will speculatively download any assets that are not already cached,
> so it is advisable to set a timeout multiplier.
>
> QEMU_TEST_NO_DOWNLOAD=1 ninja test --suite functional
>
> will fail the test if a required asset is not already cached
>
> ninja precache-functional
>
> will download and cache all assets required by the functional
> tests
>
> At the make level, precaching is always done by
>
> make check-functional
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> Tested-by: Richard Henderson <richard.henderson@linaro.org>
> [thuth: Remove the duplicated "path = os.path.basename(...)" line]
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/Makefile.include | 3 ++-
> tests/functional/meson.build | 33 +++++++++++++++++++++++--
> tests/functional/qemu_test/asset.py | 34 ++++++++++++++++++++++++++
> tests/functional/qemu_test/testcase.py | 7 ++++++
> 4 files changed, 74 insertions(+), 3 deletions(-)
>
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 66c8cc3123..010369bd3a 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -161,7 +161,8 @@ $(FUNCTIONAL_TARGETS):
>
> .PHONY: check-functional
> check-functional:
> - @$(MAKE) SPEED=thorough check-func check-func-quick
> + @$(NINJA) precache-functional
> + @QEMU_TEST_NO_DOWNLOAD=1 $(MAKE) SPEED=thorough check-func check-func-quick
>
> # Consolidated targets
>
> diff --git a/tests/functional/meson.build b/tests/functional/meson.build
> index 8a8fa0ab99..cef74b82a9 100644
> --- a/tests/functional/meson.build
> +++ b/tests/functional/meson.build
> @@ -33,6 +33,7 @@ tests_x86_64_quick = [
> tests_x86_64_thorough = [
> ]
>
> +precache_all = []
> foreach speed : ['quick', 'thorough']
> foreach dir : target_dirs
> if not dir.endswith('-softmmu')
> @@ -63,11 +64,35 @@ foreach speed : ['quick', 'thorough']
> meson.current_source_dir())
>
> foreach test : target_tests
> - test('func-@0@/@1@'.format(target_base, test),
> + testname = '@0@-@1@'.format(target_base, test)
> + testfile = 'test_' + test + '.py'
> + testpath = meson.current_source_dir() / testfile
> + teststamp = testname + '.tstamp'
> + test_precache_env = environment()
> + test_precache_env.set('QEMU_TEST_PRECACHE', meson.current_build_dir() / teststamp)
> + test_precache_env.set('PYTHONPATH', meson.project_source_root() / 'python:' +
> + meson.current_source_dir())
> + precache = custom_target('func-precache-' + testname,
> + output: teststamp,
> + command: [python, testpath],
> + depend_files: files(testpath),
> + build_by_default: false,
> + env: test_precache_env)
> + precache_all += precache
> +
> + # Ideally we would add 'precache' to 'depends' here, such that
> + # 'build_by_default: false' lets the pre-caching automatically
> + # run immediately before the test runs. In practice this is
> + # broken in meson, with it running the pre-caching in the normal
> + # compile phase https://github.com/mesonbuild/meson/issues/2518
> + # If the above bug ever gets fixed, when QEMU changes the min
> + # meson version, add the 'depends' and remove the custom
> + # 'run_target' logic below & in Makefile.include
> + test('func-' + testname,
> python,
> depends: [test_deps, test_emulator, emulator_modules],
> env: test_env,
> - args: [meson.current_source_dir() / 'test_' + test + '.py'],
> + args: [testpath],
> protocol: 'tap',
> timeout: test_timeouts.get(test, 60),
> priority: test_timeouts.get(test, 60),
> @@ -75,3 +100,7 @@ foreach speed : ['quick', 'thorough']
> endforeach
> endforeach
> endforeach
> +
> +run_target('precache-functional',
> + depends: precache_all,
> + command: ['true'])
> diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_test/asset.py
> index cbeb6278af..9250ff2b06 100644
> --- a/tests/functional/qemu_test/asset.py
> +++ b/tests/functional/qemu_test/asset.py
> @@ -9,6 +9,8 @@
> import logging
> import os
> import subprocess
> +import sys
> +import unittest
> import urllib.request
> from pathlib import Path
> from shutil import copyfileobj
> @@ -62,6 +64,9 @@ def fetch(self):
> self.cache_file, self.url)
> return str(self.cache_file)
>
> + if os.environ.get("QEMU_TEST_NO_DOWNLOAD", False):
> + raise Exception("Asset cache is invalid and downloads disabled")
> +
> self.log.info("Downloading %s to %s...", self.url, self.cache_file)
> tmp_cache_file = self.cache_file.with_suffix(".download")
>
> @@ -95,3 +100,32 @@ def fetch(self):
>
> self.log.info("Cached %s at %s" % (self.url, self.cache_file))
> return str(self.cache_file)
> +
> + def precache_test(test):
> + log = logging.getLogger('qemu-test')
> + log.setLevel(logging.DEBUG)
> + handler = logging.StreamHandler(sys.stdout)
> + handler.setLevel(logging.DEBUG)
> + formatter = logging.Formatter(
> + '%(asctime)s - %(name)s - %(levelname)s - %(message)s')
> + handler.setFormatter(formatter)
> + log.addHandler(handler)
> + for name, asset in vars(test.__class__).items():
> + if name.startswith("ASSET_") and type(asset) == Asset:
> + log.info("Attempting to cache '%s'" % asset)
> + asset.fetch()
> + log.removeHandler(handler)
> +
> + def precache_suite(suite):
> + for test in suite:
> + if isinstance(test, unittest.TestSuite):
> + Asset.precache_suite(test)
> + elif isinstance(test, unittest.TestCase):
> + Asset.precache_test(test)
> +
> + def precache_suites(path, cacheTstamp):
> + loader = unittest.loader.defaultTestLoader
> + tests = loader.loadTestsFromNames([path], None)
> +
> + with open(cacheTstamp, "w") as fh:
> + Asset.precache_suite(tests)
When using multiple jobs (-jN) I'm observing some hangs,
apparently multiple threads trying to download the same file.
The files are eventually downloaded successfully but it takes
longer. Should we acquire some exclusive lock somewhere?
> diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu_test/testcase.py
> index b2dd863c6e..18314be9d1 100644
> --- a/tests/functional/qemu_test/testcase.py
> +++ b/tests/functional/qemu_test/testcase.py
> @@ -21,6 +21,7 @@
> from qemu.machine import QEMUMachine
> from qemu.utils import kvm_available, tcg_available
>
> +from .asset import Asset
> from .cmd import run_cmd
> from .config import BUILD_DIR
>
> @@ -58,6 +59,12 @@ def tearDown(self):
>
> def main():
> path = os.path.basename(sys.argv[0])[:-3]
> +
> + cache = os.environ.get("QEMU_TEST_PRECACHE", None)
> + if cache is not None:
> + Asset.precache_suites(path, cache)
> + return
> +
> tr = pycotap.TAPTestRunner(message_log = pycotap.LogMode.LogToError,
> test_output_log = pycotap.LogMode.LogToError)
> unittest.main(module = None, testRunner = tr, argv=["__dummy__", path])
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH v4 15/35] tests/functional: enable pre-emptive caching of assets
2024-08-23 7:28 ` Philippe Mathieu-Daudé
@ 2024-08-27 13:16 ` Thomas Huth
2024-08-27 14:24 ` Thomas Huth
0 siblings, 1 reply; 69+ messages in thread
From: Thomas Huth @ 2024-08-27 13:16 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, Alex Bennée, qemu-devel,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On 23/08/2024 09.28, Philippe Mathieu-Daudé wrote:
> Hi,
>
> On 21/8/24 10:27, Thomas Huth wrote:
>> From: Daniel P. Berrangé <berrange@redhat.com>
>>
>> Many tests need to access assets stored on remote sites. We don't want
>> to download these during test execution when run by meson, since this
>> risks hitting test timeouts when data transfers are slow.
>>
>> Add support for pre-emptive caching of assets by setting the env var
>> QEMU_TEST_PRECACHE to point to a timestamp file. When this is set,
>> instead of running the test, the assets will be downloaded and saved
>> to the cache, then the timestamp file created.
>>
>> A meson custom target is created as a dependency of each test suite
>> to trigger the pre-emptive caching logic before the test runs.
>>
>> When run in caching mode, it will locate assets by looking for class
>> level variables with a name prefix "ASSET_", and type "Asset".
>>
>> At the ninja level
>>
>> ninja test --suite functional
>>
>> will speculatively download any assets that are not already cached,
>> so it is advisable to set a timeout multiplier.
>>
>> QEMU_TEST_NO_DOWNLOAD=1 ninja test --suite functional
>>
>> will fail the test if a required asset is not already cached
>>
>> ninja precache-functional
>>
>> will download and cache all assets required by the functional
>> tests
>>
>> At the make level, precaching is always done by
>>
>> make check-functional
>>
>> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
>> Tested-by: Richard Henderson <richard.henderson@linaro.org>
>> [thuth: Remove the duplicated "path = os.path.basename(...)" line]
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>> tests/Makefile.include | 3 ++-
>> tests/functional/meson.build | 33 +++++++++++++++++++++++--
>> tests/functional/qemu_test/asset.py | 34 ++++++++++++++++++++++++++
>> tests/functional/qemu_test/testcase.py | 7 ++++++
>> 4 files changed, 74 insertions(+), 3 deletions(-)
>>
>> diff --git a/tests/Makefile.include b/tests/Makefile.include
>> index 66c8cc3123..010369bd3a 100644
>> --- a/tests/Makefile.include
>> +++ b/tests/Makefile.include
>> @@ -161,7 +161,8 @@ $(FUNCTIONAL_TARGETS):
>> .PHONY: check-functional
>> check-functional:
>> - @$(MAKE) SPEED=thorough check-func check-func-quick
>> + @$(NINJA) precache-functional
>> + @QEMU_TEST_NO_DOWNLOAD=1 $(MAKE) SPEED=thorough check-func
>> check-func-quick
>> # Consolidated targets
>> diff --git a/tests/functional/meson.build b/tests/functional/meson.build
>> index 8a8fa0ab99..cef74b82a9 100644
>> --- a/tests/functional/meson.build
>> +++ b/tests/functional/meson.build
>> @@ -33,6 +33,7 @@ tests_x86_64_quick = [
>> tests_x86_64_thorough = [
>> ]
>> +precache_all = []
>> foreach speed : ['quick', 'thorough']
>> foreach dir : target_dirs
>> if not dir.endswith('-softmmu')
>> @@ -63,11 +64,35 @@ foreach speed : ['quick', 'thorough']
>> meson.current_source_dir())
>> foreach test : target_tests
>> - test('func-@0@/@1@'.format(target_base, test),
>> + testname = '@0@-@1@'.format(target_base, test)
>> + testfile = 'test_' + test + '.py'
>> + testpath = meson.current_source_dir() / testfile
>> + teststamp = testname + '.tstamp'
>> + test_precache_env = environment()
>> + test_precache_env.set('QEMU_TEST_PRECACHE',
>> meson.current_build_dir() / teststamp)
>> + test_precache_env.set('PYTHONPATH', meson.project_source_root() /
>> 'python:' +
>> + meson.current_source_dir())
>> + precache = custom_target('func-precache-' + testname,
>> + output: teststamp,
>> + command: [python, testpath],
>> + depend_files: files(testpath),
>> + build_by_default: false,
>> + env: test_precache_env)
>> + precache_all += precache
>> +
>> + # Ideally we would add 'precache' to 'depends' here, such that
>> + # 'build_by_default: false' lets the pre-caching automatically
>> + # run immediately before the test runs. In practice this is
>> + # broken in meson, with it running the pre-caching in the normal
>> + # compile phase https://github.com/mesonbuild/meson/issues/2518
>> + # If the above bug ever gets fixed, when QEMU changes the min
>> + # meson version, add the 'depends' and remove the custom
>> + # 'run_target' logic below & in Makefile.include
>> + test('func-' + testname,
>> python,
>> depends: [test_deps, test_emulator, emulator_modules],
>> env: test_env,
>> - args: [meson.current_source_dir() / 'test_' + test + '.py'],
>> + args: [testpath],
>> protocol: 'tap',
>> timeout: test_timeouts.get(test, 60),
>> priority: test_timeouts.get(test, 60),
>> @@ -75,3 +100,7 @@ foreach speed : ['quick', 'thorough']
>> endforeach
>> endforeach
>> endforeach
>> +
>> +run_target('precache-functional',
>> + depends: precache_all,
>> + command: ['true'])
>> diff --git a/tests/functional/qemu_test/asset.py
>> b/tests/functional/qemu_test/asset.py
>> index cbeb6278af..9250ff2b06 100644
>> --- a/tests/functional/qemu_test/asset.py
>> +++ b/tests/functional/qemu_test/asset.py
>> @@ -9,6 +9,8 @@
>> import logging
>> import os
>> import subprocess
>> +import sys
>> +import unittest
>> import urllib.request
>> from pathlib import Path
>> from shutil import copyfileobj
>> @@ -62,6 +64,9 @@ def fetch(self):
>> self.cache_file, self.url)
>> return str(self.cache_file)
>> + if os.environ.get("QEMU_TEST_NO_DOWNLOAD", False):
>> + raise Exception("Asset cache is invalid and downloads disabled")
>> +
>> self.log.info("Downloading %s to %s...", self.url, self.cache_file)
>> tmp_cache_file = self.cache_file.with_suffix(".download")
>> @@ -95,3 +100,32 @@ def fetch(self):
>> self.log.info("Cached %s at %s" % (self.url, self.cache_file))
>> return str(self.cache_file)
>> +
>> + def precache_test(test):
>> + log = logging.getLogger('qemu-test')
>> + log.setLevel(logging.DEBUG)
>> + handler = logging.StreamHandler(sys.stdout)
>> + handler.setLevel(logging.DEBUG)
>> + formatter = logging.Formatter(
>> + '%(asctime)s - %(name)s - %(levelname)s - %(message)s')
>> + handler.setFormatter(formatter)
>> + log.addHandler(handler)
>> + for name, asset in vars(test.__class__).items():
>> + if name.startswith("ASSET_") and type(asset) == Asset:
>> + log.info("Attempting to cache '%s'" % asset)
>> + asset.fetch()
>> + log.removeHandler(handler)
>> +
>> + def precache_suite(suite):
>> + for test in suite:
>> + if isinstance(test, unittest.TestSuite):
>> + Asset.precache_suite(test)
>> + elif isinstance(test, unittest.TestCase):
>> + Asset.precache_test(test)
>> +
>> + def precache_suites(path, cacheTstamp):
>> + loader = unittest.loader.defaultTestLoader
>> + tests = loader.loadTestsFromNames([path], None)
>> +
>> + with open(cacheTstamp, "w") as fh:
>> + Asset.precache_suite(tests)
>
>
> When using multiple jobs (-jN) I'm observing some hangs,
> apparently multiple threads trying to download the same file.
> The files are eventually downloaded successfully but it takes
> longer. Should we acquire some exclusive lock somewhere?
I haven't seen that yet ... what did you exactly run? "make check-functional
-jN" ? Or "make check-functional-<target> -jN" ?
Thomas
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH v4 15/35] tests/functional: enable pre-emptive caching of assets
2024-08-27 13:16 ` Thomas Huth
@ 2024-08-27 14:24 ` Thomas Huth
2024-08-29 10:15 ` Daniel P. Berrangé
0 siblings, 1 reply; 69+ messages in thread
From: Thomas Huth @ 2024-08-27 14:24 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, Alex Bennée, qemu-devel,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On 27/08/2024 15.16, Thomas Huth wrote:
> On 23/08/2024 09.28, Philippe Mathieu-Daudé wrote:
>> Hi,
>>
>> On 21/8/24 10:27, Thomas Huth wrote:
>>> From: Daniel P. Berrangé <berrange@redhat.com>
>>>
>>> Many tests need to access assets stored on remote sites. We don't want
>>> to download these during test execution when run by meson, since this
>>> risks hitting test timeouts when data transfers are slow.
>>>
>>> Add support for pre-emptive caching of assets by setting the env var
>>> QEMU_TEST_PRECACHE to point to a timestamp file. When this is set,
>>> instead of running the test, the assets will be downloaded and saved
>>> to the cache, then the timestamp file created.
>>>
>>> A meson custom target is created as a dependency of each test suite
>>> to trigger the pre-emptive caching logic before the test runs.
>>>
>>> When run in caching mode, it will locate assets by looking for class
>>> level variables with a name prefix "ASSET_", and type "Asset".
>>>
>>> At the ninja level
>>>
>>> ninja test --suite functional
>>>
>>> will speculatively download any assets that are not already cached,
>>> so it is advisable to set a timeout multiplier.
>>>
>>> QEMU_TEST_NO_DOWNLOAD=1 ninja test --suite functional
>>>
>>> will fail the test if a required asset is not already cached
>>>
>>> ninja precache-functional
>>>
>>> will download and cache all assets required by the functional
>>> tests
>>>
>>> At the make level, precaching is always done by
>>>
>>> make check-functional
>>>
>>> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
>>> Tested-by: Richard Henderson <richard.henderson@linaro.org>
>>> [thuth: Remove the duplicated "path = os.path.basename(...)" line]
>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>> ---
>>> tests/Makefile.include | 3 ++-
>>> tests/functional/meson.build | 33 +++++++++++++++++++++++--
>>> tests/functional/qemu_test/asset.py | 34 ++++++++++++++++++++++++++
>>> tests/functional/qemu_test/testcase.py | 7 ++++++
>>> 4 files changed, 74 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/tests/Makefile.include b/tests/Makefile.include
>>> index 66c8cc3123..010369bd3a 100644
>>> --- a/tests/Makefile.include
>>> +++ b/tests/Makefile.include
>>> @@ -161,7 +161,8 @@ $(FUNCTIONAL_TARGETS):
>>> .PHONY: check-functional
>>> check-functional:
>>> - @$(MAKE) SPEED=thorough check-func check-func-quick
>>> + @$(NINJA) precache-functional
>>> + @QEMU_TEST_NO_DOWNLOAD=1 $(MAKE) SPEED=thorough check-func
>>> check-func-quick
>>> # Consolidated targets
>>> diff --git a/tests/functional/meson.build b/tests/functional/meson.build
>>> index 8a8fa0ab99..cef74b82a9 100644
>>> --- a/tests/functional/meson.build
>>> +++ b/tests/functional/meson.build
>>> @@ -33,6 +33,7 @@ tests_x86_64_quick = [
>>> tests_x86_64_thorough = [
>>> ]
>>> +precache_all = []
>>> foreach speed : ['quick', 'thorough']
>>> foreach dir : target_dirs
>>> if not dir.endswith('-softmmu')
>>> @@ -63,11 +64,35 @@ foreach speed : ['quick', 'thorough']
>>> meson.current_source_dir())
>>> foreach test : target_tests
>>> - test('func-@0@/@1@'.format(target_base, test),
>>> + testname = '@0@-@1@'.format(target_base, test)
>>> + testfile = 'test_' + test + '.py'
>>> + testpath = meson.current_source_dir() / testfile
>>> + teststamp = testname + '.tstamp'
>>> + test_precache_env = environment()
>>> + test_precache_env.set('QEMU_TEST_PRECACHE',
>>> meson.current_build_dir() / teststamp)
>>> + test_precache_env.set('PYTHONPATH', meson.project_source_root() /
>>> 'python:' +
>>> + meson.current_source_dir())
>>> + precache = custom_target('func-precache-' + testname,
>>> + output: teststamp,
>>> + command: [python, testpath],
>>> + depend_files: files(testpath),
>>> + build_by_default: false,
>>> + env: test_precache_env)
>>> + precache_all += precache
>>> +
>>> + # Ideally we would add 'precache' to 'depends' here, such that
>>> + # 'build_by_default: false' lets the pre-caching automatically
>>> + # run immediately before the test runs. In practice this is
>>> + # broken in meson, with it running the pre-caching in the normal
>>> + # compile phase https://github.com/mesonbuild/meson/issues/2518
>>> + # If the above bug ever gets fixed, when QEMU changes the min
>>> + # meson version, add the 'depends' and remove the custom
>>> + # 'run_target' logic below & in Makefile.include
>>> + test('func-' + testname,
>>> python,
>>> depends: [test_deps, test_emulator, emulator_modules],
>>> env: test_env,
>>> - args: [meson.current_source_dir() / 'test_' + test + '.py'],
>>> + args: [testpath],
>>> protocol: 'tap',
>>> timeout: test_timeouts.get(test, 60),
>>> priority: test_timeouts.get(test, 60),
>>> @@ -75,3 +100,7 @@ foreach speed : ['quick', 'thorough']
>>> endforeach
>>> endforeach
>>> endforeach
>>> +
>>> +run_target('precache-functional',
>>> + depends: precache_all,
>>> + command: ['true'])
>>> diff --git a/tests/functional/qemu_test/asset.py
>>> b/tests/functional/qemu_test/asset.py
>>> index cbeb6278af..9250ff2b06 100644
>>> --- a/tests/functional/qemu_test/asset.py
>>> +++ b/tests/functional/qemu_test/asset.py
>>> @@ -9,6 +9,8 @@
>>> import logging
>>> import os
>>> import subprocess
>>> +import sys
>>> +import unittest
>>> import urllib.request
>>> from pathlib import Path
>>> from shutil import copyfileobj
>>> @@ -62,6 +64,9 @@ def fetch(self):
>>> self.cache_file, self.url)
>>> return str(self.cache_file)
>>> + if os.environ.get("QEMU_TEST_NO_DOWNLOAD", False):
>>> + raise Exception("Asset cache is invalid and downloads
>>> disabled")
>>> +
>>> self.log.info("Downloading %s to %s...", self.url,
>>> self.cache_file)
>>> tmp_cache_file = self.cache_file.with_suffix(".download")
>>> @@ -95,3 +100,32 @@ def fetch(self):
>>> self.log.info("Cached %s at %s" % (self.url, self.cache_file))
>>> return str(self.cache_file)
>>> +
>>> + def precache_test(test):
>>> + log = logging.getLogger('qemu-test')
>>> + log.setLevel(logging.DEBUG)
>>> + handler = logging.StreamHandler(sys.stdout)
>>> + handler.setLevel(logging.DEBUG)
>>> + formatter = logging.Formatter(
>>> + '%(asctime)s - %(name)s - %(levelname)s - %(message)s')
>>> + handler.setFormatter(formatter)
>>> + log.addHandler(handler)
>>> + for name, asset in vars(test.__class__).items():
>>> + if name.startswith("ASSET_") and type(asset) == Asset:
>>> + log.info("Attempting to cache '%s'" % asset)
>>> + asset.fetch()
>>> + log.removeHandler(handler)
>>> +
>>> + def precache_suite(suite):
>>> + for test in suite:
>>> + if isinstance(test, unittest.TestSuite):
>>> + Asset.precache_suite(test)
>>> + elif isinstance(test, unittest.TestCase):
>>> + Asset.precache_test(test)
>>> +
>>> + def precache_suites(path, cacheTstamp):
>>> + loader = unittest.loader.defaultTestLoader
>>> + tests = loader.loadTestsFromNames([path], None)
>>> +
>>> + with open(cacheTstamp, "w") as fh:
>>> + Asset.precache_suite(tests)
>>
>>
>> When using multiple jobs (-jN) I'm observing some hangs,
>> apparently multiple threads trying to download the same file.
>> The files are eventually downloaded successfully but it takes
>> longer. Should we acquire some exclusive lock somewhere?
>
> I haven't seen that yet ... what did you exactly run? "make check-functional
> -jN" ? Or "make check-functional-<target> -jN" ?
After applying some of your patches, I think I've run now into this problem,
too: It's because test_aarch64_sbsaref.py and test_aarch64_virt.py try to
download the same asset in parallel (alpine-standard-3.17.2-aarch64.iso).
Daniel, any ideas how to fix this in the Asset code?
Thomas
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH v4 15/35] tests/functional: enable pre-emptive caching of assets
2024-08-27 14:24 ` Thomas Huth
@ 2024-08-29 10:15 ` Daniel P. Berrangé
2024-08-30 7:38 ` Thomas Huth
0 siblings, 1 reply; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-08-29 10:15 UTC (permalink / raw)
To: Thomas Huth
Cc: Philippe Mathieu-Daudé, Alex Bennée, qemu-devel,
Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On Tue, Aug 27, 2024 at 04:24:59PM +0200, Thomas Huth wrote:
> On 27/08/2024 15.16, Thomas Huth wrote:
> > On 23/08/2024 09.28, Philippe Mathieu-Daudé wrote:
> > > Hi,
> > >
> > > On 21/8/24 10:27, Thomas Huth wrote:
> > > > From: Daniel P. Berrangé <berrange@redhat.com>
> > > >
> > > > Many tests need to access assets stored on remote sites. We don't want
> > > > to download these during test execution when run by meson, since this
> > > > risks hitting test timeouts when data transfers are slow.
> > > >
> > > > Add support for pre-emptive caching of assets by setting the env var
> > > > QEMU_TEST_PRECACHE to point to a timestamp file. When this is set,
> > > > instead of running the test, the assets will be downloaded and saved
> > > > to the cache, then the timestamp file created.
> > > >
> > > > A meson custom target is created as a dependency of each test suite
> > > > to trigger the pre-emptive caching logic before the test runs.
> > > >
> > > > When run in caching mode, it will locate assets by looking for class
> > > > level variables with a name prefix "ASSET_", and type "Asset".
> > > >
> > > > At the ninja level
> > > >
> > > > ninja test --suite functional
> > > >
> > > > will speculatively download any assets that are not already cached,
> > > > so it is advisable to set a timeout multiplier.
> > > >
> > > > QEMU_TEST_NO_DOWNLOAD=1 ninja test --suite functional
> > > >
> > > > will fail the test if a required asset is not already cached
> > > >
> > > > ninja precache-functional
> > > >
> > > > will download and cache all assets required by the functional
> > > > tests
> > > >
> > > > At the make level, precaching is always done by
> > > >
> > > > make check-functional
> > > >
> > > > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> > > > Tested-by: Richard Henderson <richard.henderson@linaro.org>
> > > > [thuth: Remove the duplicated "path = os.path.basename(...)" line]
> > > > Signed-off-by: Thomas Huth <thuth@redhat.com>
> > > > ---
> > > > tests/Makefile.include | 3 ++-
> > > > tests/functional/meson.build | 33 +++++++++++++++++++++++--
> > > > tests/functional/qemu_test/asset.py | 34 ++++++++++++++++++++++++++
> > > > tests/functional/qemu_test/testcase.py | 7 ++++++
> > > > 4 files changed, 74 insertions(+), 3 deletions(-)
> > > >
> > > > diff --git a/tests/Makefile.include b/tests/Makefile.include
> > > > index 66c8cc3123..010369bd3a 100644
> > > > --- a/tests/Makefile.include
> > > > +++ b/tests/Makefile.include
> > > > @@ -161,7 +161,8 @@ $(FUNCTIONAL_TARGETS):
> > > > .PHONY: check-functional
> > > > check-functional:
> > > > - @$(MAKE) SPEED=thorough check-func check-func-quick
> > > > + @$(NINJA) precache-functional
> > > > + @QEMU_TEST_NO_DOWNLOAD=1 $(MAKE) SPEED=thorough check-func
> > > > check-func-quick
> > > > # Consolidated targets
> > > > diff --git a/tests/functional/meson.build b/tests/functional/meson.build
> > > > index 8a8fa0ab99..cef74b82a9 100644
> > > > --- a/tests/functional/meson.build
> > > > +++ b/tests/functional/meson.build
> > > > @@ -33,6 +33,7 @@ tests_x86_64_quick = [
> > > > tests_x86_64_thorough = [
> > > > ]
> > > > +precache_all = []
> > > > foreach speed : ['quick', 'thorough']
> > > > foreach dir : target_dirs
> > > > if not dir.endswith('-softmmu')
> > > > @@ -63,11 +64,35 @@ foreach speed : ['quick', 'thorough']
> > > > meson.current_source_dir())
> > > > foreach test : target_tests
> > > > - test('func-@0@/@1@'.format(target_base, test),
> > > > + testname = '@0@-@1@'.format(target_base, test)
> > > > + testfile = 'test_' + test + '.py'
> > > > + testpath = meson.current_source_dir() / testfile
> > > > + teststamp = testname + '.tstamp'
> > > > + test_precache_env = environment()
> > > > + test_precache_env.set('QEMU_TEST_PRECACHE',
> > > > meson.current_build_dir() / teststamp)
> > > > + test_precache_env.set('PYTHONPATH',
> > > > meson.project_source_root() / 'python:' +
> > > > + meson.current_source_dir())
> > > > + precache = custom_target('func-precache-' + testname,
> > > > + output: teststamp,
> > > > + command: [python, testpath],
> > > > + depend_files: files(testpath),
> > > > + build_by_default: false,
> > > > + env: test_precache_env)
> > > > + precache_all += precache
> > > > +
> > > > + # Ideally we would add 'precache' to 'depends' here, such that
> > > > + # 'build_by_default: false' lets the pre-caching automatically
> > > > + # run immediately before the test runs. In practice this is
> > > > + # broken in meson, with it running the pre-caching in the normal
> > > > + # compile phase https://github.com/mesonbuild/meson/issues/2518
> > > > + # If the above bug ever gets fixed, when QEMU changes the min
> > > > + # meson version, add the 'depends' and remove the custom
> > > > + # 'run_target' logic below & in Makefile.include
> > > > + test('func-' + testname,
> > > > python,
> > > > depends: [test_deps, test_emulator, emulator_modules],
> > > > env: test_env,
> > > > - args: [meson.current_source_dir() / 'test_' + test + '.py'],
> > > > + args: [testpath],
> > > > protocol: 'tap',
> > > > timeout: test_timeouts.get(test, 60),
> > > > priority: test_timeouts.get(test, 60),
> > > > @@ -75,3 +100,7 @@ foreach speed : ['quick', 'thorough']
> > > > endforeach
> > > > endforeach
> > > > endforeach
> > > > +
> > > > +run_target('precache-functional',
> > > > + depends: precache_all,
> > > > + command: ['true'])
> > > > diff --git a/tests/functional/qemu_test/asset.py
> > > > b/tests/functional/qemu_test/asset.py
> > > > index cbeb6278af..9250ff2b06 100644
> > > > --- a/tests/functional/qemu_test/asset.py
> > > > +++ b/tests/functional/qemu_test/asset.py
> > > > @@ -9,6 +9,8 @@
> > > > import logging
> > > > import os
> > > > import subprocess
> > > > +import sys
> > > > +import unittest
> > > > import urllib.request
> > > > from pathlib import Path
> > > > from shutil import copyfileobj
> > > > @@ -62,6 +64,9 @@ def fetch(self):
> > > > self.cache_file, self.url)
> > > > return str(self.cache_file)
> > > > + if os.environ.get("QEMU_TEST_NO_DOWNLOAD", False):
> > > > + raise Exception("Asset cache is invalid and
> > > > downloads disabled")
> > > > +
> > > > self.log.info("Downloading %s to %s...", self.url,
> > > > self.cache_file)
> > > > tmp_cache_file = self.cache_file.with_suffix(".download")
> > > > @@ -95,3 +100,32 @@ def fetch(self):
> > > > self.log.info("Cached %s at %s" % (self.url, self.cache_file))
> > > > return str(self.cache_file)
> > > > +
> > > > + def precache_test(test):
> > > > + log = logging.getLogger('qemu-test')
> > > > + log.setLevel(logging.DEBUG)
> > > > + handler = logging.StreamHandler(sys.stdout)
> > > > + handler.setLevel(logging.DEBUG)
> > > > + formatter = logging.Formatter(
> > > > + '%(asctime)s - %(name)s - %(levelname)s - %(message)s')
> > > > + handler.setFormatter(formatter)
> > > > + log.addHandler(handler)
> > > > + for name, asset in vars(test.__class__).items():
> > > > + if name.startswith("ASSET_") and type(asset) == Asset:
> > > > + log.info("Attempting to cache '%s'" % asset)
> > > > + asset.fetch()
> > > > + log.removeHandler(handler)
> > > > +
> > > > + def precache_suite(suite):
> > > > + for test in suite:
> > > > + if isinstance(test, unittest.TestSuite):
> > > > + Asset.precache_suite(test)
> > > > + elif isinstance(test, unittest.TestCase):
> > > > + Asset.precache_test(test)
> > > > +
> > > > + def precache_suites(path, cacheTstamp):
> > > > + loader = unittest.loader.defaultTestLoader
> > > > + tests = loader.loadTestsFromNames([path], None)
> > > > +
> > > > + with open(cacheTstamp, "w") as fh:
> > > > + Asset.precache_suite(tests)
> > >
> > >
> > > When using multiple jobs (-jN) I'm observing some hangs,
> > > apparently multiple threads trying to download the same file.
> > > The files are eventually downloaded successfully but it takes
> > > longer. Should we acquire some exclusive lock somewhere?
> >
> > I haven't seen that yet ... what did you exactly run? "make
> > check-functional -jN" ? Or "make check-functional-<target> -jN" ?
>
> After applying some of your patches, I think I've run now into this problem,
> too: It's because test_aarch64_sbsaref.py and test_aarch64_virt.py try to
> download the same asset in parallel (alpine-standard-3.17.2-aarch64.iso).
>
> Daniel, any ideas how to fix this in the Asset code?
So when downloading we open a file with a ".download" suffix, write to
that, and then rename it to the final filename.
If we have concurrent usage, both will open the same file and try to
write to it. Assuming both are downloading the same content we would
probably "get lucky" and have a consistent file at the end, but clearly
it is bad to rely on luck.
The lame option is to use NamedTemporaryFile for the teporary file.
This ensures both processes will write to different temp files, and
the final rename is atomic. This guarantees safety, but still has
the double download penalty.
The serious option is to use fcntl.lockf(..., fcntl.LOCK_EX) on the
temp file. If we can't acquire the lock then just immediately close
the temp file (don't delete it) and assume another thread is going to
finish its download.
On windows we'll need msvcrt.locking(..., msvcrt.LK_WLCK, ...)
instead of fcntl.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH v4 15/35] tests/functional: enable pre-emptive caching of assets
2024-08-29 10:15 ` Daniel P. Berrangé
@ 2024-08-30 7:38 ` Thomas Huth
2024-08-30 7:42 ` Daniel P. Berrangé
0 siblings, 1 reply; 69+ messages in thread
From: Thomas Huth @ 2024-08-30 7:38 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Philippe Mathieu-Daudé, Alex Bennée, qemu-devel,
Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On 29/08/2024 12.15, Daniel P. Berrangé wrote:
> On Tue, Aug 27, 2024 at 04:24:59PM +0200, Thomas Huth wrote:
>> On 27/08/2024 15.16, Thomas Huth wrote:
>>> On 23/08/2024 09.28, Philippe Mathieu-Daudé wrote:
>>>> Hi,
>>>>
>>>> On 21/8/24 10:27, Thomas Huth wrote:
>>>>> From: Daniel P. Berrangé <berrange@redhat.com>
>>>>>
>>>>> Many tests need to access assets stored on remote sites. We don't want
>>>>> to download these during test execution when run by meson, since this
>>>>> risks hitting test timeouts when data transfers are slow.
>>>>>
>>>>> Add support for pre-emptive caching of assets by setting the env var
>>>>> QEMU_TEST_PRECACHE to point to a timestamp file. When this is set,
>>>>> instead of running the test, the assets will be downloaded and saved
>>>>> to the cache, then the timestamp file created.
...
>>>>
>>>> When using multiple jobs (-jN) I'm observing some hangs,
>>>> apparently multiple threads trying to download the same file.
>>>> The files are eventually downloaded successfully but it takes
>>>> longer. Should we acquire some exclusive lock somewhere?
>>>
>>> I haven't seen that yet ... what did you exactly run? "make
>>> check-functional -jN" ? Or "make check-functional-<target> -jN" ?
>>
>> After applying some of your patches, I think I've run now into this problem,
>> too: It's because test_aarch64_sbsaref.py and test_aarch64_virt.py try to
>> download the same asset in parallel (alpine-standard-3.17.2-aarch64.iso).
>>
>> Daniel, any ideas how to fix this in the Asset code?
>
> So when downloading we open a file with a ".download" suffix, write to
> that, and then rename it to the final filename.
>
> If we have concurrent usage, both will open the same file and try to
> write to it. Assuming both are downloading the same content we would
> probably "get lucky" and have a consistent file at the end, but clearly
> it is bad to rely on luck.
>
> The lame option is to use NamedTemporaryFile for the teporary file.
> This ensures both processes will write to different temp files, and
> the final rename is atomic. This guarantees safety, but still has
> the double download penalty.
>
> The serious option is to use fcntl.lockf(..., fcntl.LOCK_EX) on the
> temp file. If we can't acquire the lock then just immediately close
> the temp file (don't delete it) and assume another thread is going to
> finish its download.
>
> On windows we'll need msvcrt.locking(..., msvcrt.LK_WLCK, ...)
> instead of fcntl.
While looking for portable solutions, I noticed that newer versions
of Python have a "x" mode for creating files only if they do not
exist yet. So I think something like this could be a solution:
@@ -71,17 +72,26 @@ def fetch(self):
tmp_cache_file = self.cache_file.with_suffix(".download")
try:
- resp = urllib.request.urlopen(self.url)
+ with tmp_cache_file.open("xb") as dst:
+ with urllib.request.urlopen(self.url) as resp:
+ copyfileobj(resp, dst)
+ except FileExistsError:
+ # Another thread already seems to download this asset,
+ # so wait until it is done
+ self.log.debug("%s already exists, waiting for other thread to finish...",
+ tmp_cache_file)
+ i = 0
+ while i < 600 and os.path.exists(tmp_cache_file):
+ sleep(1)
+ i += 1
+ if os.path.exists(self.cache_file):
+ return str(self.cache_file)
+ raise
except Exception as e:
self.log.error("Unable to download %s: %s", self.url, e)
- raise
-
- try:
- with tmp_cache_file.open("wb+") as dst:
- copyfileobj(resp, dst)
- except:
tmp_cache_file.unlink()
raise
+
try:
# Set these just for informational purposes
os.setxattr(str(tmp_cache_file), "user.qemu-asset-url",
What do you think, does it look reasonable?
Thomas
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH v4 15/35] tests/functional: enable pre-emptive caching of assets
2024-08-30 7:38 ` Thomas Huth
@ 2024-08-30 7:42 ` Daniel P. Berrangé
2024-08-30 11:27 ` Thomas Huth
0 siblings, 1 reply; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-08-30 7:42 UTC (permalink / raw)
To: Thomas Huth
Cc: Philippe Mathieu-Daudé, Alex Bennée, qemu-devel,
Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On Fri, Aug 30, 2024 at 09:38:17AM +0200, Thomas Huth wrote:
> On 29/08/2024 12.15, Daniel P. Berrangé wrote:
> > On Tue, Aug 27, 2024 at 04:24:59PM +0200, Thomas Huth wrote:
> > > On 27/08/2024 15.16, Thomas Huth wrote:
> > > > On 23/08/2024 09.28, Philippe Mathieu-Daudé wrote:
> > > > > Hi,
> > > > >
> > > > > On 21/8/24 10:27, Thomas Huth wrote:
> > > > > > From: Daniel P. Berrangé <berrange@redhat.com>
> > > > > >
> > > > > > Many tests need to access assets stored on remote sites. We don't want
> > > > > > to download these during test execution when run by meson, since this
> > > > > > risks hitting test timeouts when data transfers are slow.
> > > > > >
> > > > > > Add support for pre-emptive caching of assets by setting the env var
> > > > > > QEMU_TEST_PRECACHE to point to a timestamp file. When this is set,
> > > > > > instead of running the test, the assets will be downloaded and saved
> > > > > > to the cache, then the timestamp file created.
> ...
> > > > >
> > > > > When using multiple jobs (-jN) I'm observing some hangs,
> > > > > apparently multiple threads trying to download the same file.
> > > > > The files are eventually downloaded successfully but it takes
> > > > > longer. Should we acquire some exclusive lock somewhere?
> > > >
> > > > I haven't seen that yet ... what did you exactly run? "make
> > > > check-functional -jN" ? Or "make check-functional-<target> -jN" ?
> > >
> > > After applying some of your patches, I think I've run now into this problem,
> > > too: It's because test_aarch64_sbsaref.py and test_aarch64_virt.py try to
> > > download the same asset in parallel (alpine-standard-3.17.2-aarch64.iso).
> > >
> > > Daniel, any ideas how to fix this in the Asset code?
> >
> > So when downloading we open a file with a ".download" suffix, write to
> > that, and then rename it to the final filename.
> >
> > If we have concurrent usage, both will open the same file and try to
> > write to it. Assuming both are downloading the same content we would
> > probably "get lucky" and have a consistent file at the end, but clearly
> > it is bad to rely on luck.
> >
> > The lame option is to use NamedTemporaryFile for the teporary file.
> > This ensures both processes will write to different temp files, and
> > the final rename is atomic. This guarantees safety, but still has
> > the double download penalty.
> >
> > The serious option is to use fcntl.lockf(..., fcntl.LOCK_EX) on the
> > temp file. If we can't acquire the lock then just immediately close
> > the temp file (don't delete it) and assume another thread is going to
> > finish its download.
> >
> > On windows we'll need msvcrt.locking(..., msvcrt.LK_WLCK, ...)
> > instead of fcntl.
>
> While looking for portable solutions, I noticed that newer versions
> of Python have a "x" mode for creating files only if they do not
> exist yet. So I think something like this could be a solution:
>
> @@ -71,17 +72,26 @@ def fetch(self):
> tmp_cache_file = self.cache_file.with_suffix(".download")
> try:
> - resp = urllib.request.urlopen(self.url)
> + with tmp_cache_file.open("xb") as dst:
> + with urllib.request.urlopen(self.url) as resp:
> + copyfileobj(resp, dst)
> + except FileExistsError:
> + # Another thread already seems to download this asset,
> + # so wait until it is done
> + self.log.debug("%s already exists, waiting for other thread to finish...",
> + tmp_cache_file)
> + i = 0
> + while i < 600 and os.path.exists(tmp_cache_file):
> + sleep(1)
> + i += 1
> + if os.path.exists(self.cache_file):
> + return str(self.cache_file)
> + raise
> except Exception as e:
> self.log.error("Unable to download %s: %s", self.url, e)
> - raise
> -
> - try:
> - with tmp_cache_file.open("wb+") as dst:
> - copyfileobj(resp, dst)
> - except:
> tmp_cache_file.unlink()
> raise
> +
> try:
> # Set these just for informational purposes
> os.setxattr(str(tmp_cache_file), "user.qemu-asset-url",
>
> What do you think, does it look reasonable?
The main risk with this, as opposed to fcntl locking, is that it is not
crash-safe. If a download is interrupted, subsequent cache runs will
wait for a process that doesn't exist to finish downloading and then
raise an exception, requiring manual user cleanup of the partial
download.
Perhaps if we see the tmp_cache_file, and it doesn't change in size
after N seconds, we could force unlink it, and create a new download,
so we gracefully recover ?
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH v4 15/35] tests/functional: enable pre-emptive caching of assets
2024-08-30 7:42 ` Daniel P. Berrangé
@ 2024-08-30 11:27 ` Thomas Huth
2024-08-30 11:37 ` Daniel P. Berrangé
0 siblings, 1 reply; 69+ messages in thread
From: Thomas Huth @ 2024-08-30 11:27 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Philippe Mathieu-Daudé, Alex Bennée, qemu-devel,
Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On 30/08/2024 09.42, Daniel P. Berrangé wrote:
> On Fri, Aug 30, 2024 at 09:38:17AM +0200, Thomas Huth wrote:
>> On 29/08/2024 12.15, Daniel P. Berrangé wrote:
>>> On Tue, Aug 27, 2024 at 04:24:59PM +0200, Thomas Huth wrote:
>>>> On 27/08/2024 15.16, Thomas Huth wrote:
>>>>> On 23/08/2024 09.28, Philippe Mathieu-Daudé wrote:
>>>>>> Hi,
>>>>>>
>>>>>> On 21/8/24 10:27, Thomas Huth wrote:
>>>>>>> From: Daniel P. Berrangé <berrange@redhat.com>
>>>>>>>
>>>>>>> Many tests need to access assets stored on remote sites. We don't want
>>>>>>> to download these during test execution when run by meson, since this
>>>>>>> risks hitting test timeouts when data transfers are slow.
>>>>>>>
>>>>>>> Add support for pre-emptive caching of assets by setting the env var
>>>>>>> QEMU_TEST_PRECACHE to point to a timestamp file. When this is set,
>>>>>>> instead of running the test, the assets will be downloaded and saved
>>>>>>> to the cache, then the timestamp file created.
>> ...
>>>>>>
>>>>>> When using multiple jobs (-jN) I'm observing some hangs,
>>>>>> apparently multiple threads trying to download the same file.
>>>>>> The files are eventually downloaded successfully but it takes
>>>>>> longer. Should we acquire some exclusive lock somewhere?
>>>>>
>>>>> I haven't seen that yet ... what did you exactly run? "make
>>>>> check-functional -jN" ? Or "make check-functional-<target> -jN" ?
>>>>
>>>> After applying some of your patches, I think I've run now into this problem,
>>>> too: It's because test_aarch64_sbsaref.py and test_aarch64_virt.py try to
>>>> download the same asset in parallel (alpine-standard-3.17.2-aarch64.iso).
>>>>
>>>> Daniel, any ideas how to fix this in the Asset code?
>>>
>>> So when downloading we open a file with a ".download" suffix, write to
>>> that, and then rename it to the final filename.
>>>
>>> If we have concurrent usage, both will open the same file and try to
>>> write to it. Assuming both are downloading the same content we would
>>> probably "get lucky" and have a consistent file at the end, but clearly
>>> it is bad to rely on luck.
>>>
>>> The lame option is to use NamedTemporaryFile for the teporary file.
>>> This ensures both processes will write to different temp files, and
>>> the final rename is atomic. This guarantees safety, but still has
>>> the double download penalty.
>>>
>>> The serious option is to use fcntl.lockf(..., fcntl.LOCK_EX) on the
>>> temp file. If we can't acquire the lock then just immediately close
>>> the temp file (don't delete it) and assume another thread is going to
>>> finish its download.
>>>
>>> On windows we'll need msvcrt.locking(..., msvcrt.LK_WLCK, ...)
>>> instead of fcntl.
>>
>> While looking for portable solutions, I noticed that newer versions
>> of Python have a "x" mode for creating files only if they do not
>> exist yet. So I think something like this could be a solution:
>>
>> @@ -71,17 +72,26 @@ def fetch(self):
>> tmp_cache_file = self.cache_file.with_suffix(".download")
>> try:
>> - resp = urllib.request.urlopen(self.url)
>> + with tmp_cache_file.open("xb") as dst:
>> + with urllib.request.urlopen(self.url) as resp:
>> + copyfileobj(resp, dst)
>> + except FileExistsError:
>> + # Another thread already seems to download this asset,
>> + # so wait until it is done
>> + self.log.debug("%s already exists, waiting for other thread to finish...",
>> + tmp_cache_file)
>> + i = 0
>> + while i < 600 and os.path.exists(tmp_cache_file):
>> + sleep(1)
>> + i += 1
>> + if os.path.exists(self.cache_file):
>> + return str(self.cache_file)
>> + raise
>> except Exception as e:
>> self.log.error("Unable to download %s: %s", self.url, e)
>> - raise
>> -
>> - try:
>> - with tmp_cache_file.open("wb+") as dst:
>> - copyfileobj(resp, dst)
>> - except:
>> tmp_cache_file.unlink()
>> raise
>> +
>> try:
>> # Set these just for informational purposes
>> os.setxattr(str(tmp_cache_file), "user.qemu-asset-url",
>>
>> What do you think, does it look reasonable?
>
> The main risk with this, as opposed to fcntl locking, is that it is not
> crash-safe. If a download is interrupted, subsequent cache runs will
> wait for a process that doesn't exist to finish downloading and then
> raise an exception, requiring manual user cleanup of the partial
> download.
>
> Perhaps if we see the tmp_cache_file, and it doesn't change in size
> after N seconds, we could force unlink it, and create a new download,
> so we gracefully recover ?
Sounds like a plan ... does this look acceptable:
@@ -70,18 +71,52 @@ def fetch(self):
self.log.info("Downloading %s to %s...", self.url, self.cache_file)
tmp_cache_file = self.cache_file.with_suffix(".download")
- try:
- resp = urllib.request.urlopen(self.url)
- except Exception as e:
- self.log.error("Unable to download %s: %s", self.url, e)
- raise
+ for retries in range(3):
+ try:
+ with tmp_cache_file.open("xb") as dst:
+ with urllib.request.urlopen(self.url) as resp:
+ copyfileobj(resp, dst)
+ break
+ except FileExistsError:
+ # Another thread already seems to download this asset,
+ # so wait until it is done
+ self.log.debug("%s already exists, "
+ "waiting for other thread to finish...",
+ tmp_cache_file)
+ try:
+ current_size = tmp_cache_file.stat().st_size
+ new_size = current_size
+ except:
+ if os.path.exists(self.cache_file):
+ return str(self.cache_file)
+ raise
+ waittime = lastchange = 600
+ while waittime > 0:
+ sleep(1)
+ waittime -= 1
+ try:
+ new_size = tmp_cache_file.stat().st_size
+ except:
+ if os.path.exists(self.cache_file):
+ return str(self.cache_file)
+ raise
+ if new_size != current_size:
+ lastchange = waittime
+ current_size = new_size
+ elif lastchange - waittime > 90:
+ self.log.debug("%s seems to be stale ... "
+ "deleting and retrying download...",
+ tmp_cache_file)
+ tmp_cache_file.unlink()
+ break
+ if waittime > 0:
+ continue
+ raise
+ except Exception as e:
+ self.log.error("Unable to download %s: %s", self.url, e)
+ tmp_cache_file.unlink()
+ raise
- try:
- with tmp_cache_file.open("wb+") as dst:
- copyfileobj(resp, dst)
- except:
- tmp_cache_file.unlink()
- raise
try:
# Set these just for informational purposes
os.setxattr(str(tmp_cache_file), "user.qemu-asset-url",
?
I tried it with a stale file in my cache, and it seems to work - after 90
seconds, one of the threads is properly trying to redownload the file.
Thomas
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH v4 15/35] tests/functional: enable pre-emptive caching of assets
2024-08-30 11:27 ` Thomas Huth
@ 2024-08-30 11:37 ` Daniel P. Berrangé
0 siblings, 0 replies; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-08-30 11:37 UTC (permalink / raw)
To: Thomas Huth
Cc: Philippe Mathieu-Daudé, Alex Bennée, qemu-devel,
Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On Fri, Aug 30, 2024 at 01:27:15PM +0200, Thomas Huth wrote:
> On 30/08/2024 09.42, Daniel P. Berrangé wrote:
> > On Fri, Aug 30, 2024 at 09:38:17AM +0200, Thomas Huth wrote:
> > > On 29/08/2024 12.15, Daniel P. Berrangé wrote:
> > > > On Tue, Aug 27, 2024 at 04:24:59PM +0200, Thomas Huth wrote:
> > > > > On 27/08/2024 15.16, Thomas Huth wrote:
> > > > > > On 23/08/2024 09.28, Philippe Mathieu-Daudé wrote:
> > > > > > > Hi,
> > > > > > >
> > > > > > > On 21/8/24 10:27, Thomas Huth wrote:
> > > > > > > > From: Daniel P. Berrangé <berrange@redhat.com>
> > > > > > > >
> > > > > > > > Many tests need to access assets stored on remote sites. We don't want
> > > > > > > > to download these during test execution when run by meson, since this
> > > > > > > > risks hitting test timeouts when data transfers are slow.
> > > > > > > >
> > > > > > > > Add support for pre-emptive caching of assets by setting the env var
> > > > > > > > QEMU_TEST_PRECACHE to point to a timestamp file. When this is set,
> > > > > > > > instead of running the test, the assets will be downloaded and saved
> > > > > > > > to the cache, then the timestamp file created.
> > > ...
> > > > > > >
> > > > > > > When using multiple jobs (-jN) I'm observing some hangs,
> > > > > > > apparently multiple threads trying to download the same file.
> > > > > > > The files are eventually downloaded successfully but it takes
> > > > > > > longer. Should we acquire some exclusive lock somewhere?
> > > > > >
> > > > > > I haven't seen that yet ... what did you exactly run? "make
> > > > > > check-functional -jN" ? Or "make check-functional-<target> -jN" ?
> > > > >
> > > > > After applying some of your patches, I think I've run now into this problem,
> > > > > too: It's because test_aarch64_sbsaref.py and test_aarch64_virt.py try to
> > > > > download the same asset in parallel (alpine-standard-3.17.2-aarch64.iso).
> > > > >
> > > > > Daniel, any ideas how to fix this in the Asset code?
> > > >
> > > > So when downloading we open a file with a ".download" suffix, write to
> > > > that, and then rename it to the final filename.
> > > >
> > > > If we have concurrent usage, both will open the same file and try to
> > > > write to it. Assuming both are downloading the same content we would
> > > > probably "get lucky" and have a consistent file at the end, but clearly
> > > > it is bad to rely on luck.
> > > >
> > > > The lame option is to use NamedTemporaryFile for the teporary file.
> > > > This ensures both processes will write to different temp files, and
> > > > the final rename is atomic. This guarantees safety, but still has
> > > > the double download penalty.
> > > >
> > > > The serious option is to use fcntl.lockf(..., fcntl.LOCK_EX) on the
> > > > temp file. If we can't acquire the lock then just immediately close
> > > > the temp file (don't delete it) and assume another thread is going to
> > > > finish its download.
> > > >
> > > > On windows we'll need msvcrt.locking(..., msvcrt.LK_WLCK, ...)
> > > > instead of fcntl.
> > >
> > > While looking for portable solutions, I noticed that newer versions
> > > of Python have a "x" mode for creating files only if they do not
> > > exist yet. So I think something like this could be a solution:
> > >
> > > @@ -71,17 +72,26 @@ def fetch(self):
> > > tmp_cache_file = self.cache_file.with_suffix(".download")
> > > try:
> > > - resp = urllib.request.urlopen(self.url)
> > > + with tmp_cache_file.open("xb") as dst:
> > > + with urllib.request.urlopen(self.url) as resp:
> > > + copyfileobj(resp, dst)
> > > + except FileExistsError:
> > > + # Another thread already seems to download this asset,
> > > + # so wait until it is done
> > > + self.log.debug("%s already exists, waiting for other thread to finish...",
> > > + tmp_cache_file)
> > > + i = 0
> > > + while i < 600 and os.path.exists(tmp_cache_file):
> > > + sleep(1)
> > > + i += 1
> > > + if os.path.exists(self.cache_file):
> > > + return str(self.cache_file)
> > > + raise
> > > except Exception as e:
> > > self.log.error("Unable to download %s: %s", self.url, e)
> > > - raise
> > > -
> > > - try:
> > > - with tmp_cache_file.open("wb+") as dst:
> > > - copyfileobj(resp, dst)
> > > - except:
> > > tmp_cache_file.unlink()
> > > raise
> > > +
> > > try:
> > > # Set these just for informational purposes
> > > os.setxattr(str(tmp_cache_file), "user.qemu-asset-url",
> > >
> > > What do you think, does it look reasonable?
> >
> > The main risk with this, as opposed to fcntl locking, is that it is not
> > crash-safe. If a download is interrupted, subsequent cache runs will
> > wait for a process that doesn't exist to finish downloading and then
> > raise an exception, requiring manual user cleanup of the partial
> > download.
> >
> > Perhaps if we see the tmp_cache_file, and it doesn't change in size
> > after N seconds, we could force unlink it, and create a new download,
> > so we gracefully recover ?
>
> Sounds like a plan ... does this look acceptable:
>
> @@ -70,18 +71,52 @@ def fetch(self):
> self.log.info("Downloading %s to %s...", self.url, self.cache_file)
> tmp_cache_file = self.cache_file.with_suffix(".download")
>
> - try:
> - resp = urllib.request.urlopen(self.url)
> - except Exception as e:
> - self.log.error("Unable to download %s: %s", self.url, e)
> - raise
> + for retries in range(3):
> + try:
> + with tmp_cache_file.open("xb") as dst:
> + with urllib.request.urlopen(self.url) as resp:
> + copyfileobj(resp, dst)
> + break
> + except FileExistsError:
> + # Another thread already seems to download this asset,
> + # so wait until it is done
> + self.log.debug("%s already exists, "
> + "waiting for other thread to finish...",
> + tmp_cache_file)
> + try:
> + current_size = tmp_cache_file.stat().st_size
> + new_size = current_size
> + except:
> + if os.path.exists(self.cache_file):
> + return str(self.cache_file)
> + raise
> + waittime = lastchange = 600
> + while waittime > 0:
> + sleep(1)
> + waittime -= 1
> + try:
> + new_size = tmp_cache_file.stat().st_size
> + except:
> + if os.path.exists(self.cache_file):
> + return str(self.cache_file)
> + raise
> + if new_size != current_size:
> + lastchange = waittime
> + current_size = new_size
> + elif lastchange - waittime > 90:
> + self.log.debug("%s seems to be stale ... "
> + "deleting and retrying download...",
> + tmp_cache_file)
> + tmp_cache_file.unlink()
> + break
> + if waittime > 0:
> + continue
> + raise
> + except Exception as e:
> + self.log.error("Unable to download %s: %s", self.url, e)
> + tmp_cache_file.unlink()
> + raise
>
> - try:
> - with tmp_cache_file.open("wb+") as dst:
> - copyfileobj(resp, dst)
> - except:
> - tmp_cache_file.unlink()
> - raise
> try:
> # Set these just for informational purposes
> os.setxattr(str(tmp_cache_file), "user.qemu-asset-url",
>
> ?
>
> I tried it with a stale file in my cache, and it seems to work - after 90
> seconds, one of the threads is properly trying to redownload the file.
Yeah, I think that'll work ok
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 69+ messages in thread
* [PATCH v4 16/35] tests/functional: Convert some tests that download files via fetch_asset()
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (14 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 15/35] tests/functional: enable pre-emptive caching of assets Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 8:27 ` [PATCH v4 17/35] tests/functional: Add a function for extracting files from an archive Thomas Huth
` (18 subsequent siblings)
34 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
Now that we've got the Asset class with pre-caching, we can convert
some Avocado tests that use fetch_asset() for downloading their
required files.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 12 ++---
tests/avocado/machine_mips_loongson3v.py | 39 --------------
tests/functional/meson.build | 22 ++++++++
.../test_arm_n8x0.py} | 36 ++++++-------
.../test_avr_mega2560.py} | 22 ++++----
.../test_loongarch64_virt.py} | 46 +++++++++--------
tests/functional/test_mips64el_loongson3v.py | 39 ++++++++++++++
.../test_netdev_ethtool.py} | 51 +++++++------------
.../ppc_405.py => functional/test_ppc_405.py} | 29 ++++++-----
9 files changed, 156 insertions(+), 140 deletions(-)
delete mode 100644 tests/avocado/machine_mips_loongson3v.py
rename tests/{avocado/machine_arm_n8x0.py => functional/test_arm_n8x0.py} (51%)
mode change 100644 => 100755
rename tests/{avocado/machine_avr6.py => functional/test_avr_mega2560.py} (72%)
mode change 100644 => 100755
rename tests/{avocado/machine_loongarch.py => functional/test_loongarch64_virt.py} (54%)
mode change 100644 => 100755
create mode 100755 tests/functional/test_mips64el_loongson3v.py
rename tests/{avocado/netdev-ethtool.py => functional/test_netdev_ethtool.py} (66%)
mode change 100644 => 100755
rename tests/{avocado/ppc_405.py => functional/test_ppc_405.py} (53%)
mode change 100644 => 100755
diff --git a/MAINTAINERS b/MAINTAINERS
index 4912f8d0da..6545662eaf 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -222,7 +222,7 @@ S: Maintained
F: docs/system/target-avr.rst
F: gdb-xml/avr-cpu.xml
F: target/avr/
-F: tests/avocado/machine_avr6.py
+F: tests/functional/test_avr_mega2560.py
CRIS TCG CPUs
M: Edgar E. Iglesias <edgar.iglesias@gmail.com>
@@ -266,7 +266,7 @@ M: Song Gao <gaosong@loongson.cn>
S: Maintained
F: target/loongarch/
F: tests/tcg/loongarch64/
-F: tests/avocado/machine_loongarch.py
+F: tests/functional/test_loongarch64_virt.py
M68K TCG CPUs
M: Laurent Vivier <laurent@vivier.eu>
@@ -886,7 +886,7 @@ F: include/hw/display/blizzard.h
F: include/hw/input/lm832x.h
F: include/hw/input/tsc2xxx.h
F: include/hw/misc/cbus.h
-F: tests/avocado/machine_arm_n8x0.py
+F: tests/functional/test_arm_n8x0.py
F: docs/system/arm/nseries.rst
Palm
@@ -1397,7 +1397,7 @@ F: hw/mips/loongson3_virt.c
F: include/hw/intc/loongson_ipi_common.h
F: include/hw/intc/loongson_ipi.h
F: include/hw/intc/loongson_liointc.h
-F: tests/avocado/machine_mips_loongson3v.py
+F: tests/functional/test_mips64el_loongson3v.py
Boston
M: Paul Burton <paulburton@kernel.org>
@@ -1423,7 +1423,7 @@ PowerPC Machines
L: qemu-ppc@nongnu.org
S: Orphan
F: hw/ppc/ppc405*
-F: tests/avocado/ppc_405.py
+F: tests/functional/test_ppc_405.py
Bamboo
L: qemu-ppc@nongnu.org
@@ -2495,7 +2495,7 @@ R: Sriram Yagnaraman <sriram.yagnaraman@ericsson.com>
S: Maintained
F: docs/system/devices/igb.rst
F: hw/net/igb*
-F: tests/avocado/netdev-ethtool.py
+F: tests/functional/test_netdev_ethtool.py
F: tests/qtest/igb-test.c
F: tests/qtest/libqos/igb.c
diff --git a/tests/avocado/machine_mips_loongson3v.py b/tests/avocado/machine_mips_loongson3v.py
deleted file mode 100644
index 5194cf18c9..0000000000
--- a/tests/avocado/machine_mips_loongson3v.py
+++ /dev/null
@@ -1,39 +0,0 @@
-# Functional tests for the Generic Loongson-3 Platform.
-#
-# Copyright (c) 2021 Jiaxun Yang <jiaxun.yang@flygoat.com>
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or later.
-# See the COPYING file in the top-level directory.
-#
-# SPDX-License-Identifier: GPL-2.0-or-later
-
-import os
-import time
-
-from avocado import skipUnless
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
-
-class MipsLoongson3v(QemuSystemTest):
- timeout = 60
-
- @skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
- def test_pmon_serial_console(self):
- """
- :avocado: tags=arch:mips64el
- :avocado: tags=endian:little
- :avocado: tags=machine:loongson3-virt
- :avocado: tags=cpu:Loongson-3A1000
- :avocado: tags=device:liointc
- :avocado: tags=device:goldfish_rtc
- """
-
- pmon_hash = '7c8b45dd81ccfc55ff28f5aa267a41c3'
- pmon_path = self.fetch_asset('https://github.com/loongson-community/pmon/'
- 'releases/download/20210112/pmon-3avirt.bin',
- asset_hash=pmon_hash, algorithm='md5')
-
- self.vm.set_console()
- self.vm.add_args('-bios', pmon_path)
- self.vm.launch()
- wait_for_console_pattern(self, 'CPU GODSON3 BogoMIPS:')
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index cef74b82a9..32c7cc2d3c 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -11,6 +11,7 @@ endif
# Timeouts for individual tests that can be slow e.g. with debugging enabled
test_timeouts = {
+ 'netdev_ethtool' : 180,
}
tests_generic = [
@@ -19,10 +20,30 @@ tests_generic = [
'version',
]
+tests_arm_thorough = [
+ 'arm_n8x0',
+]
+
+tests_avr_thorough = [
+ 'avr_mega2560',
+]
+
+tests_loongarch64_thorough = [
+ 'loongarch64_virt',
+]
+
+tests_mips64el_thorough = [
+ 'mips64el_loongson3v',
+]
+
tests_ppc_quick = [
'ppc_74xx',
]
+tests_ppc_thorough = [
+ 'ppc_405',
+]
+
tests_x86_64_quick = [
'cpu_queries',
'mem_addr_space',
@@ -31,6 +52,7 @@ tests_x86_64_quick = [
]
tests_x86_64_thorough = [
+ 'netdev_ethtool',
]
precache_all = []
diff --git a/tests/avocado/machine_arm_n8x0.py b/tests/functional/test_arm_n8x0.py
old mode 100644
new mode 100755
similarity index 51%
rename from tests/avocado/machine_arm_n8x0.py
rename to tests/functional/test_arm_n8x0.py
index 12e9a6803b..ffcf468fe8
--- a/tests/avocado/machine_arm_n8x0.py
+++ b/tests/functional/test_arm_n8x0.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Functional test that boots a Linux kernel and checks the console
#
# Copyright (c) 2020 Red Hat, Inc.
@@ -10,21 +12,22 @@
import os
-from avocado import skipUnless
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
+from unittest import skipUnless
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
class N8x0Machine(QemuSystemTest):
"""Boots the Linux kernel and checks that the console is operational"""
timeout = 90
+ ASSET_KERNEL = Asset(
+ ('http://stskeeps.subnetmask.net/meego-n8x0/'
+ 'meego-arm-n8x0-1.0.80.20100712.1431-vmlinuz-2.6.35~rc4-129.1-n8x0'),
+ '629f4616caacacd116d23d7572178f93691b0b918b25e28d7736c24f888d4292')
+
def __do_test_n8x0(self):
- kernel_url = ('http://stskeeps.subnetmask.net/meego-n8x0/'
- 'meego-arm-n8x0-1.0.80.20100712.1431-'
- 'vmlinuz-2.6.35~rc4-129.1-n8x0')
- kernel_hash = 'e9d5ab8d7548923a0061b6fbf601465e479ed269'
- kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+ kernel_path = self.ASSET_KERNEL.fetch()
self.vm.set_console(console_index=1)
self.vm.add_args('-kernel', kernel_path,
@@ -32,18 +35,15 @@ def __do_test_n8x0(self):
self.vm.launch()
wait_for_console_pattern(self, 'TSC2005 driver initializing')
- @skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
+ @skipUnless(os.getenv('QEMU_TEST_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
def test_n800(self):
- """
- :avocado: tags=arch:arm
- :avocado: tags=machine:n800
- """
+ self.set_machine('n800')
self.__do_test_n8x0()
- @skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
+ @skipUnless(os.getenv('QEMU_TEST_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
def test_n810(self):
- """
- :avocado: tags=arch:arm
- :avocado: tags=machine:n810
- """
+ self.set_machine('n810')
self.__do_test_n8x0()
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/machine_avr6.py b/tests/functional/test_avr_mega2560.py
old mode 100644
new mode 100755
similarity index 72%
rename from tests/avocado/machine_avr6.py
rename to tests/functional/test_avr_mega2560.py
index 5485db79c6..8e47b4200b
--- a/tests/avocado/machine_avr6.py
+++ b/tests/functional/test_avr_mega2560.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
#
# QEMU AVR integration tests
#
@@ -19,26 +20,24 @@
import time
-from avocado_qemu import QemuSystemTest
+from qemu_test import QemuSystemTest, Asset
class AVR6Machine(QemuSystemTest):
timeout = 5
+ ASSET_ROM = Asset(('https://github.com/seharris/qemu-avr-tests'
+ '/raw/36c3e67b8755dcf/free-rtos/Demo'
+ '/AVR_ATMega2560_GCC/demo.elf'),
+ 'ee4833bd65fc69e84a79ed1c608affddbd499a60e63acf87d9113618401904e4')
+
def test_freertos(self):
- """
- :avocado: tags=arch:avr
- :avocado: tags=machine:arduino-mega-2560-v3
- """
"""
https://github.com/seharris/qemu-avr-tests/raw/master/free-rtos/Demo/AVR_ATMega2560_GCC/demo.elf
constantly prints out 'ABCDEFGHIJKLMNOPQRSTUVWXABCDEFGHIJKLMNOPQRSTUVWX'
"""
- rom_url = ('https://github.com/seharris/qemu-avr-tests'
- '/raw/36c3e67b8755dcf/free-rtos/Demo'
- '/AVR_ATMega2560_GCC/demo.elf')
- rom_hash = '7eb521f511ca8f2622e0a3c5e8dd686efbb911d4'
- rom_path = self.fetch_asset(rom_url, asset_hash=rom_hash)
+ rom_path = self.ASSET_ROM.fetch()
+ self.set_machine('arduino-mega-2560-v3')
self.vm.add_args('-bios', rom_path)
self.vm.add_args('-nographic')
self.vm.launch()
@@ -48,3 +47,6 @@ def test_freertos(self):
self.assertIn('ABCDEFGHIJKLMNOPQRSTUVWXABCDEFGHIJKLMNOPQRSTUVWX',
self.vm.get_log())
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/machine_loongarch.py b/tests/functional/test_loongarch64_virt.py
old mode 100644
new mode 100755
similarity index 54%
rename from tests/avocado/machine_loongarch.py
rename to tests/functional/test_loongarch64_virt.py
index 8de308f2d6..2b8baa2c2a
--- a/tests/avocado/machine_loongarch.py
+++ b/tests/functional/test_loongarch64_virt.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# SPDX-License-Identifier: GPL-2.0-or-later
#
# LoongArch virt test.
@@ -5,15 +7,28 @@
# Copyright (c) 2023 Loongson Technology Corporation Limited
#
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import exec_command_and_wait_for_pattern
-from avocado_qemu import wait_for_console_pattern
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import exec_command_and_wait_for_pattern
+from qemu_test import wait_for_console_pattern
class LoongArchMachine(QemuSystemTest):
KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
timeout = 120
+ ASSET_KERNEL = Asset(
+ ('https://github.com/yangxiaojuan-loongson/qemu-binary/'
+ 'releases/download/2024-05-30/vmlinuz.efi'),
+ '08b88a45f48a5fd92260bae895be4e5175be2397481a6f7821b9f39b2965b79e')
+ ASSET_INITRD = Asset(
+ ('https://github.com/yangxiaojuan-loongson/qemu-binary/'
+ 'releases/download/2024-05-30/ramdisk'),
+ '03d6fb6f8ee64ecac961120a0bdacf741f17b3bee2141f17fa01908c8baf176a')
+ ASSET_BIOS = Asset(
+ ('https://github.com/yangxiaojuan-loongson/qemu-binary/'
+ 'releases/download/2024-05-30/QEMU_EFI.fd'),
+ '937c1e7815e2340150c194a9f8f0474259038a3d7b8845ed62cc08163c46bea1')
+
def wait_for_console_pattern(self, success_message, vm=None):
wait_for_console_pattern(self, success_message,
failure_message='Kernel panic - not syncing',
@@ -21,25 +36,11 @@ def wait_for_console_pattern(self, success_message, vm=None):
def test_loongarch64_devices(self):
- """
- :avocado: tags=arch:loongarch64
- :avocado: tags=machine:virt
- """
-
- kernel_url = ('https://github.com/yangxiaojuan-loongson/qemu-binary/'
- 'releases/download/2024-05-30/vmlinuz.efi')
- kernel_hash = '951b485b16e3788b6db03a3e1793c067009e31a2'
- kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+ self.set_machine('virt')
- initrd_url = ('https://github.com/yangxiaojuan-loongson/qemu-binary/'
- 'releases/download/2024-05-30/ramdisk')
- initrd_hash = 'c67658d9b2a447ce7db2f73ba3d373c9b2b90ab2'
- initrd_path = self.fetch_asset(initrd_url, asset_hash=initrd_hash)
-
- bios_url = ('https://github.com/yangxiaojuan-loongson/qemu-binary/'
- 'releases/download/2024-05-30/QEMU_EFI.fd')
- bios_hash = ('f4d0966b5117d4cd82327c050dd668741046be69')
- bios_path = self.fetch_asset(bios_url, asset_hash=bios_hash)
+ kernel_path = self.ASSET_KERNEL.fetch()
+ initrd_path = self.ASSET_INITRD.fetch()
+ bios_path = self.ASSET_BIOS.fetch()
self.vm.set_console()
kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
@@ -56,3 +57,6 @@ def test_loongarch64_devices(self):
self.wait_for_console_pattern('Run /sbin/init as init process')
exec_command_and_wait_for_pattern(self, 'cat /proc/cpuinfo',
'processor : 3')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/functional/test_mips64el_loongson3v.py b/tests/functional/test_mips64el_loongson3v.py
new file mode 100755
index 0000000000..55d62928c7
--- /dev/null
+++ b/tests/functional/test_mips64el_loongson3v.py
@@ -0,0 +1,39 @@
+#!/usr/bin/env python3
+#
+# Functional tests for the Generic Loongson-3 Platform.
+#
+# Copyright (c) 2021 Jiaxun Yang <jiaxun.yang@flygoat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+import os
+import time
+
+from unittest import skipUnless
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
+
+class MipsLoongson3v(QemuSystemTest):
+ timeout = 60
+
+ ASSET_PMON = Asset(
+ ('https://github.com/loongson-community/pmon/'
+ 'releases/download/20210112/pmon-3avirt.bin'),
+ 'fcdf6bb2cb7885a4a62f31fcb0d5e368bac7b6cea28f40c6dfa678af22fea20a')
+
+ @skipUnless(os.getenv('QEMU_TEST_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
+ def test_pmon_serial_console(self):
+ self.set_machine('loongson3-virt')
+
+ pmon_path = self.ASSET_PMON.fetch()
+
+ self.vm.set_console()
+ self.vm.add_args('-bios', pmon_path)
+ self.vm.launch()
+ wait_for_console_pattern(self, 'CPU GODSON3 BogoMIPS:')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/netdev-ethtool.py b/tests/functional/test_netdev_ethtool.py
old mode 100644
new mode 100755
similarity index 66%
rename from tests/avocado/netdev-ethtool.py
rename to tests/functional/test_netdev_ethtool.py
index 5f33288f81..d5b911c918
--- a/tests/avocado/netdev-ethtool.py
+++ b/tests/functional/test_netdev_ethtool.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# ethtool tests for emulated network devices
#
# This test leverages ethtool's --test sequence to validate network
@@ -5,39 +7,33 @@
#
# SPDX-License-Identifier: GPL-2.0-or-late
-from avocado import skip
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
+from unittest import skip
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
class NetDevEthtool(QemuSystemTest):
- """
- :avocado: tags=arch:x86_64
- :avocado: tags=machine:q35
- """
# Runs in about 17s under KVM, 19s under TCG, 25s under GCOV
timeout = 45
# Fetch assets from the netdev-ethtool subdir of my shared test
# images directory on fileserver.linaro.org.
- def get_asset(self, name, sha1):
- base_url = ('https://fileserver.linaro.org/s/'
- 'kE4nCFLdQcoBF9t/download?'
- 'path=%2Fnetdev-ethtool&files=' )
- url = base_url + name
- # use explicit name rather than failing to neatly parse the
- # URL into a unique one
- return self.fetch_asset(name=name, locations=(url), asset_hash=sha1)
+ ASSET_BASEURL = ('https://fileserver.linaro.org/s/kE4nCFLdQcoBF9t/'
+ 'download?path=%2Fnetdev-ethtool&files=')
+ ASSET_BZIMAGE = Asset(
+ ASSET_BASEURL + "bzImage",
+ "ed62ee06ea620b1035747f3f66a5e9fc5d3096b29f75562ada888b04cd1c4baf")
+ ASSET_ROOTFS = Asset(
+ ASSET_BASEURL + "rootfs.squashfs",
+ "8f0207e3c4d40832ae73c1a927e42ca30ccb1e71f047acb6ddb161ba422934e6")
def common_test_code(self, netdev, extra_args=None):
+ self.set_machine('q35')
# This custom kernel has drivers for all the supported network
# devices we can emulate in QEMU
- kernel = self.get_asset("bzImage",
- "33469d7802732d5815226166581442395cb289e2")
-
- rootfs = self.get_asset("rootfs.squashfs",
- "9793cea7021414ae844bda51f558bd6565b50cdc")
+ kernel = self.ASSET_BZIMAGE.fetch()
+ rootfs = self.ASSET_ROOTFS.fetch()
append = 'printk.time=0 console=ttyS0 '
append += 'root=/dev/sr0 rootfstype=squashfs '
@@ -68,15 +64,9 @@ def common_test_code(self, netdev, extra_args=None):
self.vm.kill()
def test_igb(self):
- """
- :avocado: tags=device:igb
- """
self.common_test_code("igb")
def test_igb_nomsi(self):
- """
- :avocado: tags=device:igb
- """
self.common_test_code("igb", "pci=nomsi")
# It seems the other popular cards we model in QEMU currently fail
@@ -88,14 +78,11 @@ def test_igb_nomsi(self):
@skip("Incomplete reg 0x00178 support")
def test_e1000(self):
- """
- :avocado: tags=device:e1000
- """
self.common_test_code("e1000")
@skip("Incomplete reg 0x00178 support")
def test_i82550(self):
- """
- :avocado: tags=device:i82550
- """
self.common_test_code("i82550")
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/ppc_405.py b/tests/functional/test_ppc_405.py
old mode 100644
new mode 100755
similarity index 53%
rename from tests/avocado/ppc_405.py
rename to tests/functional/test_ppc_405.py
index 4e7e01aa76..9851c03ee9
--- a/tests/avocado/ppc_405.py
+++ b/tests/functional/test_ppc_405.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Test that the U-Boot firmware boots on ppc 405 machines and check the console
#
# Copyright (c) 2021 Red Hat, Inc.
@@ -5,20 +7,21 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado.utils import archive
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
-from avocado_qemu import exec_command_and_wait_for_pattern
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
+from qemu_test import exec_command_and_wait_for_pattern
class Ppc405Machine(QemuSystemTest):
timeout = 90
+ ASSET_UBOOT = Asset(
+ ('https://gitlab.com/huth/u-boot/-/raw/taihu-2021-10-09/'
+ 'u-boot-taihu.bin'),
+ 'a076bb6cdeaafa406330e51e074b66d8878d9036d67d4caa0137be03ee4c112c')
+
def do_test_ppc405(self):
- uboot_url = ('https://gitlab.com/huth/u-boot/-/raw/'
- 'taihu-2021-10-09/u-boot-taihu.bin')
- uboot_hash = ('3208940e908a5edc7c03eab072c60f0dcfadc2ab');
- file_path = self.fetch_asset(uboot_url, asset_hash=uboot_hash)
+ file_path = self.ASSET_UBOOT.fetch()
self.vm.set_console(console_index=1)
self.vm.add_args('-bios', file_path)
self.vm.launch()
@@ -26,11 +29,9 @@ def do_test_ppc405(self):
exec_command_and_wait_for_pattern(self, 'reset', 'AMCC PowerPC 405EP')
def test_ppc_ref405ep(self):
- """
- :avocado: tags=arch:ppc
- :avocado: tags=machine:ref405ep
- :avocado: tags=cpu:405ep
- :avocado: tags=accel:tcg
- """
self.require_accelerator("tcg")
+ self.set_machine('ref405ep')
self.do_test_ppc405()
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH v4 17/35] tests/functional: Add a function for extracting files from an archive
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (15 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 16/35] tests/functional: Convert some tests that download files via fetch_asset() Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 8:27 ` [PATCH v4 18/35] tests/functional: Convert some avocado tests that needed avocado.utils.archive Thomas Huth
` (17 subsequent siblings)
34 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
Some Avocado-based tests use the "archive" module from avocado.utils
to extract files from an archive. To be able to use these tests
without Avocado, we have to provide our own function for extracting
files. Fortunately, there is already the tarfile module that will
provide us with this functionality, so let's just add a nice wrapper
function around that.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/functional/qemu_test/utils.py | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
create mode 100644 tests/functional/qemu_test/utils.py
diff --git a/tests/functional/qemu_test/utils.py b/tests/functional/qemu_test/utils.py
new file mode 100644
index 0000000000..4eb5e5d5e5
--- /dev/null
+++ b/tests/functional/qemu_test/utils.py
@@ -0,0 +1,21 @@
+# Utilities for python-based QEMU tests
+#
+# Copyright 2024 Red Hat, Inc.
+#
+# Authors:
+# Thomas Huth <thuth@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import tarfile
+
+def archive_extract(archive, dest_dir, member=None):
+ with tarfile.open(archive) as tf:
+ if hasattr(tarfile, 'data_filter'):
+ tf.extraction_filter = getattr(tarfile, 'data_filter',
+ (lambda member, path: member))
+ if member:
+ tf.extract(member=member, path=dest_dir)
+ else:
+ tf.extractall(path=dest_dir)
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH v4 18/35] tests/functional: Convert some avocado tests that needed avocado.utils.archive
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (16 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 17/35] tests/functional: Add a function for extracting files from an archive Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 8:27 ` [PATCH v4 19/35] tests/functional: Convert the s390x avocado tests into standalone tests Thomas Huth
` (16 subsequent siblings)
34 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
Instead of using the "archive" module from avocado.utils, switch
these tests to use the new wrapper function that is based on the
"tarfile" module instead.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 6 +--
tests/avocado/machine_sparc64_sun4u.py | 36 ----------------
tests/functional/meson.build | 6 +++
.../test_arm_canona1100.py} | 30 ++++++++------
.../test_ppc_bamboo.py} | 33 +++++++--------
tests/functional/test_sparc64_sun4u.py | 41 +++++++++++++++++++
6 files changed, 84 insertions(+), 68 deletions(-)
delete mode 100644 tests/avocado/machine_sparc64_sun4u.py
rename tests/{avocado/machine_arm_canona1100.py => functional/test_arm_canona1100.py} (51%)
mode change 100644 => 100755
rename tests/{avocado/ppc_bamboo.py => functional/test_ppc_bamboo.py} (60%)
mode change 100644 => 100755
create mode 100755 tests/functional/test_sparc64_sun4u.py
diff --git a/MAINTAINERS b/MAINTAINERS
index 6545662eaf..56dcd56375 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -733,7 +733,7 @@ S: Odd Fixes
F: include/hw/arm/digic.h
F: hw/*/digic*
F: include/hw/*/digic*
-F: tests/avocado/machine_arm_canona1100.py
+F: tests/functional/test_arm_canona1100.py
F: docs/system/arm/digic.rst
Goldfish RTC
@@ -1430,7 +1430,7 @@ L: qemu-ppc@nongnu.org
S: Orphan
F: hw/ppc/ppc440_bamboo.c
F: hw/pci-host/ppc4xx_pci.c
-F: tests/avocado/ppc_bamboo.py
+F: tests/functional/test_ppc_bamboo.py
e500
L: qemu-ppc@nongnu.org
@@ -1716,7 +1716,7 @@ F: include/hw/pci-host/sabre.h
F: hw/pci-bridge/simba.c
F: include/hw/pci-bridge/simba.h
F: pc-bios/openbios-sparc64
-F: tests/avocado/machine_sparc64_sun4u.py
+F: tests/functional/test_sparc64_sun4u.py
Sun4v
M: Artyom Tarasenko <atar4qemu@gmail.com>
diff --git a/tests/avocado/machine_sparc64_sun4u.py b/tests/avocado/machine_sparc64_sun4u.py
deleted file mode 100644
index d333c0ae91..0000000000
--- a/tests/avocado/machine_sparc64_sun4u.py
+++ /dev/null
@@ -1,36 +0,0 @@
-# Functional test that boots a Linux kernel and checks the console
-#
-# Copyright (c) 2020 Red Hat, Inc.
-#
-# Author:
-# Thomas Huth <thuth@redhat.com>
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or
-# later. See the COPYING file in the top-level directory.
-
-import os
-
-from avocado_qemu import wait_for_console_pattern
-from avocado.utils import archive
-from boot_linux_console import LinuxKernelTest
-
-class Sun4uMachine(LinuxKernelTest):
- """Boots the Linux kernel and checks that the console is operational"""
-
- timeout = 90
-
- def test_sparc64_sun4u(self):
- """
- :avocado: tags=arch:sparc64
- :avocado: tags=machine:sun4u
- """
- tar_url = ('https://qemu-advcal.gitlab.io'
- '/qac-best-of-multiarch/download/day23.tar.xz')
- tar_hash = '142db83cd974ffadc4f75c8a5cad5bcc5722c240'
- file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
- archive.extract(file_path, self.workdir)
- self.vm.set_console()
- self.vm.add_args('-kernel', self.workdir + '/day23/vmlinux',
- '-append', self.KERNEL_COMMON_COMMAND_LINE)
- self.vm.launch()
- wait_for_console_pattern(self, 'Starting logging: OK')
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 32c7cc2d3c..91bf80f0f4 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -21,6 +21,7 @@ tests_generic = [
]
tests_arm_thorough = [
+ 'arm_canona1100',
'arm_n8x0',
]
@@ -42,6 +43,11 @@ tests_ppc_quick = [
tests_ppc_thorough = [
'ppc_405',
+ 'ppc_bamboo',
+]
+
+tests_sparc64_thorough = [
+ 'sparc64_sun4u',
]
tests_x86_64_quick = [
diff --git a/tests/avocado/machine_arm_canona1100.py b/tests/functional/test_arm_canona1100.py
old mode 100644
new mode 100755
similarity index 51%
rename from tests/avocado/machine_arm_canona1100.py
rename to tests/functional/test_arm_canona1100.py
index a42d8b0f2b..65f1228296
--- a/tests/avocado/machine_arm_canona1100.py
+++ b/tests/functional/test_arm_canona1100.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Functional test that boots the canon-a1100 machine with firmware
#
# Copyright (c) 2020 Red Hat, Inc.
@@ -8,28 +10,30 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
-from avocado.utils import archive
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
+from qemu_test.utils import archive_extract
class CanonA1100Machine(QemuSystemTest):
"""Boots the barebox firmware and checks that the console is operational"""
timeout = 90
+ ASSET_BIOS = Asset(('https://qemu-advcal.gitlab.io'
+ '/qac-best-of-multiarch/download/day18.tar.xz'),
+ '28e71874ce985be66b7fd1345ed88cb2523b982f899c8d2900d6353054a1be49')
+
def test_arm_canona1100(self):
- """
- :avocado: tags=arch:arm
- :avocado: tags=machine:canon-a1100
- :avocado: tags=device:pflash_cfi02
- """
- tar_url = ('https://qemu-advcal.gitlab.io'
- '/qac-best-of-multiarch/download/day18.tar.xz')
- tar_hash = '068b5fc4242b29381acee94713509f8a876e9db6'
- file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
- archive.extract(file_path, self.workdir)
+ self.set_machine('canon-a1100')
+
+ file_path = self.ASSET_BIOS.fetch()
+ archive_extract(file_path, dest_dir=self.workdir,
+ member="day18/barebox.canon-a1100.bin")
self.vm.set_console()
self.vm.add_args('-bios',
self.workdir + '/day18/barebox.canon-a1100.bin')
self.vm.launch()
wait_for_console_pattern(self, 'running /env/bin/init')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/ppc_bamboo.py b/tests/functional/test_ppc_bamboo.py
old mode 100644
new mode 100755
similarity index 60%
rename from tests/avocado/ppc_bamboo.py
rename to tests/functional/test_ppc_bamboo.py
index a81be3d608..e72cbdee12
--- a/tests/avocado/ppc_bamboo.py
+++ b/tests/functional/test_ppc_bamboo.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Test that Linux kernel boots on the ppc bamboo board and check the console
#
# Copyright (c) 2021 Red Hat
@@ -5,30 +7,26 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado.utils import archive
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
-from avocado_qemu import exec_command_and_wait_for_pattern
+from qemu_test.utils import archive_extract
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
+from qemu_test import exec_command_and_wait_for_pattern
class BambooMachine(QemuSystemTest):
timeout = 90
+ ASSET_IMAGE = Asset(
+ ('http://landley.net/aboriginal/downloads/binaries/'
+ 'system-image-powerpc-440fp.tar.gz'),
+ 'c12b58f841c775a0e6df4832a55afe6b74814d1565d08ddeafc1fb949a075c5e')
+
def test_ppc_bamboo(self):
- """
- :avocado: tags=arch:ppc
- :avocado: tags=machine:bamboo
- :avocado: tags=cpu:440epb
- :avocado: tags=device:rtl8139
- :avocado: tags=accel:tcg
- """
+ self.set_machine('bamboo')
self.require_accelerator("tcg")
self.require_netdev('user')
- tar_url = ('http://landley.net/aboriginal/downloads/binaries/'
- 'system-image-powerpc-440fp.tar.gz')
- tar_hash = '53e5f16414b195b82d2c70272f81c2eedb39bad9'
- file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
- archive.extract(file_path, self.workdir)
+ file_path = self.ASSET_IMAGE.fetch()
+ archive_extract(file_path, self.workdir)
self.vm.set_console()
self.vm.add_args('-kernel', self.workdir +
'/system-image-powerpc-440fp/linux',
@@ -40,3 +38,6 @@ def test_ppc_bamboo(self):
exec_command_and_wait_for_pattern(self, 'ping 10.0.2.2',
'10.0.2.2 is alive!')
exec_command_and_wait_for_pattern(self, 'halt', 'System Halted')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/functional/test_sparc64_sun4u.py b/tests/functional/test_sparc64_sun4u.py
new file mode 100755
index 0000000000..32e245f4ad
--- /dev/null
+++ b/tests/functional/test_sparc64_sun4u.py
@@ -0,0 +1,41 @@
+#!/usr/bin/env python3
+#
+# Functional test that boots a Linux kernel and checks the console
+#
+# Copyright (c) 2020 Red Hat, Inc.
+#
+# Author:
+# Thomas Huth <thuth@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import os
+
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
+from qemu_test.utils import archive_extract
+
+class Sun4uMachine(QemuSystemTest):
+ """Boots the Linux kernel and checks that the console is operational"""
+
+ timeout = 90
+
+ ASSET_IMAGE = Asset(
+ ('https://qemu-advcal.gitlab.io/qac-best-of-multiarch/download/'
+ 'day23.tar.xz'),
+ 'a3ed92450704af244178351afd0e769776e7decb298e95a63abfd9a6e3f6c854')
+
+ def test_sparc64_sun4u(self):
+ self.set_machine('sun4u')
+ file_path = self.ASSET_IMAGE.fetch()
+ kernel_name = 'day23/vmlinux'
+ archive_extract(file_path, self.workdir, kernel_name)
+ self.vm.set_console()
+ self.vm.add_args('-kernel', os.path.join(self.workdir, kernel_name),
+ '-append', 'printk.time=0')
+ self.vm.launch()
+ wait_for_console_pattern(self, 'Starting logging: OK')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH v4 19/35] tests/functional: Convert the s390x avocado tests into standalone tests
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (17 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 18/35] tests/functional: Convert some avocado tests that needed avocado.utils.archive Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 8:27 ` [PATCH v4 20/35] tests/functional: Convert the x86_cpu_model_versions test Thomas Huth
` (15 subsequent siblings)
34 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
These tests use archive.lzma_uncompress() from the Avocado utils,
so provide a small helper function for this, based on the
standard lzma module from Python instead.
And while we're at it, replace the MD5 hashes in the topology test
with proper SHA256 hashes, since MD5 should not be used anymore
nowadays.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 4 +-
tests/functional/meson.build | 6 ++
tests/functional/qemu_test/utils.py | 14 +++
.../test_s390x_ccw_virtio.py} | 79 ++++++++---------
.../test_s390x_topology.py} | 86 ++++++++-----------
5 files changed, 94 insertions(+), 95 deletions(-)
rename tests/{avocado/machine_s390_ccw_virtio.py => functional/test_s390x_ccw_virtio.py} (85%)
mode change 100644 => 100755
rename tests/{avocado/s390_topology.py => functional/test_s390x_topology.py} (88%)
mode change 100644 => 100755
diff --git a/MAINTAINERS b/MAINTAINERS
index 56dcd56375..7fad267307 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1743,7 +1743,7 @@ S: Supported
F: hw/s390x/
F: include/hw/s390x/
F: configs/devices/s390x-softmmu/default.mak
-F: tests/avocado/machine_s390_ccw_virtio.py
+F: tests/functional/test_s390x_ccw_virtio.py
T: git https://github.com/borntraeger/qemu.git s390-next
L: qemu-s390x@nongnu.org
@@ -1806,7 +1806,7 @@ F: hw/s390x/cpu-topology.c
F: target/s390x/kvm/stsi-topology.c
F: docs/devel/s390-cpu-topology.rst
F: docs/system/s390x/cpu-topology.rst
-F: tests/avocado/s390_topology.py
+F: tests/functional/test_s390x_topology.py
X86 Machines
------------
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 91bf80f0f4..ce5e5be2f3 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -12,6 +12,7 @@ endif
# Timeouts for individual tests that can be slow e.g. with debugging enabled
test_timeouts = {
'netdev_ethtool' : 180,
+ 's390x_ccw_virtio' : 180,
}
tests_generic = [
@@ -46,6 +47,11 @@ tests_ppc_thorough = [
'ppc_bamboo',
]
+tests_s390x_thorough = [
+ 's390x_ccw_virtio',
+ 's390x_topology',
+]
+
tests_sparc64_thorough = [
'sparc64_sun4u',
]
diff --git a/tests/functional/qemu_test/utils.py b/tests/functional/qemu_test/utils.py
index 4eb5e5d5e5..a12dac51b6 100644
--- a/tests/functional/qemu_test/utils.py
+++ b/tests/functional/qemu_test/utils.py
@@ -8,6 +8,9 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
+import lzma
+import os
+import shutil
import tarfile
def archive_extract(archive, dest_dir, member=None):
@@ -19,3 +22,14 @@ def archive_extract(archive, dest_dir, member=None):
tf.extract(member=member, path=dest_dir)
else:
tf.extractall(path=dest_dir)
+
+def lzma_uncompress(xz_path, output_path):
+ if os.path.exists(output_path):
+ return
+ with lzma.open(xz_path, 'rb') as lzma_in:
+ try:
+ with open(output_path, 'wb') as raw_out:
+ shutil.copyfileobj(lzma_in, raw_out)
+ except:
+ os.remove(output_path)
+ raise
diff --git a/tests/avocado/machine_s390_ccw_virtio.py b/tests/functional/test_s390x_ccw_virtio.py
old mode 100644
new mode 100755
similarity index 85%
rename from tests/avocado/machine_s390_ccw_virtio.py
rename to tests/functional/test_s390x_ccw_virtio.py
index 7a214110fc..f7acd90a89
--- a/tests/avocado/machine_s390_ccw_virtio.py
+++ b/tests/functional/test_s390x_ccw_virtio.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Functional test that boots an s390x Linux guest with ccw and PCI devices
# attached and checks whether the devices are recognized by Linux
#
@@ -12,17 +14,38 @@
import os
import tempfile
-from avocado import skipUnless
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import exec_command_and_wait_for_pattern
-from avocado_qemu import wait_for_console_pattern
-from avocado.utils import archive
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import exec_command_and_wait_for_pattern
+from qemu_test import wait_for_console_pattern
+from qemu_test.utils import lzma_uncompress
class S390CCWVirtioMachine(QemuSystemTest):
KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
timeout = 120
+ ASSET_BUSTER_KERNEL = Asset(
+ ('https://snapshot.debian.org/archive/debian/'
+ '20201126T092837Z/dists/buster/main/installer-s390x/'
+ '20190702+deb10u6/images/generic/kernel.debian'),
+ 'd411d17c39ae7ad38d27534376cbe88b68b403c325739364122c2e6f1537e818')
+ ASSET_BUSTER_INITRD = Asset(
+ ('https://snapshot.debian.org/archive/debian/'
+ '20201126T092837Z/dists/buster/main/installer-s390x/'
+ '20190702+deb10u6/images/generic/initrd.debian'),
+ '836bbd0fe6a5ca81274c28c2b063ea315ce1868660866e9b60180c575fef9fd5')
+
+ ASSET_F31_KERNEL = Asset(
+ ('https://archives.fedoraproject.org/pub/archive'
+ '/fedora-secondary/releases/31/Server/s390x/os'
+ '/images/kernel.img'),
+ '480859574f3f44caa6cd35c62d70e1ac0609134e22ce2a954bbed9b110c06e0b')
+ ASSET_F31_INITRD = Asset(
+ ('https://archives.fedoraproject.org/pub/archive'
+ '/fedora-secondary/releases/31/Server/s390x/os'
+ '/images/initrd.img'),
+ '04c46095b2c49020b1c2327158898b7db747e4892ae319726192fb949716aa9c')
+
def wait_for_console_pattern(self, success_message, vm=None):
wait_for_console_pattern(self, success_message,
failure_message='Kernel panic - not syncing',
@@ -41,23 +64,10 @@ def clear_guest_dmesg(self):
self.dmesg_clear_count += 1
def test_s390x_devices(self):
+ self.set_machine('s390-ccw-virtio')
- """
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
- """
-
- kernel_url = ('https://snapshot.debian.org/archive/debian/'
- '20201126T092837Z/dists/buster/main/installer-s390x/'
- '20190702+deb10u6/images/generic/kernel.debian')
- kernel_hash = '5821fbee57d6220a067a8b967d24595621aa1eb6'
- kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
-
- initrd_url = ('https://snapshot.debian.org/archive/debian/'
- '20201126T092837Z/dists/buster/main/installer-s390x/'
- '20190702+deb10u6/images/generic/initrd.debian')
- initrd_hash = '81ba09c97bef46e8f4660ac25b4ac0a5be3a94d6'
- initrd_path = self.fetch_asset(initrd_url, asset_hash=initrd_hash)
+ kernel_path = self.ASSET_BUSTER_KERNEL.fetch()
+ initrd_path = self.ASSET_BUSTER_INITRD.fetch()
self.vm.set_console()
kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
@@ -160,29 +170,13 @@ def test_s390x_devices(self):
def test_s390x_fedora(self):
+ self.set_machine('s390-ccw-virtio')
- """
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
- :avocado: tags=device:virtio-gpu
- :avocado: tags=device:virtio-crypto
- :avocado: tags=device:virtio-net
- :avocado: tags=flaky
- """
-
- kernel_url = ('https://archives.fedoraproject.org/pub/archive'
- '/fedora-secondary/releases/31/Server/s390x/os'
- '/images/kernel.img')
- kernel_hash = 'b93d1efcafcf29c1673a4ce371a1f8b43941cfeb'
- kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+ kernel_path = self.ASSET_F31_KERNEL.fetch()
- initrd_url = ('https://archives.fedoraproject.org/pub/archive'
- '/fedora-secondary/releases/31/Server/s390x/os'
- '/images/initrd.img')
- initrd_hash = '3de45d411df5624b8d8ef21cd0b44419ab59b12f'
- initrd_path_xz = self.fetch_asset(initrd_url, asset_hash=initrd_hash)
+ initrd_path_xz = self.ASSET_F31_INITRD.fetch()
initrd_path = os.path.join(self.workdir, 'initrd-raw.img')
- archive.lzma_uncompress(initrd_path_xz, initrd_path)
+ lzma_uncompress(initrd_path_xz, initrd_path)
self.vm.set_console()
kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + ' audit=0 '
@@ -277,3 +271,6 @@ def test_s390x_fedora(self):
exec_command_and_wait_for_pattern(self,
'while ! (dmesg -c | grep Start.virtcrypto_remove) ; do'
' sleep 1 ; done', 'Start virtcrypto_remove.')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/s390_topology.py b/tests/functional/test_s390x_topology.py
old mode 100644
new mode 100755
similarity index 88%
rename from tests/avocado/s390_topology.py
rename to tests/functional/test_s390x_topology.py
index 9154ac8776..20727f6bdf
--- a/tests/avocado/s390_topology.py
+++ b/tests/functional/test_s390x_topology.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Functional test that boots a Linux kernel and checks the console
#
# Copyright IBM Corp. 2023
@@ -9,16 +11,13 @@
# later. See the COPYING file in the top-level directory.
import os
-import shutil
import time
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import exec_command
-from avocado_qemu import exec_command_and_wait_for_pattern
-from avocado_qemu import interrupt_interactive_console_until_pattern
-from avocado_qemu import wait_for_console_pattern
-from avocado.utils import process
-from avocado.utils import archive
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import exec_command
+from qemu_test import exec_command_and_wait_for_pattern
+from qemu_test import wait_for_console_pattern
+from qemu_test.utils import lzma_uncompress
class S390CPUTopology(QemuSystemTest):
@@ -47,6 +46,17 @@ class S390CPUTopology(QemuSystemTest):
'root=/dev/ram '
'selinux=0 '
'rdinit=/bin/sh')
+ ASSET_F35_KERNEL = Asset(
+ ('https://archives.fedoraproject.org/pub/archive'
+ '/fedora-secondary/releases/35/Server/s390x/os'
+ '/images/kernel.img'),
+ '1f2dddfd11bb1393dd2eb2e784036fbf6fc11057a6d7d27f9eb12d3edc67ef73')
+
+ ASSET_F35_INITRD = Asset(
+ ('https://archives.fedoraproject.org/pub/archive'
+ '/fedora-secondary/releases/35/Server/s390x/os'
+ '/images/initrd.img'),
+ '1100145fbca00240c8c372ae4b89b48c99844bc189b3dfbc3f481dc60055ca46')
def wait_until_booted(self):
wait_for_console_pattern(self, 'no job control',
@@ -78,21 +88,10 @@ def kernel_init(self):
We need a minimal root filesystem with a shell.
"""
self.require_accelerator("kvm")
- kernel_url = ('https://archives.fedoraproject.org/pub/archive'
- '/fedora-secondary/releases/35/Server/s390x/os'
- '/images/kernel.img')
- kernel_hash = '0d1aaaf303f07cf0160c8c48e56fe638'
- kernel_path = self.fetch_asset(kernel_url, algorithm='md5',
- asset_hash=kernel_hash)
-
- initrd_url = ('https://archives.fedoraproject.org/pub/archive'
- '/fedora-secondary/releases/35/Server/s390x/os'
- '/images/initrd.img')
- initrd_hash = 'a122057d95725ac030e2ec51df46e172'
- initrd_path_xz = self.fetch_asset(initrd_url, algorithm='md5',
- asset_hash=initrd_hash)
+ kernel_path = self.ASSET_F35_KERNEL.fetch()
+ initrd_path_xz = self.ASSET_F35_INITRD.fetch()
initrd_path = os.path.join(self.workdir, 'initrd-raw.img')
- archive.lzma_uncompress(initrd_path_xz, initrd_path)
+ lzma_uncompress(initrd_path_xz, initrd_path)
self.vm.set_console()
kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE
@@ -115,10 +114,8 @@ def system_init(self):
def test_single(self):
"""
This test checks the simplest topology with a single CPU.
-
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
"""
+ self.set_machine('s390-ccw-virtio')
self.kernel_init()
self.vm.launch()
self.wait_until_booted()
@@ -127,10 +124,8 @@ def test_single(self):
def test_default(self):
"""
This test checks the implicit topology.
-
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
"""
+ self.set_machine('s390-ccw-virtio')
self.kernel_init()
self.vm.add_args('-smp',
'13,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
@@ -154,10 +149,8 @@ def test_move(self):
"""
This test checks the topology modification by moving a CPU
to another socket: CPU 0 is moved from socket 0 to socket 2.
-
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
"""
+ self.set_machine('s390-ccw-virtio')
self.kernel_init()
self.vm.add_args('-smp',
'1,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
@@ -174,10 +167,8 @@ def test_dash_device(self):
"""
This test verifies that a CPU defined with the '-device'
command line option finds its right place inside the topology.
-
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
"""
+ self.set_machine('s390-ccw-virtio')
self.kernel_init()
self.vm.add_args('-smp',
'1,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
@@ -221,10 +212,8 @@ def test_polarization(self):
"""
This test verifies that QEMU modifies the entitlement change after
several guest polarization change requests.
-
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
"""
+ self.set_machine('s390-ccw-virtio')
self.kernel_init()
self.vm.launch()
self.wait_until_booted()
@@ -267,10 +256,8 @@ def test_entitlement(self):
"""
This test verifies that QEMU modifies the entitlement
after a guest request and that the guest sees the change.
-
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
"""
+ self.set_machine('s390-ccw-virtio')
self.kernel_init()
self.vm.launch()
self.wait_until_booted()
@@ -313,10 +300,8 @@ def test_dedicated(self):
CPU is made dedicated.
QEMU retains the entitlement value when horizontal polarization is in effect.
For the guest, the field shows the effective value of the entitlement.
-
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
"""
+ self.set_machine('s390-ccw-virtio')
self.kernel_init()
self.vm.launch()
self.wait_until_booted()
@@ -345,10 +330,8 @@ def test_socket_full(self):
This test verifies that QEMU does not accept to overload a socket.
The socket-id 0 on book-id 0 already contains CPUs 0 and 1 and can
not accept any new CPU while socket-id 0 on book-id 1 is free.
-
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
"""
+ self.set_machine('s390-ccw-virtio')
self.kernel_init()
self.vm.add_args('-smp',
'3,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
@@ -369,10 +352,8 @@ def test_dedicated_error(self):
"""
This test verifies that QEMU refuses to lower the entitlement
of a dedicated CPU
-
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
"""
+ self.set_machine('s390-ccw-virtio')
self.kernel_init()
self.vm.launch()
self.wait_until_booted()
@@ -417,10 +398,8 @@ def test_move_error(self):
"""
This test verifies that QEMU refuses to move a CPU to an
nonexistent location
-
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
"""
+ self.set_machine('s390-ccw-virtio')
self.kernel_init()
self.vm.launch()
self.wait_until_booted()
@@ -437,3 +416,6 @@ def test_move_error(self):
self.assertEqual(res['error']['class'], 'GenericError')
self.check_topology(0, 0, 0, 0, 'medium', False)
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH v4 20/35] tests/functional: Convert the x86_cpu_model_versions test
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (18 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 19/35] tests/functional: Convert the s390x avocado tests into standalone tests Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 8:27 ` [PATCH v4 21/35] tests/functional: Convert the microblaze avocado tests into standalone tests Thomas Huth
` (14 subsequent siblings)
34 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
Nothing thrilling in here, it's just a straight forward conversion.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 1 +
tests/functional/meson.build | 1 +
.../test_x86_cpu_model_versions.py} | 65 ++++++-------------
3 files changed, 21 insertions(+), 46 deletions(-)
rename tests/{avocado/x86_cpu_model_versions.py => functional/test_x86_cpu_model_versions.py} (92%)
mode change 100644 => 100755
diff --git a/MAINTAINERS b/MAINTAINERS
index 7fad267307..bbb8fc78c5 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1836,6 +1836,7 @@ F: tests/unit/test-x86-topo.c
F: tests/qtest/test-x86-cpuid-compat.c
F: tests/functional/test_mem_addr_space.py
F: tests/functional/test_pc_cpu_hotplug_props.py
+F: tests/functional/test_x86_cpu_model_versions.py
PC Chipset
M: Michael S. Tsirkin <mst@redhat.com>
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index ce5e5be2f3..e85d869948 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -61,6 +61,7 @@ tests_x86_64_quick = [
'mem_addr_space',
'pc_cpu_hotplug_props',
'virtio_version',
+ 'x86_cpu_model_versions',
]
tests_x86_64_thorough = [
diff --git a/tests/avocado/x86_cpu_model_versions.py b/tests/functional/test_x86_cpu_model_versions.py
old mode 100644
new mode 100755
similarity index 92%
rename from tests/avocado/x86_cpu_model_versions.py
rename to tests/functional/test_x86_cpu_model_versions.py
index 11101e02b9..bd18acd44f
--- a/tests/avocado/x86_cpu_model_versions.py
+++ b/tests/functional/test_x86_cpu_model_versions.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
#
# Basic validation of x86 versioned CPU models and CPU model aliases
#
@@ -20,15 +21,13 @@
# License along with this library; if not, see <http://www.gnu.org/licenses/>.
#
-
-import avocado_qemu
import re
-class X86CPUModelAliases(avocado_qemu.QemuSystemTest):
+from qemu_test import QemuSystemTest
+
+class X86CPUModelAliases(QemuSystemTest):
"""
Validation of PC CPU model versions and CPU model aliases
-
- :avocado: tags=arch:x86_64
"""
def validate_aliases(self, cpus):
for c in cpus.values():
@@ -76,9 +75,8 @@ def validate_variant_aliases(self, cpus):
def test_4_0_alias_compatibility(self):
"""
Check if pc-*-4.0 unversioned CPU model won't be reported as aliases
-
- :avocado: tags=machine:pc-i440fx-4.0
"""
+ self.set_machine('pc-i440fx-4.0')
# pc-*-4.0 won't expose non-versioned CPU models as aliases
# We do this to help management software to keep compatibility
# with older QEMU versions that didn't have the versioned CPU model
@@ -110,9 +108,8 @@ def test_4_0_alias_compatibility(self):
def test_4_1_alias(self):
"""
Check if unversioned CPU model is an alias pointing to right version
-
- :avocado: tags=machine:pc-i440fx-4.1
"""
+ self.set_machine('pc-i440fx-4.1')
self.vm.add_args('-S')
self.vm.launch()
@@ -217,9 +214,8 @@ def test_4_1_alias(self):
def test_none_alias(self):
"""
Check if unversioned CPU model is an alias pointing to some version
-
- :avocado: tags=machine:none
"""
+ self.set_machine('none')
self.vm.add_args('-S')
self.vm.launch()
@@ -243,21 +239,16 @@ def test_none_alias(self):
self.validate_aliases(cpus)
-class CascadelakeArchCapabilities(avocado_qemu.QemuSystemTest):
+class CascadelakeArchCapabilities(QemuSystemTest):
"""
Validation of Cascadelake arch-capabilities
-
- :avocado: tags=arch:x86_64
"""
def get_cpu_prop(self, prop):
cpu_path = self.vm.cmd('query-cpus-fast')[0].get('qom-path')
return self.vm.cmd('qom-get', path=cpu_path, property=prop)
def test_4_1(self):
- """
- :avocado: tags=machine:pc-i440fx-4.1
- :avocado: tags=cpu:Cascadelake-Server
- """
+ self.set_machine('pc-i440fx-4.1')
# machine-type only:
self.vm.add_args('-S')
self.set_vm_arg('-cpu',
@@ -268,10 +259,7 @@ def test_4_1(self):
'pc-i440fx-4.1 + Cascadelake-Server should not have arch-capabilities')
def test_4_0(self):
- """
- :avocado: tags=machine:pc-i440fx-4.0
- :avocado: tags=cpu:Cascadelake-Server
- """
+ self.set_machine('pc-i440fx-4.0')
self.vm.add_args('-S')
self.set_vm_arg('-cpu',
'Cascadelake-Server,x-force-features=on,check=off,'
@@ -281,10 +269,7 @@ def test_4_0(self):
'pc-i440fx-4.0 + Cascadelake-Server should not have arch-capabilities')
def test_set_4_0(self):
- """
- :avocado: tags=machine:pc-i440fx-4.0
- :avocado: tags=cpu:Cascadelake-Server
- """
+ self.set_machine('pc-i440fx-4.0')
# command line must override machine-type if CPU model is not versioned:
self.vm.add_args('-S')
self.set_vm_arg('-cpu',
@@ -295,10 +280,7 @@ def test_set_4_0(self):
'pc-i440fx-4.0 + Cascadelake-Server,+arch-capabilities should have arch-capabilities')
def test_unset_4_1(self):
- """
- :avocado: tags=machine:pc-i440fx-4.1
- :avocado: tags=cpu:Cascadelake-Server
- """
+ self.set_machine('pc-i440fx-4.1')
self.vm.add_args('-S')
self.set_vm_arg('-cpu',
'Cascadelake-Server,x-force-features=on,check=off,'
@@ -308,10 +290,7 @@ def test_unset_4_1(self):
'pc-i440fx-4.1 + Cascadelake-Server,-arch-capabilities should not have arch-capabilities')
def test_v1_4_0(self):
- """
- :avocado: tags=machine:pc-i440fx-4.0
- :avocado: tags=cpu:Cascadelake-Server
- """
+ self.set_machine('pc-i440fx-4.0')
# versioned CPU model overrides machine-type:
self.vm.add_args('-S')
self.set_vm_arg('-cpu',
@@ -322,10 +301,7 @@ def test_v1_4_0(self):
'pc-i440fx-4.0 + Cascadelake-Server-v1 should not have arch-capabilities')
def test_v2_4_0(self):
- """
- :avocado: tags=machine:pc-i440fx-4.0
- :avocado: tags=cpu:Cascadelake-Server
- """
+ self.set_machine('pc-i440fx-4.0')
self.vm.add_args('-S')
self.set_vm_arg('-cpu',
'Cascadelake-Server-v2,x-force-features=on,check=off,'
@@ -335,10 +311,7 @@ def test_v2_4_0(self):
'pc-i440fx-4.0 + Cascadelake-Server-v2 should have arch-capabilities')
def test_v1_set_4_0(self):
- """
- :avocado: tags=machine:pc-i440fx-4.0
- :avocado: tags=cpu:Cascadelake-Server
- """
+ self.set_machine('pc-i440fx-4.0')
# command line must override machine-type and versioned CPU model:
self.vm.add_args('-S')
self.set_vm_arg('-cpu',
@@ -349,10 +322,7 @@ def test_v1_set_4_0(self):
'pc-i440fx-4.0 + Cascadelake-Server-v1,+arch-capabilities should have arch-capabilities')
def test_v2_unset_4_1(self):
- """
- :avocado: tags=machine:pc-i440fx-4.1
- :avocado: tags=cpu:Cascadelake-Server
- """
+ self.set_machine('pc-i440fx-4.1')
self.vm.add_args('-S')
self.set_vm_arg('-cpu',
'Cascadelake-Server-v2,x-force-features=on,check=off,'
@@ -360,3 +330,6 @@ def test_v2_unset_4_1(self):
self.vm.launch()
self.assertFalse(self.get_cpu_prop('arch-capabilities'),
'pc-i440fx-4.1 + Cascadelake-Server-v2,-arch-capabilities should not have arch-capabilities')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH v4 21/35] tests/functional: Convert the microblaze avocado tests into standalone tests
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (19 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 20/35] tests/functional: Convert the x86_cpu_model_versions test Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 8:27 ` [PATCH v4 22/35] tests/functional: Convert the riscv_opensbi avocado test into a standalone test Thomas Huth
` (13 subsequent siblings)
34 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
The machine_microblaze.py file contained two tests, one for each
endianness. Since we only support one QEMU target binary per file
in the new functional test environment, we have to split this file
up into two files now.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 2 +-
tests/avocado/machine_microblaze.py | 61 -------------------
tests/functional/meson.build | 8 +++
.../functional/test_microblaze_s3adsp1800.py | 40 ++++++++++++
.../test_microblazeel_s3adsp1800.py | 42 +++++++++++++
5 files changed, 91 insertions(+), 62 deletions(-)
delete mode 100644 tests/avocado/machine_microblaze.py
create mode 100755 tests/functional/test_microblaze_s3adsp1800.py
create mode 100755 tests/functional/test_microblazeel_s3adsp1800.py
diff --git a/MAINTAINERS b/MAINTAINERS
index bbb8fc78c5..91f5a0545f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1329,7 +1329,7 @@ M: Edgar E. Iglesias <edgar.iglesias@gmail.com>
S: Maintained
F: hw/microblaze/petalogix_s3adsp1800_mmu.c
F: include/hw/char/xilinx_uartlite.h
-F: tests/avocado/machine_microblaze.py
+F: tests/functional/test_microblaze*.py
petalogix_ml605
M: Edgar E. Iglesias <edgar.iglesias@gmail.com>
diff --git a/tests/avocado/machine_microblaze.py b/tests/avocado/machine_microblaze.py
deleted file mode 100644
index 807709cd11..0000000000
--- a/tests/avocado/machine_microblaze.py
+++ /dev/null
@@ -1,61 +0,0 @@
-# Functional test that boots a microblaze Linux kernel and checks the console
-#
-# Copyright (c) 2018, 2021 Red Hat, Inc.
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or
-# later. See the COPYING file in the top-level directory.
-
-import time
-from avocado_qemu import exec_command, exec_command_and_wait_for_pattern
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
-from avocado.utils import archive
-
-class MicroblazeMachine(QemuSystemTest):
-
- timeout = 90
-
- def test_microblaze_s3adsp1800(self):
- """
- :avocado: tags=arch:microblaze
- :avocado: tags=machine:petalogix-s3adsp1800
- """
-
- tar_url = ('https://qemu-advcal.gitlab.io'
- '/qac-best-of-multiarch/download/day17.tar.xz')
- tar_hash = '08bf3e3bfb6b6c7ce1e54ab65d54e189f2caf13f'
- file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
- archive.extract(file_path, self.workdir)
- self.vm.set_console()
- self.vm.add_args('-kernel', self.workdir + '/day17/ballerina.bin')
- self.vm.launch()
- wait_for_console_pattern(self, 'This architecture does not have '
- 'kernel memory protection')
- # Note:
- # The kernel sometimes gets stuck after the "This architecture ..."
- # message, that's why we don't test for a later string here. This
- # needs some investigation by a microblaze wizard one day...
-
- def test_microblazeel_s3adsp1800(self):
- """
- :avocado: tags=arch:microblazeel
- :avocado: tags=machine:petalogix-s3adsp1800
- """
-
- self.require_netdev('user')
- tar_url = ('http://www.qemu-advent-calendar.org/2023/download/'
- 'day13.tar.gz')
- tar_hash = '6623d5fff5f84cfa8f34e286f32eff6a26546f44'
- file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
- archive.extract(file_path, self.workdir)
- self.vm.set_console()
- self.vm.add_args('-kernel', self.workdir + '/day13/xmaton.bin')
- self.vm.add_args('-nic', 'user,tftp=' + self.workdir + '/day13/')
- self.vm.launch()
- wait_for_console_pattern(self, 'QEMU Advent Calendar 2023')
- time.sleep(0.1)
- exec_command(self, 'root')
- time.sleep(0.1)
- exec_command_and_wait_for_pattern(self,
- 'tftp -g -r xmaton.png 10.0.2.2 ; md5sum xmaton.png',
- '821cd3cab8efd16ad6ee5acc3642a8ea')
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index e85d869948..d6a1a3a116 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -34,6 +34,14 @@ tests_loongarch64_thorough = [
'loongarch64_virt',
]
+tests_microblaze_thorough = [
+ 'microblaze_s3adsp1800'
+]
+
+tests_microblazeel_thorough = [
+ 'microblazeel_s3adsp1800'
+]
+
tests_mips64el_thorough = [
'mips64el_loongson3v',
]
diff --git a/tests/functional/test_microblaze_s3adsp1800.py b/tests/functional/test_microblaze_s3adsp1800.py
new file mode 100755
index 0000000000..4f692ffdb1
--- /dev/null
+++ b/tests/functional/test_microblaze_s3adsp1800.py
@@ -0,0 +1,40 @@
+#!/usr/bin/env python3
+#
+# Functional test that boots a microblaze Linux kernel and checks the console
+#
+# Copyright (c) 2018, 2021 Red Hat, Inc.
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import time
+from qemu_test import exec_command, exec_command_and_wait_for_pattern
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
+from qemu_test.utils import archive_extract
+
+class MicroblazeMachine(QemuSystemTest):
+
+ timeout = 90
+
+ ASSET_IMAGE = Asset(
+ ('https://qemu-advcal.gitlab.io/qac-best-of-multiarch/download/'
+ 'day17.tar.xz'),
+ '3ba7439dfbea7af4876662c97f8e1f0cdad9231fc166e4861d17042489270057')
+
+ def test_microblaze_s3adsp1800(self):
+ self.set_machine('petalogix-s3adsp1800')
+ file_path = self.ASSET_IMAGE.fetch()
+ archive_extract(file_path, self.workdir)
+ self.vm.set_console()
+ self.vm.add_args('-kernel', self.workdir + '/day17/ballerina.bin')
+ self.vm.launch()
+ wait_for_console_pattern(self, 'This architecture does not have '
+ 'kernel memory protection')
+ # Note:
+ # The kernel sometimes gets stuck after the "This architecture ..."
+ # message, that's why we don't test for a later string here. This
+ # needs some investigation by a microblaze wizard one day...
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/functional/test_microblazeel_s3adsp1800.py b/tests/functional/test_microblazeel_s3adsp1800.py
new file mode 100755
index 0000000000..faa3927f2e
--- /dev/null
+++ b/tests/functional/test_microblazeel_s3adsp1800.py
@@ -0,0 +1,42 @@
+#!/usr/bin/env python3
+#
+# Functional test that boots a microblaze Linux kernel and checks the console
+#
+# Copyright (c) 2018, 2021 Red Hat, Inc.
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import time
+from qemu_test import exec_command, exec_command_and_wait_for_pattern
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
+from qemu_test.utils import archive_extract
+
+class MicroblazeelMachine(QemuSystemTest):
+
+ timeout = 90
+
+ ASSET_IMAGE = Asset(
+ ('http://www.qemu-advent-calendar.org/2023/download/day13.tar.gz'),
+ 'b9b3d43c5dd79db88ada495cc6e0d1f591153fe41355e925d791fbf44de50c22')
+
+ def test_microblazeel_s3adsp1800(self):
+ self.require_netdev('user')
+ self.set_machine('petalogix-s3adsp1800')
+ file_path = self.ASSET_IMAGE.fetch()
+ archive_extract(file_path, self.workdir)
+ self.vm.set_console()
+ self.vm.add_args('-kernel', self.workdir + '/day13/xmaton.bin')
+ self.vm.add_args('-nic', 'user,tftp=' + self.workdir + '/day13/')
+ self.vm.launch()
+ wait_for_console_pattern(self, 'QEMU Advent Calendar 2023')
+ time.sleep(0.1)
+ exec_command(self, 'root')
+ time.sleep(0.1)
+ exec_command_and_wait_for_pattern(self,
+ 'tftp -g -r xmaton.png 10.0.2.2 ; md5sum xmaton.png',
+ '821cd3cab8efd16ad6ee5acc3642a8ea')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH v4 22/35] tests/functional: Convert the riscv_opensbi avocado test into a standalone test
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (20 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 21/35] tests/functional: Convert the microblaze avocado tests into standalone tests Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 8:27 ` [PATCH v4 23/35] tests/functional: Convert the virtio_gpu " Thomas Huth
` (12 subsequent siblings)
34 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
The avocado test defined test functions for both, riscv32 and riscv64.
Since we can run the whole file with multiple targets in the new
framework, we can now consolidate the functions so we have to only
define one function per machine now.
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 1 +
tests/avocado/riscv_opensbi.py | 63 --------------------------
tests/functional/meson.build | 8 ++++
tests/functional/test_riscv_opensbi.py | 36 +++++++++++++++
4 files changed, 45 insertions(+), 63 deletions(-)
delete mode 100644 tests/avocado/riscv_opensbi.py
create mode 100755 tests/functional/test_riscv_opensbi.py
diff --git a/MAINTAINERS b/MAINTAINERS
index 91f5a0545f..11cefa015e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -337,6 +337,7 @@ F: hw/intc/riscv*
F: include/hw/riscv/
F: linux-user/host/riscv32/
F: linux-user/host/riscv64/
+F: tests/functional/test_riscv*
F: tests/tcg/riscv64/
RISC-V XThead* extensions
diff --git a/tests/avocado/riscv_opensbi.py b/tests/avocado/riscv_opensbi.py
deleted file mode 100644
index bfff9cc3c3..0000000000
--- a/tests/avocado/riscv_opensbi.py
+++ /dev/null
@@ -1,63 +0,0 @@
-# OpenSBI boot test for RISC-V machines
-#
-# Copyright (c) 2022, Ventana Micro
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or
-# later. See the COPYING file in the top-level directory.
-
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
-
-class RiscvOpenSBI(QemuSystemTest):
- """
- :avocado: tags=accel:tcg
- """
- timeout = 5
-
- def boot_opensbi(self):
- self.vm.set_console()
- self.vm.launch()
- wait_for_console_pattern(self, 'Platform Name')
- wait_for_console_pattern(self, 'Boot HART MEDELEG')
-
- def test_riscv32_spike(self):
- """
- :avocado: tags=arch:riscv32
- :avocado: tags=machine:spike
- """
- self.boot_opensbi()
-
- def test_riscv64_spike(self):
- """
- :avocado: tags=arch:riscv64
- :avocado: tags=machine:spike
- """
- self.boot_opensbi()
-
- def test_riscv32_sifive_u(self):
- """
- :avocado: tags=arch:riscv32
- :avocado: tags=machine:sifive_u
- """
- self.boot_opensbi()
-
- def test_riscv64_sifive_u(self):
- """
- :avocado: tags=arch:riscv64
- :avocado: tags=machine:sifive_u
- """
- self.boot_opensbi()
-
- def test_riscv32_virt(self):
- """
- :avocado: tags=arch:riscv32
- :avocado: tags=machine:virt
- """
- self.boot_opensbi()
-
- def test_riscv64_virt(self):
- """
- :avocado: tags=arch:riscv64
- :avocado: tags=machine:virt
- """
- self.boot_opensbi()
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index d6a1a3a116..55021c4e4d 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -55,6 +55,14 @@ tests_ppc_thorough = [
'ppc_bamboo',
]
+tests_riscv32_quick = [
+ 'riscv_opensbi',
+]
+
+tests_riscv64_quick = [
+ 'riscv_opensbi',
+]
+
tests_s390x_thorough = [
's390x_ccw_virtio',
's390x_topology',
diff --git a/tests/functional/test_riscv_opensbi.py b/tests/functional/test_riscv_opensbi.py
new file mode 100755
index 0000000000..d077e40f42
--- /dev/null
+++ b/tests/functional/test_riscv_opensbi.py
@@ -0,0 +1,36 @@
+#!/usr/bin/env python3
+#
+# OpenSBI boot test for RISC-V machines
+#
+# Copyright (c) 2022, Ventana Micro
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+from qemu_test import QemuSystemTest
+from qemu_test import wait_for_console_pattern
+
+class RiscvOpenSBI(QemuSystemTest):
+
+ timeout = 5
+
+ def boot_opensbi(self):
+ self.vm.set_console()
+ self.vm.launch()
+ wait_for_console_pattern(self, 'Platform Name')
+ wait_for_console_pattern(self, 'Boot HART MEDELEG')
+
+ def test_riscv_spike(self):
+ self.set_machine('spike')
+ self.boot_opensbi()
+
+ def test_riscv_sifive_u(self):
+ self.set_machine('sifive_u')
+ self.boot_opensbi()
+
+ def test_riscv_virt(self):
+ self.set_machine('virt')
+ self.boot_opensbi()
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH v4 23/35] tests/functional: Convert the virtio_gpu avocado test into a standalone test
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (21 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 22/35] tests/functional: Convert the riscv_opensbi avocado test into a standalone test Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 8:27 ` [PATCH v4 24/35] tests/functional: Convert most ppc avocado tests into standalone tests Thomas Huth
` (11 subsequent siblings)
34 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
Nothing thrilling in here, it's just a straight forward conversion.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/functional/meson.build | 1 +
.../test_virtio_gpu.py} | 64 +++++++++----------
2 files changed, 30 insertions(+), 35 deletions(-)
rename tests/{avocado/virtio-gpu.py => functional/test_virtio_gpu.py} (73%)
mode change 100644 => 100755
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 55021c4e4d..8fbd7c4129 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -82,6 +82,7 @@ tests_x86_64_quick = [
tests_x86_64_thorough = [
'netdev_ethtool',
+ 'virtio_gpu',
]
precache_all = []
diff --git a/tests/avocado/virtio-gpu.py b/tests/functional/test_virtio_gpu.py
old mode 100644
new mode 100755
similarity index 73%
rename from tests/avocado/virtio-gpu.py
rename to tests/functional/test_virtio_gpu.py
index 6091f614a4..441cbdcf2d
--- a/tests/avocado/virtio-gpu.py
+++ b/tests/functional/test_virtio_gpu.py
@@ -1,14 +1,16 @@
+#!/usr/bin/env python3
+#
# virtio-gpu tests
#
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado_qemu import BUILD_DIR
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
-from avocado_qemu import exec_command_and_wait_for_pattern
-from avocado_qemu import is_readable_executable_file
+from qemu_test import BUILD_DIR
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
+from qemu_test import exec_command_and_wait_for_pattern
+from qemu_test import is_readable_executable_file
from qemu.utils import kvm_available
@@ -28,25 +30,18 @@ def pick_default_vug_bin():
class VirtioGPUx86(QemuSystemTest):
- """
- :avocado: tags=virtio-gpu
- :avocado: tags=arch:x86_64
- :avocado: tags=cpu:host
- """
KERNEL_COMMAND_LINE = "printk.time=0 console=ttyS0 rdinit=/bin/bash"
- KERNEL_URL = (
- "https://archives.fedoraproject.org/pub/archive/fedora"
- "/linux/releases/33/Everything/x86_64/os/images"
- "/pxeboot/vmlinuz"
- )
- KERNEL_HASH = '1433cfe3f2ffaa44de4ecfb57ec25dc2399cdecf'
- INITRD_URL = (
- "https://archives.fedoraproject.org/pub/archive/fedora"
- "/linux/releases/33/Everything/x86_64/os/images"
- "/pxeboot/initrd.img"
- )
- INITRD_HASH = 'c828d68a027b53e5220536585efe03412332c2d9'
+ ASSET_KERNEL = Asset(
+ ("https://archives.fedoraproject.org/pub/archive/fedora"
+ "/linux/releases/33/Everything/x86_64/os/images"
+ "/pxeboot/vmlinuz"),
+ '2dc5fb5cfe9ac278fa45640f3602d9b7a08cc189ed63fd9b162b07073e4df397')
+ ASSET_INITRD = Asset(
+ ("https://archives.fedoraproject.org/pub/archive/fedora"
+ "/linux/releases/33/Everything/x86_64/os/images"
+ "/pxeboot/initrd.img"),
+ 'c49b97f893a5349e4883452178763e402bdc5caa8845b226a2d1329b5f356045')
def wait_for_console_pattern(self, success_message, vm=None):
wait_for_console_pattern(
@@ -57,16 +52,14 @@ def wait_for_console_pattern(self, success_message, vm=None):
)
def test_virtio_vga_virgl(self):
- """
- :avocado: tags=device:virtio-vga-gl
- """
# FIXME: should check presence of virtio, virgl etc
self.require_accelerator('kvm')
- kernel_path = self.fetch_asset(self.KERNEL_URL, self.KERNEL_HASH)
- initrd_path = self.fetch_asset(self.INITRD_URL, self.INITRD_HASH)
+ kernel_path = self.ASSET_KERNEL.fetch()
+ initrd_path = self.ASSET_INITRD.fetch()
self.vm.set_console()
+ self.vm.add_args("-cpu", "host")
self.vm.add_args("-m", "2G")
self.vm.add_args("-machine", "pc,accel=kvm")
self.vm.add_args("-device", "virtio-vga-gl")
@@ -83,7 +76,7 @@ def test_virtio_vga_virgl(self):
self.vm.launch()
except:
# TODO: probably fails because we are missing the VirGL features
- self.cancel("VirGL not enabled?")
+ self.skipTest("VirGL not enabled?")
self.wait_for_console_pattern("as init process")
exec_command_and_wait_for_pattern(
@@ -92,18 +85,15 @@ def test_virtio_vga_virgl(self):
self.wait_for_console_pattern("features: +virgl +edid")
def test_vhost_user_vga_virgl(self):
- """
- :avocado: tags=device:vhost-user-vga
- """
# FIXME: should check presence of vhost-user-gpu, virgl, memfd etc
self.require_accelerator('kvm')
vug = pick_default_vug_bin()
if not vug:
- self.cancel("Could not find vhost-user-gpu")
+ self.skipTest("Could not find vhost-user-gpu")
- kernel_path = self.fetch_asset(self.KERNEL_URL, self.KERNEL_HASH)
- initrd_path = self.fetch_asset(self.INITRD_URL, self.INITRD_HASH)
+ kernel_path = self.ASSET_KERNEL.fetch()
+ initrd_path = self.ASSET_INITRD.fetch()
# Create socketpair to connect proxy and remote processes
qemu_sock, vug_sock = socket.socketpair(
@@ -129,6 +119,7 @@ def test_vhost_user_vga_virgl(self):
)
self.vm.set_console()
+ self.vm.add_args("-cpu", "host")
self.vm.add_args("-m", "2G")
self.vm.add_args("-object", "memory-backend-memfd,id=mem,size=2G")
self.vm.add_args("-machine", "pc,memory-backend=mem,accel=kvm")
@@ -147,7 +138,7 @@ def test_vhost_user_vga_virgl(self):
self.vm.launch()
except:
# TODO: probably fails because we are missing the VirGL features
- self.cancel("VirGL not enabled?")
+ self.skipTest("VirGL not enabled?")
self.wait_for_console_pattern("as init process")
exec_command_and_wait_for_pattern(self, "/usr/sbin/modprobe virtio_gpu",
"features: +virgl +edid")
@@ -155,3 +146,6 @@ def test_vhost_user_vga_virgl(self):
qemu_sock.close()
vugp.terminate()
vugp.wait()
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH v4 24/35] tests/functional: Convert most ppc avocado tests into standalone tests
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (22 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 23/35] tests/functional: Convert the virtio_gpu " Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 8:27 ` [PATCH v4 25/35] tests/functional: Convert the ppc_amiga avocado test into a standalone test Thomas Huth
` (10 subsequent siblings)
34 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
Nothing thrilling in here, just straight forward conversions.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 9 +--
tests/functional/meson.build | 11 ++++
.../test_ppc64_powernv.py} | 56 +++++-----------
.../test_ppc64_pseries.py} | 56 +++++-----------
.../test_ppc_40p.py} | 65 +++++++++----------
.../test_ppc_mpc8544ds.py} | 29 +++++----
.../test_ppc_virtex_ml507.py} | 29 +++++----
7 files changed, 113 insertions(+), 142 deletions(-)
rename tests/{avocado/ppc_powernv.py => functional/test_ppc64_powernv.py} (71%)
mode change 100644 => 100755
rename tests/{avocado/ppc_pseries.py => functional/test_ppc64_pseries.py} (76%)
mode change 100644 => 100755
rename tests/{avocado/ppc_prep_40p.py => functional/test_ppc_40p.py} (51%)
mode change 100644 => 100755
rename tests/{avocado/ppc_mpc8544ds.py => functional/test_ppc_mpc8544ds.py} (55%)
mode change 100644 => 100755
rename tests/{avocado/ppc_virtex_ml507.py => functional/test_ppc_virtex_ml507.py} (60%)
mode change 100644 => 100755
diff --git a/MAINTAINERS b/MAINTAINERS
index 11cefa015e..26c9fba7c5 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1454,7 +1454,7 @@ L: qemu-ppc@nongnu.org
S: Orphan
F: hw/ppc/mpc8544ds.c
F: hw/ppc/mpc8544_guts.c
-F: tests/avocado/ppc_mpc8544ds.py
+F: tests/functional/test_ppc_mpc8544ds.py
New World (mac99)
M: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
@@ -1507,7 +1507,7 @@ F: hw/dma/i82374.c
F: hw/rtc/m48t59-isa.c
F: include/hw/isa/pc87312.h
F: include/hw/rtc/m48t59.h
-F: tests/avocado/ppc_prep_40p.py
+F: tests/functional/test_ppc_40p.py
sPAPR (pseries)
M: Nicholas Piggin <npiggin@gmail.com>
@@ -1531,7 +1531,7 @@ F: tests/qtest/spapr*
F: tests/qtest/libqos/*spapr*
F: tests/qtest/rtas*
F: tests/qtest/libqos/rtas*
-F: tests/avocado/ppc_pseries.py
+F: tests/functional/test_ppc64_pseries.py
F: tests/avocado/ppc_hv_tests.py
PowerNV (Non-Virtualized)
@@ -1549,6 +1549,7 @@ F: include/hw/ppc/pnv*
F: include/hw/pci-host/pnv*
F: pc-bios/skiboot.lid
F: tests/qtest/pnv*
+F: tests/functional/test_ppc64_powernv.py
pca955x
M: Glenn Miles <milesg@linux.ibm.com>
@@ -1563,7 +1564,7 @@ M: Edgar E. Iglesias <edgar.iglesias@gmail.com>
L: qemu-ppc@nongnu.org
S: Odd Fixes
F: hw/ppc/virtex_ml507.c
-F: tests/avocado/ppc_virtex_ml507.py
+F: tests/functional/test_ppc_virtex_ml507.py
sam460ex
M: BALATON Zoltan <balaton@eik.bme.hu>
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 8fbd7c4129..b4f5c9e38e 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -12,6 +12,9 @@ endif
# Timeouts for individual tests that can be slow e.g. with debugging enabled
test_timeouts = {
'netdev_ethtool' : 180,
+ 'ppc_40p' : 240,
+ 'ppc64_powernv' : 120,
+ 'ppc64_pseries' : 120,
's390x_ccw_virtio' : 180,
}
@@ -52,7 +55,15 @@ tests_ppc_quick = [
tests_ppc_thorough = [
'ppc_405',
+ 'ppc_40p',
'ppc_bamboo',
+ 'ppc_mpc8544ds',
+ 'ppc_virtex_ml507',
+]
+
+tests_ppc64_thorough = [
+ 'ppc64_powernv',
+ 'ppc64_pseries',
]
tests_riscv32_quick = [
diff --git a/tests/avocado/ppc_powernv.py b/tests/functional/test_ppc64_powernv.py
old mode 100644
new mode 100755
similarity index 71%
rename from tests/avocado/ppc_powernv.py
rename to tests/functional/test_ppc64_powernv.py
index 4342941d5d..67497d6404
--- a/tests/avocado/ppc_powernv.py
+++ b/tests/functional/test_ppc64_powernv.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Test that Linux kernel boots on ppc powernv machines and check the console
#
# Copyright (c) 2018, 2020 Red Hat, Inc.
@@ -5,9 +7,8 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado.utils import archive
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
class powernvMachine(QemuSystemTest):
@@ -16,13 +17,14 @@ class powernvMachine(QemuSystemTest):
panic_message = 'Kernel panic - not syncing'
good_message = 'VFS: Cannot open root device'
+ ASSET_KERNEL = Asset(
+ ('https://archives.fedoraproject.org/pub/archive/fedora-secondary/'
+ 'releases/29/Everything/ppc64le/os/ppc/ppc64/vmlinuz'),
+ '383c2f5c23bc0d9d32680c3924d3fd7ee25cc5ef97091ac1aa5e1d853422fc5f')
+
def do_test_linux_boot(self, command_line = KERNEL_COMMON_COMMAND_LINE):
self.require_accelerator("tcg")
- kernel_url = ('https://archives.fedoraproject.org/pub/archive'
- '/fedora-secondary/releases/29/Everything/ppc64le/os'
- '/ppc/ppc64/vmlinuz')
- kernel_hash = '3fe04abfc852b66653b8c3c897a59a689270bc77'
- kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+ kernel_path = self.ASSET_KERNEL.fetch()
self.vm.set_console()
self.vm.add_args('-kernel', kernel_path,
@@ -30,23 +32,13 @@ def do_test_linux_boot(self, command_line = KERNEL_COMMON_COMMAND_LINE):
self.vm.launch()
def test_linux_boot(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:powernv
- :avocado: tags=accel:tcg
- """
-
+ self.set_machine('powernv')
self.do_test_linux_boot()
console_pattern = 'VFS: Cannot open root device'
wait_for_console_pattern(self, console_pattern, self.panic_message)
def test_linux_smp_boot(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:powernv
- :avocado: tags=accel:tcg
- """
-
+ self.set_machine('powernv')
self.vm.add_args('-smp', '4')
self.do_test_linux_boot()
console_pattern = 'smp: Brought up 1 node, 4 CPUs'
@@ -54,12 +46,7 @@ def test_linux_smp_boot(self):
wait_for_console_pattern(self, self.good_message, self.panic_message)
def test_linux_smp_hpt_boot(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:powernv
- :avocado: tags=accel:tcg
- """
-
+ self.set_machine('powernv')
self.vm.add_args('-smp', '4')
self.do_test_linux_boot(self.KERNEL_COMMON_COMMAND_LINE +
'disable_radix')
@@ -70,12 +57,7 @@ def test_linux_smp_hpt_boot(self):
wait_for_console_pattern(self, self.good_message, self.panic_message)
def test_linux_smt_boot(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:powernv
- :avocado: tags=accel:tcg
- """
-
+ self.set_machine('powernv')
self.vm.add_args('-smp', '4,threads=4')
self.do_test_linux_boot()
console_pattern = 'CPU maps initialized for 4 threads per core'
@@ -85,12 +67,7 @@ def test_linux_smt_boot(self):
wait_for_console_pattern(self, self.good_message, self.panic_message)
def test_linux_big_boot(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:powernv
- :avocado: tags=accel:tcg
- """
-
+ self.set_machine('powernv')
self.vm.add_args('-smp', '16,threads=4,cores=2,sockets=2')
# powernv does not support NUMA
@@ -100,3 +77,6 @@ def test_linux_big_boot(self):
console_pattern = 'smp: Brought up 2 nodes, 16 CPUs'
wait_for_console_pattern(self, console_pattern, self.panic_message)
wait_for_console_pattern(self, self.good_message, self.panic_message)
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/ppc_pseries.py b/tests/functional/test_ppc64_pseries.py
old mode 100644
new mode 100755
similarity index 76%
rename from tests/avocado/ppc_pseries.py
rename to tests/functional/test_ppc64_pseries.py
index 74aaa4ac4a..fdc404ed03
--- a/tests/avocado/ppc_pseries.py
+++ b/tests/functional/test_ppc64_pseries.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Test that Linux kernel boots on ppc machines and check the console
#
# Copyright (c) 2018, 2020 Red Hat, Inc.
@@ -5,9 +7,8 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado.utils import archive
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
class pseriesMachine(QemuSystemTest):
@@ -16,12 +17,13 @@ class pseriesMachine(QemuSystemTest):
panic_message = 'Kernel panic - not syncing'
good_message = 'VFS: Cannot open root device'
+ ASSET_KERNEL = Asset(
+ ('https://archives.fedoraproject.org/pub/archive/fedora-secondary/'
+ 'releases/29/Everything/ppc64le/os/ppc/ppc64/vmlinuz'),
+ '383c2f5c23bc0d9d32680c3924d3fd7ee25cc5ef97091ac1aa5e1d853422fc5f')
+
def do_test_ppc64_linux_boot(self, kernel_command_line = KERNEL_COMMON_COMMAND_LINE):
- kernel_url = ('https://archives.fedoraproject.org/pub/archive'
- '/fedora-secondary/releases/29/Everything/ppc64le/os'
- '/ppc/ppc64/vmlinuz')
- kernel_hash = '3fe04abfc852b66653b8c3c897a59a689270bc77'
- kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+ kernel_path = self.ASSET_KERNEL.fetch()
self.vm.set_console()
self.vm.add_args('-kernel', kernel_path,
@@ -29,32 +31,20 @@ def do_test_ppc64_linux_boot(self, kernel_command_line = KERNEL_COMMON_COMMAND_L
self.vm.launch()
def test_ppc64_vof_linux_boot(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:pseries
- """
-
+ self.set_machine('pseries')
self.vm.add_args('-machine', 'x-vof=on')
self.do_test_ppc64_linux_boot()
console_pattern = 'VFS: Cannot open root device'
wait_for_console_pattern(self, console_pattern, self.panic_message)
def test_ppc64_linux_boot(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:pseries
- """
-
+ self.set_machine('pseries')
self.do_test_ppc64_linux_boot()
console_pattern = 'VFS: Cannot open root device'
wait_for_console_pattern(self, console_pattern, self.panic_message)
def test_ppc64_linux_smp_boot(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:pseries
- """
-
+ self.set_machine('pseries')
self.vm.add_args('-smp', '4')
self.do_test_ppc64_linux_boot()
console_pattern = 'smp: Brought up 1 node, 4 CPUs'
@@ -62,11 +52,7 @@ def test_ppc64_linux_smp_boot(self):
wait_for_console_pattern(self, self.good_message, self.panic_message)
def test_ppc64_linux_hpt_smp_boot(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:pseries
- """
-
+ self.set_machine('pseries')
self.vm.add_args('-smp', '4')
self.do_test_ppc64_linux_boot(self.KERNEL_COMMON_COMMAND_LINE +
'disable_radix')
@@ -77,11 +63,6 @@ def test_ppc64_linux_hpt_smp_boot(self):
wait_for_console_pattern(self, self.good_message, self.panic_message)
def test_ppc64_linux_smt_boot(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:pseries
- """
-
self.vm.add_args('-smp', '4,threads=4')
self.do_test_ppc64_linux_boot()
console_pattern = 'CPU maps initialized for 4 threads per core'
@@ -91,11 +72,7 @@ def test_ppc64_linux_smt_boot(self):
wait_for_console_pattern(self, self.good_message, self.panic_message)
def test_ppc64_linux_big_boot(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:pseries
- """
-
+ self.set_machine('pseries')
self.vm.add_args('-smp', '16,threads=4,cores=2,sockets=2')
self.vm.add_args('-m', '512M',
'-object', 'memory-backend-ram,size=256M,id=m0',
@@ -108,3 +85,6 @@ def test_ppc64_linux_big_boot(self):
console_pattern = 'smp: Brought up 2 nodes, 16 CPUs'
wait_for_console_pattern(self, console_pattern, self.panic_message)
wait_for_console_pattern(self, self.good_message, self.panic_message)
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/ppc_prep_40p.py b/tests/functional/test_ppc_40p.py
old mode 100644
new mode 100755
similarity index 51%
rename from tests/avocado/ppc_prep_40p.py
rename to tests/functional/test_ppc_40p.py
index d4f1eb7e1d..c64e876c1f
--- a/tests/avocado/ppc_prep_40p.py
+++ b/tests/functional/test_ppc_40p.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Functional test that boots a PReP/40p machine and checks its serial console.
#
# Copyright (c) Philippe Mathieu-Daudé <f4bug@amsat.org>
@@ -7,39 +9,40 @@
import os
-from avocado import skipUnless
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
+from unittest import skipUnless
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
class IbmPrep40pMachine(QemuSystemTest):
timeout = 60
+ ASSET_BIOS = Asset(
+ ('http://ftpmirror.your.org/pub/misc/'
+ 'ftp.software.ibm.com/rs6000/firmware/'
+ '7020-40p/P12H0456.IMG'),
+ 'd957f79c73f760d1455d2286fcd901ed6d06167320eb73511b478a939be25b3f')
+ ASSET_NETBSD40 = Asset(
+ ('https://archive.netbsd.org/pub/NetBSD-archive/'
+ 'NetBSD-4.0/prep/installation/floppy/generic_com0.fs'),
+ 'f86236e9d01b3f0dd0f5d3b8d5bbd40c68e78b4db560a108358f5ad58e636619')
+ ASSET_NETBSD71 = Asset(
+ ('https://archive.netbsd.org/pub/NetBSD-archive/'
+ 'NetBSD-7.1.2/iso/NetBSD-7.1.2-prep.iso'),
+ 'cc7cb290b06aaa839362deb7bd9f417ac5015557db24088508330f76c3f825ec')
+
# 12H0455 PPS Firmware Licensed Materials
# Property of IBM (C) Copyright IBM Corp. 1994.
# All rights reserved.
# U.S. Government Users Restricted Rights - Use, duplication or disclosure
# restricted by GSA ADP Schedule Contract with IBM Corp.
- @skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
+ @skipUnless(os.getenv('QEMU_TEST_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
def test_factory_firmware_and_netbsd(self):
- """
- :avocado: tags=arch:ppc
- :avocado: tags=machine:40p
- :avocado: tags=os:netbsd
- :avocado: tags=slowness:high
- :avocado: tags=accel:tcg
- """
+ self.set_machine('40p')
self.require_accelerator("tcg")
- bios_url = ('http://ftpmirror.your.org/pub/misc/'
- 'ftp.software.ibm.com/rs6000/firmware/'
- '7020-40p/P12H0456.IMG')
- bios_hash = '1775face4e6dc27f3a6ed955ef6eb331bf817f03'
- bios_path = self.fetch_asset(bios_url, asset_hash=bios_hash)
- drive_url = ('https://archive.netbsd.org/pub/NetBSD-archive/'
- 'NetBSD-4.0/prep/installation/floppy/generic_com0.fs')
- drive_hash = 'dbcfc09912e71bd5f0d82c7c1ee43082fb596ceb'
- drive_path = self.fetch_asset(drive_url, asset_hash=drive_hash)
+ bios_path = self.ASSET_BIOS.fetch()
+ drive_path = self.ASSET_NETBSD40.fetch()
self.vm.set_console()
self.vm.add_args('-bios', bios_path,
@@ -50,11 +53,7 @@ def test_factory_firmware_and_netbsd(self):
wait_for_console_pattern(self, 'Model: IBM PPS Model 6015')
def test_openbios_192m(self):
- """
- :avocado: tags=arch:ppc
- :avocado: tags=machine:40p
- :avocado: tags=accel:tcg
- """
+ self.set_machine('40p')
self.require_accelerator("tcg")
self.vm.set_console()
self.vm.add_args('-m', '192') # test fw_cfg
@@ -65,21 +64,15 @@ def test_openbios_192m(self):
wait_for_console_pattern(self, '>> CPU type PowerPC,604')
def test_openbios_and_netbsd(self):
- """
- :avocado: tags=arch:ppc
- :avocado: tags=machine:40p
- :avocado: tags=os:netbsd
- :avocado: tags=accel:tcg
- """
+ self.set_machine('40p')
self.require_accelerator("tcg")
- drive_url = ('https://archive.netbsd.org/pub/NetBSD-archive/'
- 'NetBSD-7.1.2/iso/NetBSD-7.1.2-prep.iso')
- drive_hash = 'ac6fa2707d888b36d6fa64de6e7fe48e'
- drive_path = self.fetch_asset(drive_url, asset_hash=drive_hash,
- algorithm='md5')
+ drive_path = self.ASSET_NETBSD71.fetch()
self.vm.set_console()
self.vm.add_args('-cdrom', drive_path,
'-boot', 'd')
self.vm.launch()
wait_for_console_pattern(self, 'NetBSD/prep BOOT, Revision 1.9')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/ppc_mpc8544ds.py b/tests/functional/test_ppc_mpc8544ds.py
old mode 100644
new mode 100755
similarity index 55%
rename from tests/avocado/ppc_mpc8544ds.py
rename to tests/functional/test_ppc_mpc8544ds.py
index b599fb1cc9..2b3f0894ae
--- a/tests/avocado/ppc_mpc8544ds.py
+++ b/tests/functional/test_ppc_mpc8544ds.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Test that Linux kernel boots on ppc machines and check the console
#
# Copyright (c) 2018, 2020 Red Hat, Inc.
@@ -5,9 +7,9 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado.utils import archive
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
+from qemu_test.utils import archive_extract
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
class Mpc8544dsMachine(QemuSystemTest):
@@ -15,20 +17,21 @@ class Mpc8544dsMachine(QemuSystemTest):
KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
panic_message = 'Kernel panic - not syncing'
+ ASSET_IMAGE = Asset(
+ ('https://qemu-advcal.gitlab.io/qac-best-of-multiarch/download/'
+ 'day04.tar.xz'),
+ '88bc83f3c9f3d633bcfc108a6342d677abca247066a2fb8d4636744a0d319f94')
+
def test_ppc_mpc8544ds(self):
- """
- :avocado: tags=arch:ppc
- :avocado: tags=machine:mpc8544ds
- :avocado: tags=accel:tcg
- """
self.require_accelerator("tcg")
- tar_url = ('https://qemu-advcal.gitlab.io'
- '/qac-best-of-multiarch/download/day04.tar.xz')
- tar_hash = 'f46724d281a9f30fa892d458be7beb7d34dc25f9'
- file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
- archive.extract(file_path, self.workdir)
+ self.set_machine('mpc8544ds')
+ file_path = self.ASSET_IMAGE.fetch()
+ archive_extract(file_path, self.workdir, member='creek/creek.bin')
self.vm.set_console()
self.vm.add_args('-kernel', self.workdir + '/creek/creek.bin')
self.vm.launch()
wait_for_console_pattern(self, 'QEMU advent calendar 2020',
self.panic_message)
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/ppc_virtex_ml507.py b/tests/functional/test_ppc_virtex_ml507.py
old mode 100644
new mode 100755
similarity index 60%
rename from tests/avocado/ppc_virtex_ml507.py
rename to tests/functional/test_ppc_virtex_ml507.py
index a73f8ae396..ffa9a0633e
--- a/tests/avocado/ppc_virtex_ml507.py
+++ b/tests/functional/test_ppc_virtex_ml507.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Test that Linux kernel boots on ppc machines and check the console
#
# Copyright (c) 2018, 2020 Red Hat, Inc.
@@ -5,9 +7,9 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado.utils import archive
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
+from qemu_test.utils import archive_extract
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
class VirtexMl507Machine(QemuSystemTest):
@@ -15,18 +17,16 @@ class VirtexMl507Machine(QemuSystemTest):
KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
panic_message = 'Kernel panic - not syncing'
+ ASSET_IMAGE = Asset(
+ ('https://qemu-advcal.gitlab.io/qac-best-of-multiarch/download/'
+ 'day08.tar.xz'),
+ 'cefe5b8aeb5e9d2d1d4fd22dcf48d917d68d5a765132bf2ddd6332dc393b824c')
+
def test_ppc_virtex_ml507(self):
- """
- :avocado: tags=arch:ppc
- :avocado: tags=machine:virtex-ml507
- :avocado: tags=accel:tcg
- """
self.require_accelerator("tcg")
- tar_url = ('https://qemu-advcal.gitlab.io'
- '/qac-best-of-multiarch/download/day08.tar.xz')
- tar_hash = '74c68f5af7a7b8f21c03097b298f3bb77ff52c1f'
- file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
- archive.extract(file_path, self.workdir)
+ self.set_machine('virtex-ml507')
+ file_path = self.ASSET_IMAGE.fetch()
+ archive_extract(file_path, self.workdir)
self.vm.set_console()
self.vm.add_args('-kernel', self.workdir + '/hippo/hippo.linux',
'-dtb', self.workdir + '/hippo/virtex440-ml507.dtb',
@@ -34,3 +34,6 @@ def test_ppc_virtex_ml507(self):
self.vm.launch()
wait_for_console_pattern(self, 'QEMU advent calendar 2020',
self.panic_message)
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH v4 25/35] tests/functional: Convert the ppc_amiga avocado test into a standalone test
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (23 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 24/35] tests/functional: Convert most ppc avocado tests into standalone tests Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 8:27 ` [PATCH v4 26/35] tests/functional: Convert the ppc_hv " Thomas Huth
` (9 subsequent siblings)
34 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
Use the Python standard zipfile module instead of avocado.utils for
extracting the ZIP file that we download here, and use the standard
subprocess module for running the "tail" command.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 1 +
tests/avocado/ppc_amiga.py | 38 --------------------------
tests/functional/meson.build | 1 +
tests/functional/test_ppc_amiga.py | 43 ++++++++++++++++++++++++++++++
4 files changed, 45 insertions(+), 38 deletions(-)
delete mode 100644 tests/avocado/ppc_amiga.py
create mode 100755 tests/functional/test_ppc_amiga.py
diff --git a/MAINTAINERS b/MAINTAINERS
index 26c9fba7c5..f01e865025 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1597,6 +1597,7 @@ S: Maintained
F: hw/ppc/amigaone.c
F: hw/pci-host/articia.c
F: include/hw/pci-host/articia.h
+F: tests/functional/test_ppc_amiga.py
Virtual Open Firmware (VOF)
M: Alexey Kardashevskiy <aik@ozlabs.ru>
diff --git a/tests/avocado/ppc_amiga.py b/tests/avocado/ppc_amiga.py
deleted file mode 100644
index b6f866f91d..0000000000
--- a/tests/avocado/ppc_amiga.py
+++ /dev/null
@@ -1,38 +0,0 @@
-# Test AmigaNG boards
-#
-# Copyright (c) 2023 BALATON Zoltan
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or
-# later. See the COPYING file in the top-level directory.
-
-from avocado.utils import archive
-from avocado.utils import process
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
-
-class AmigaOneMachine(QemuSystemTest):
-
- timeout = 90
-
- def test_ppc_amigaone(self):
- """
- :avocado: tags=arch:ppc
- :avocado: tags=machine:amigaone
- :avocado: tags=device:articia
- :avocado: tags=accel:tcg
- """
- self.require_accelerator("tcg")
- tar_name = 'A1Firmware_Floppy_05-Mar-2005.zip'
- tar_url = ('https://www.hyperion-entertainment.com/index.php/'
- 'downloads?view=download&format=raw&file=25')
- tar_hash = 'c52e59bc73e31d8bcc3cc2106778f7ac84f6c755'
- zip_file = self.fetch_asset(tar_name, locations=tar_url,
- asset_hash=tar_hash)
- archive.extract(zip_file, self.workdir)
- cmd = f"tail -c 524288 {self.workdir}/floppy_edition/updater.image >{self.workdir}/u-boot-amigaone.bin"
- process.run(cmd, shell=True)
-
- self.vm.set_console()
- self.vm.add_args('-bios', self.workdir + '/u-boot-amigaone.bin')
- self.vm.launch()
- wait_for_console_pattern(self, 'FLASH:')
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index b4f5c9e38e..fba3891e16 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -56,6 +56,7 @@ tests_ppc_quick = [
tests_ppc_thorough = [
'ppc_405',
'ppc_40p',
+ 'ppc_amiga',
'ppc_bamboo',
'ppc_mpc8544ds',
'ppc_virtex_ml507',
diff --git a/tests/functional/test_ppc_amiga.py b/tests/functional/test_ppc_amiga.py
new file mode 100755
index 0000000000..b793b5c432
--- /dev/null
+++ b/tests/functional/test_ppc_amiga.py
@@ -0,0 +1,43 @@
+#!/usr/bin/env python3
+#
+# Test AmigaNG boards
+#
+# Copyright (c) 2023 BALATON Zoltan
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import subprocess
+
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern, run_cmd
+from zipfile import ZipFile
+
+class AmigaOneMachine(QemuSystemTest):
+
+ timeout = 90
+
+ ASSET_IMAGE = Asset(
+ ('https://www.hyperion-entertainment.com/index.php/'
+ 'downloads?view=download&format=raw&file=25'),
+ '8ff39330ba47d4f64de4ee8fd6809e9c010a9ef17fe51e95c3c1d53437cb481f')
+
+ def test_ppc_amigaone(self):
+ self.require_accelerator("tcg")
+ self.set_machine('amigaone')
+ tar_name = 'A1Firmware_Floppy_05-Mar-2005.zip'
+ zip_file = self.ASSET_IMAGE.fetch()
+ with ZipFile(zip_file, 'r') as zf:
+ zf.extractall(path=self.workdir)
+ bios_fh = open(self.workdir + "/u-boot-amigaone.bin", "wb")
+ subprocess.run(['tail', '-c', '524288',
+ self.workdir + "/floppy_edition/updater.image"],
+ stdout=bios_fh)
+
+ self.vm.set_console()
+ self.vm.add_args('-bios', self.workdir + '/u-boot-amigaone.bin')
+ self.vm.launch()
+ wait_for_console_pattern(self, 'FLASH:')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH v4 26/35] tests/functional: Convert the ppc_hv avocado test into a standalone test
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (24 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 25/35] tests/functional: Convert the ppc_amiga avocado test into a standalone test Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 9:43 ` Philippe Mathieu-Daudé
2024-08-21 11:47 ` Philippe Mathieu-Daudé
2024-08-21 8:27 ` [PATCH v4 27/35] tests/functional: Convert the m68k nextcube test with tesseract Thomas Huth
` (8 subsequent siblings)
34 siblings, 2 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
A straight forward conversion, we just also have to remove the decorator
@skipUnless(os.getenv('SPEED')) since all non-trivial functional tests
are running in SPEED=thorough mode now. Also make sure that the extracted
assets are writable, so that the test does not fail if it gets re-run
and there are stale read-only files already around.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 2 +-
tests/functional/meson.build | 2 +
.../test_ppc64_hv.py} | 53 ++++++++-----------
3 files changed, 25 insertions(+), 32 deletions(-)
rename tests/{avocado/ppc_hv_tests.py => functional/test_ppc64_hv.py} (85%)
mode change 100644 => 100755
diff --git a/MAINTAINERS b/MAINTAINERS
index f01e865025..cca48683ce 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1532,7 +1532,7 @@ F: tests/qtest/libqos/*spapr*
F: tests/qtest/rtas*
F: tests/qtest/libqos/rtas*
F: tests/functional/test_ppc64_pseries.py
-F: tests/avocado/ppc_hv_tests.py
+F: tests/functional/test_ppc64_hv.py
PowerNV (Non-Virtualized)
M: Cédric Le Goater <clg@kaod.org>
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index fba3891e16..f90fb08d6a 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -13,6 +13,7 @@ endif
test_timeouts = {
'netdev_ethtool' : 180,
'ppc_40p' : 240,
+ 'ppc64_hv' : 1000,
'ppc64_powernv' : 120,
'ppc64_pseries' : 120,
's390x_ccw_virtio' : 180,
@@ -63,6 +64,7 @@ tests_ppc_thorough = [
]
tests_ppc64_thorough = [
+ 'ppc64_hv',
'ppc64_powernv',
'ppc64_pseries',
]
diff --git a/tests/avocado/ppc_hv_tests.py b/tests/functional/test_ppc64_hv.py
old mode 100644
new mode 100755
similarity index 85%
rename from tests/avocado/ppc_hv_tests.py
rename to tests/functional/test_ppc64_hv.py
index 0e83bbac71..df59667e07
--- a/tests/avocado/ppc_hv_tests.py
+++ b/tests/functional/test_ppc64_hv.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Tests that specifically try to exercise hypervisor features of the
# target machines. powernv supports the Power hypervisor ISA, and
# pseries supports the nested-HV hypervisor spec.
@@ -7,10 +9,10 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado import skipIf, skipUnless
-from avocado.utils import archive
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern, exec_command
+from unittest import skipIf, skipUnless
+from qemu_test.utils import archive_extract
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern, exec_command
import os
import time
import subprocess
@@ -45,8 +47,7 @@ def missing_deps():
# QEMU already installed and use that.
# XXX: The order of these tests seems to matter, see git blame.
@skipIf(missing_deps(), 'dependencies (%s) not installed' % ','.join(deps))
-@skipUnless(os.getenv('AVOCADO_ALLOW_LARGE_STORAGE'), 'storage limited')
-@skipUnless(os.getenv('SPEED') == 'slow', 'runtime limited')
+@skipUnless(os.getenv('QEMU_TEST_ALLOW_LARGE_STORAGE'), 'storage limited')
class HypervisorTest(QemuSystemTest):
timeout = 1000
@@ -54,6 +55,11 @@ class HypervisorTest(QemuSystemTest):
panic_message = 'Kernel panic - not syncing'
good_message = 'VFS: Cannot open root device'
+ ASSET_ISO = Asset(
+ ('https://dl-cdn.alpinelinux.org/alpine/v3.18/'
+ 'releases/ppc64le/alpine-standard-3.18.4-ppc64le.iso'),
+ 'c26b8d3e17c2f3f0fed02b4b1296589c2390e6d5548610099af75300edd7b3ff')
+
def extract_from_iso(self, iso, path):
"""
Extracts a file from an iso file into the test workdir
@@ -72,6 +78,7 @@ def extract_from_iso(self, iso, path):
subprocess.run(cmd.split(),
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
+ os.chmod(filename, 0o600)
os.chdir(cwd)
# Return complete path to extracted file. Because callers to
@@ -83,16 +90,9 @@ def extract_from_iso(self, iso, path):
def setUp(self):
super().setUp()
- iso_url = ('https://dl-cdn.alpinelinux.org/alpine/v3.18/releases/ppc64le/alpine-standard-3.18.4-ppc64le.iso')
-
- # Alpine use sha256 so I recalculated this myself
- iso_sha256 = 'c26b8d3e17c2f3f0fed02b4b1296589c2390e6d5548610099af75300edd7b3ff'
- iso_path = self.fetch_asset(iso_url, asset_hash=iso_sha256,
- algorithm = "sha256")
-
- self.iso_path = iso_path
- self.vmlinuz = self.extract_from_iso(iso_path, '/boot/vmlinuz-lts')
- self.initramfs = self.extract_from_iso(iso_path, '/boot/initramfs-lts')
+ self.iso_path = self.ASSET_ISO.fetch()
+ self.vmlinuz = self.extract_from_iso(self.iso_path, '/boot/vmlinuz-lts')
+ self.initramfs = self.extract_from_iso(self.iso_path, '/boot/initramfs-lts')
def do_start_alpine(self):
self.vm.set_console()
@@ -158,12 +158,8 @@ def do_test_kvm(self, hpt=False):
wait_for_console_pattern(self, 'alpine:~#')
def test_hv_pseries(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:pseries
- :avocado: tags=accel:tcg
- """
self.require_accelerator("tcg")
+ self.set_machine('pseries')
self.vm.add_args("-accel", "tcg,thread=multi")
self.vm.add_args('-device', 'nvme,serial=1234,drive=drive0')
self.vm.add_args("-machine", "x-vof=on,cap-nested-hv=on")
@@ -173,12 +169,8 @@ def test_hv_pseries(self):
self.do_stop_alpine()
def test_hv_pseries_kvm(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:pseries
- :avocado: tags=accel:kvm
- """
self.require_accelerator("kvm")
+ self.set_machine('pseries')
self.vm.add_args("-accel", "kvm")
self.vm.add_args('-device', 'nvme,serial=1234,drive=drive0')
self.vm.add_args("-machine", "x-vof=on,cap-nested-hv=on,cap-ccf-assist=off")
@@ -188,12 +180,8 @@ def test_hv_pseries_kvm(self):
self.do_stop_alpine()
def test_hv_powernv(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:powernv
- :avocado: tags=accel:tcg
- """
self.require_accelerator("tcg")
+ self.set_machine('powernv')
self.vm.add_args("-accel", "tcg,thread=multi")
self.vm.add_args('-device', 'nvme,bus=pcie.2,addr=0x0,serial=1234,drive=drive0',
'-device', 'e1000e,netdev=net0,mac=C0:FF:EE:00:00:02,bus=pcie.0,addr=0x0',
@@ -203,3 +191,6 @@ def test_hv_powernv(self):
self.do_test_kvm()
self.do_test_kvm(True)
self.do_stop_alpine()
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* Re: [PATCH v4 26/35] tests/functional: Convert the ppc_hv avocado test into a standalone test
2024-08-21 8:27 ` [PATCH v4 26/35] tests/functional: Convert the ppc_hv " Thomas Huth
@ 2024-08-21 9:43 ` Philippe Mathieu-Daudé
2024-08-21 10:11 ` Thomas Huth
2024-08-21 11:47 ` Philippe Mathieu-Daudé
1 sibling, 1 reply; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-21 9:43 UTC (permalink / raw)
To: Thomas Huth, Alex Bennée, qemu-devel, Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On 21/8/24 10:27, Thomas Huth wrote:
> A straight forward conversion, we just also have to remove the decorator
> @skipUnless(os.getenv('SPEED')) since all non-trivial functional tests
> are running in SPEED=thorough mode now. Also make sure that the extracted
> assets are writable, so that the test does not fail if it gets re-run
> and there are stale read-only files already around.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> MAINTAINERS | 2 +-
> tests/functional/meson.build | 2 +
> .../test_ppc64_hv.py} | 53 ++++++++-----------
> 3 files changed, 25 insertions(+), 32 deletions(-)
> rename tests/{avocado/ppc_hv_tests.py => functional/test_ppc64_hv.py} (85%)
> mode change 100644 => 100755
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index f01e865025..cca48683ce 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1532,7 +1532,7 @@ F: tests/qtest/libqos/*spapr*
> F: tests/qtest/rtas*
> F: tests/qtest/libqos/rtas*
> F: tests/functional/test_ppc64_pseries.py
> -F: tests/avocado/ppc_hv_tests.py
> +F: tests/functional/test_ppc64_hv.py
>
> PowerNV (Non-Virtualized)
> M: Cédric Le Goater <clg@kaod.org>
> diff --git a/tests/functional/meson.build b/tests/functional/meson.build
> index fba3891e16..f90fb08d6a 100644
> --- a/tests/functional/meson.build
> +++ b/tests/functional/meson.build
> @@ -13,6 +13,7 @@ endif
> test_timeouts = {
> 'netdev_ethtool' : 180,
> 'ppc_40p' : 240,
> + 'ppc64_hv' : 1000,
> 'ppc64_powernv' : 120,
> 'ppc64_pseries' : 120,
> 's390x_ccw_virtio' : 180,
> @@ -63,6 +64,7 @@ tests_ppc_thorough = [
> ]
>
> tests_ppc64_thorough = [
> + 'ppc64_hv',
> 'ppc64_powernv',
> 'ppc64_pseries',
> ]
> diff --git a/tests/avocado/ppc_hv_tests.py b/tests/functional/test_ppc64_hv.py
> old mode 100644
> new mode 100755
> similarity index 85%
> rename from tests/avocado/ppc_hv_tests.py
> rename to tests/functional/test_ppc64_hv.py
> index 0e83bbac71..df59667e07
> --- a/tests/avocado/ppc_hv_tests.py
> +++ b/tests/functional/test_ppc64_hv.py
> @@ -1,3 +1,5 @@
> +#!/usr/bin/env python3
> +#
> # Tests that specifically try to exercise hypervisor features of the
> # target machines. powernv supports the Power hypervisor ISA, and
> # pseries supports the nested-HV hypervisor spec.
> @@ -7,10 +9,10 @@
> # This work is licensed under the terms of the GNU GPL, version 2 or
> # later. See the COPYING file in the top-level directory.
>
> -from avocado import skipIf, skipUnless
> -from avocado.utils import archive
> -from avocado_qemu import QemuSystemTest
> -from avocado_qemu import wait_for_console_pattern, exec_command
> +from unittest import skipIf, skipUnless
> +from qemu_test.utils import archive_extract
> +from qemu_test import QemuSystemTest, Asset
> +from qemu_test import wait_for_console_pattern, exec_command
> import os
> import time
> import subprocess
> @@ -45,8 +47,7 @@ def missing_deps():
> # QEMU already installed and use that.
> # XXX: The order of these tests seems to matter, see git blame.
> @skipIf(missing_deps(), 'dependencies (%s) not installed' % ','.join(deps))
> -@skipUnless(os.getenv('AVOCADO_ALLOW_LARGE_STORAGE'), 'storage limited')
> -@skipUnless(os.getenv('SPEED') == 'slow', 'runtime limited')
> +@skipUnless(os.getenv('QEMU_TEST_ALLOW_LARGE_STORAGE'), 'storage limited')
> class HypervisorTest(QemuSystemTest):
>
> timeout = 1000
> @@ -54,6 +55,11 @@ class HypervisorTest(QemuSystemTest):
> panic_message = 'Kernel panic - not syncing'
> good_message = 'VFS: Cannot open root device'
>
> + ASSET_ISO = Asset(
> + ('https://dl-cdn.alpinelinux.org/alpine/v3.18/'
> + 'releases/ppc64le/alpine-standard-3.18.4-ppc64le.iso'),
> + 'c26b8d3e17c2f3f0fed02b4b1296589c2390e6d5548610099af75300edd7b3ff')
> +
> def extract_from_iso(self, iso, path):
> """
> Extracts a file from an iso file into the test workdir
> @@ -72,6 +78,7 @@ def extract_from_iso(self, iso, path):
> subprocess.run(cmd.split(),
> stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
>
> + os.chmod(filename, 0o600)
> os.chdir(cwd)
>
> # Return complete path to extracted file. Because callers to
> @@ -83,16 +90,9 @@ def extract_from_iso(self, iso, path):
> def setUp(self):
> super().setUp()
>
> - iso_url = ('https://dl-cdn.alpinelinux.org/alpine/v3.18/releases/ppc64le/alpine-standard-3.18.4-ppc64le.iso')
> -
> - # Alpine use sha256 so I recalculated this myself
> - iso_sha256 = 'c26b8d3e17c2f3f0fed02b4b1296589c2390e6d5548610099af75300edd7b3ff'
> - iso_path = self.fetch_asset(iso_url, asset_hash=iso_sha256,
> - algorithm = "sha256")
> -
> - self.iso_path = iso_path
> - self.vmlinuz = self.extract_from_iso(iso_path, '/boot/vmlinuz-lts')
> - self.initramfs = self.extract_from_iso(iso_path, '/boot/initramfs-lts')
> + self.iso_path = self.ASSET_ISO.fetch()
> + self.vmlinuz = self.extract_from_iso(self.iso_path, '/boot/vmlinuz-lts')
> + self.initramfs = self.extract_from_iso(self.iso_path, '/boot/initramfs-lts')
>
> def do_start_alpine(self):
> self.vm.set_console()
Expanding context:
kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE
self.vm.add_args("-kernel", self.vmlinuz)
self.vm.add_args("-initrd", self.initramfs)
self.vm.add_args("-smp", "4", "-m", "2g")
self.vm.add_args("-drive",
f"file={self.iso_path},format=raw,if=none,id=drive0")
Should we use ,media=cdrom here too?
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH v4 26/35] tests/functional: Convert the ppc_hv avocado test into a standalone test
2024-08-21 9:43 ` Philippe Mathieu-Daudé
@ 2024-08-21 10:11 ` Thomas Huth
0 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 10:11 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, Alex Bennée, qemu-devel,
Daniel P . Berrange, Nicholas Piggin
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On 21/08/2024 11.43, Philippe Mathieu-Daudé wrote:
> On 21/8/24 10:27, Thomas Huth wrote:
>> A straight forward conversion, we just also have to remove the decorator
>> @skipUnless(os.getenv('SPEED')) since all non-trivial functional tests
>> are running in SPEED=thorough mode now. Also make sure that the extracted
>> assets are writable, so that the test does not fail if it gets re-run
>> and there are stale read-only files already around.
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>> MAINTAINERS | 2 +-
>> tests/functional/meson.build | 2 +
>> .../test_ppc64_hv.py} | 53 ++++++++-----------
>> 3 files changed, 25 insertions(+), 32 deletions(-)
>> rename tests/{avocado/ppc_hv_tests.py => functional/test_ppc64_hv.py} (85%)
>> mode change 100644 => 100755
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index f01e865025..cca48683ce 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -1532,7 +1532,7 @@ F: tests/qtest/libqos/*spapr*
>> F: tests/qtest/rtas*
>> F: tests/qtest/libqos/rtas*
>> F: tests/functional/test_ppc64_pseries.py
>> -F: tests/avocado/ppc_hv_tests.py
>> +F: tests/functional/test_ppc64_hv.py
>> PowerNV (Non-Virtualized)
>> M: Cédric Le Goater <clg@kaod.org>
>> diff --git a/tests/functional/meson.build b/tests/functional/meson.build
>> index fba3891e16..f90fb08d6a 100644
>> --- a/tests/functional/meson.build
>> +++ b/tests/functional/meson.build
>> @@ -13,6 +13,7 @@ endif
>> test_timeouts = {
>> 'netdev_ethtool' : 180,
>> 'ppc_40p' : 240,
>> + 'ppc64_hv' : 1000,
>> 'ppc64_powernv' : 120,
>> 'ppc64_pseries' : 120,
>> 's390x_ccw_virtio' : 180,
>> @@ -63,6 +64,7 @@ tests_ppc_thorough = [
>> ]
>> tests_ppc64_thorough = [
>> + 'ppc64_hv',
>> 'ppc64_powernv',
>> 'ppc64_pseries',
>> ]
>> diff --git a/tests/avocado/ppc_hv_tests.py
>> b/tests/functional/test_ppc64_hv.py
>> old mode 100644
>> new mode 100755
>> similarity index 85%
>> rename from tests/avocado/ppc_hv_tests.py
>> rename to tests/functional/test_ppc64_hv.py
>> index 0e83bbac71..df59667e07
>> --- a/tests/avocado/ppc_hv_tests.py
>> +++ b/tests/functional/test_ppc64_hv.py
>> @@ -1,3 +1,5 @@
>> +#!/usr/bin/env python3
>> +#
>> # Tests that specifically try to exercise hypervisor features of the
>> # target machines. powernv supports the Power hypervisor ISA, and
>> # pseries supports the nested-HV hypervisor spec.
>> @@ -7,10 +9,10 @@
>> # This work is licensed under the terms of the GNU GPL, version 2 or
>> # later. See the COPYING file in the top-level directory.
>> -from avocado import skipIf, skipUnless
>> -from avocado.utils import archive
>> -from avocado_qemu import QemuSystemTest
>> -from avocado_qemu import wait_for_console_pattern, exec_command
>> +from unittest import skipIf, skipUnless
>> +from qemu_test.utils import archive_extract
>> +from qemu_test import QemuSystemTest, Asset
>> +from qemu_test import wait_for_console_pattern, exec_command
>> import os
>> import time
>> import subprocess
>> @@ -45,8 +47,7 @@ def missing_deps():
>> # QEMU already installed and use that.
>> # XXX: The order of these tests seems to matter, see git blame.
>> @skipIf(missing_deps(), 'dependencies (%s) not installed' % ','.join(deps))
>> -@skipUnless(os.getenv('AVOCADO_ALLOW_LARGE_STORAGE'), 'storage limited')
>> -@skipUnless(os.getenv('SPEED') == 'slow', 'runtime limited')
>> +@skipUnless(os.getenv('QEMU_TEST_ALLOW_LARGE_STORAGE'), 'storage limited')
>> class HypervisorTest(QemuSystemTest):
>> timeout = 1000
>> @@ -54,6 +55,11 @@ class HypervisorTest(QemuSystemTest):
>> panic_message = 'Kernel panic - not syncing'
>> good_message = 'VFS: Cannot open root device'
>> + ASSET_ISO = Asset(
>> + ('https://dl-cdn.alpinelinux.org/alpine/v3.18/'
>> + 'releases/ppc64le/alpine-standard-3.18.4-ppc64le.iso'),
>> + 'c26b8d3e17c2f3f0fed02b4b1296589c2390e6d5548610099af75300edd7b3ff')
>> +
>> def extract_from_iso(self, iso, path):
>> """
>> Extracts a file from an iso file into the test workdir
>> @@ -72,6 +78,7 @@ def extract_from_iso(self, iso, path):
>> subprocess.run(cmd.split(),
>> stdout=subprocess.DEVNULL,
>> stderr=subprocess.DEVNULL)
>> + os.chmod(filename, 0o600)
>> os.chdir(cwd)
>> # Return complete path to extracted file. Because callers to
>> @@ -83,16 +90,9 @@ def extract_from_iso(self, iso, path):
>> def setUp(self):
>> super().setUp()
>> - iso_url =
>> ('https://dl-cdn.alpinelinux.org/alpine/v3.18/releases/ppc64le/alpine-standard-3.18.4-ppc64le.iso')
>> -
>> - # Alpine use sha256 so I recalculated this myself
>> - iso_sha256 =
>> 'c26b8d3e17c2f3f0fed02b4b1296589c2390e6d5548610099af75300edd7b3ff'
>> - iso_path = self.fetch_asset(iso_url, asset_hash=iso_sha256,
>> - algorithm = "sha256")
>> -
>> - self.iso_path = iso_path
>> - self.vmlinuz = self.extract_from_iso(iso_path, '/boot/vmlinuz-lts')
>> - self.initramfs = self.extract_from_iso(iso_path,
>> '/boot/initramfs-lts')
>> + self.iso_path = self.ASSET_ISO.fetch()
>> + self.vmlinuz = self.extract_from_iso(self.iso_path,
>> '/boot/vmlinuz-lts')
>> + self.initramfs = self.extract_from_iso(self.iso_path,
>> '/boot/initramfs-lts')
>> def do_start_alpine(self):
>> self.vm.set_console()
>
> Expanding context:
>
> kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE
> self.vm.add_args("-kernel", self.vmlinuz)
> self.vm.add_args("-initrd", self.initramfs)
> self.vm.add_args("-smp", "4", "-m", "2g")
> self.vm.add_args("-drive",
> f"file={self.iso_path},format=raw,if=none,id=drive0")
>
> Should we use ,media=cdrom here too?
I guess that's rather a question for Nicholas (now on CC:)...
Anyway, it should be a separate patch, I think.
Thomas
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v4 26/35] tests/functional: Convert the ppc_hv avocado test into a standalone test
2024-08-21 8:27 ` [PATCH v4 26/35] tests/functional: Convert the ppc_hv " Thomas Huth
2024-08-21 9:43 ` Philippe Mathieu-Daudé
@ 2024-08-21 11:47 ` Philippe Mathieu-Daudé
1 sibling, 0 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-21 11:47 UTC (permalink / raw)
To: Thomas Huth, Alex Bennée, qemu-devel, Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On 21/8/24 10:27, Thomas Huth wrote:
> A straight forward conversion, we just also have to remove the decorator
> @skipUnless(os.getenv('SPEED')) since all non-trivial functional tests
> are running in SPEED=thorough mode now. Also make sure that the extracted
> assets are writable, so that the test does not fail if it gets re-run
> and there are stale read-only files already around.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> MAINTAINERS | 2 +-
> tests/functional/meson.build | 2 +
> .../test_ppc64_hv.py} | 53 ++++++++-----------
> 3 files changed, 25 insertions(+), 32 deletions(-)
> rename tests/{avocado/ppc_hv_tests.py => functional/test_ppc64_hv.py} (85%)
> mode change 100644 => 100755
> diff --git a/tests/avocado/ppc_hv_tests.py b/tests/functional/test_ppc64_hv.py
> old mode 100644
> new mode 100755
> similarity index 85%
> rename from tests/avocado/ppc_hv_tests.py
> rename to tests/functional/test_ppc64_hv.py
> index 0e83bbac71..df59667e07
> --- a/tests/avocado/ppc_hv_tests.py
> +++ b/tests/functional/test_ppc64_hv.py
> @@ -1,3 +1,5 @@
> +#!/usr/bin/env python3
> +#
> # Tests that specifically try to exercise hypervisor features of the
> # target machines. powernv supports the Power hypervisor ISA, and
> # pseries supports the nested-HV hypervisor spec.
> @@ -7,10 +9,10 @@
> # This work is licensed under the terms of the GNU GPL, version 2 or
> # later. See the COPYING file in the top-level directory.
>
> -from avocado import skipIf, skipUnless
> -from avocado.utils import archive
Pre-existing, "archive" is not used.
> -from avocado_qemu import QemuSystemTest
> -from avocado_qemu import wait_for_console_pattern, exec_command
> +from unittest import skipIf, skipUnless
> +from qemu_test.utils import archive_extract
> +from qemu_test import QemuSystemTest, Asset
> +from qemu_test import wait_for_console_pattern, exec_command
^ permalink raw reply [flat|nested] 69+ messages in thread
* [PATCH v4 27/35] tests/functional: Convert the m68k nextcube test with tesseract
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (25 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 26/35] tests/functional: Convert the ppc_hv " Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 8:27 ` [PATCH v4 28/35] tests/functional: Convert the acpi-bits test into a standalone test Thomas Huth
` (7 subsequent siblings)
34 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
The code that handles running of tesseract needs to be tweaked a little
bit to be able to run without the functions from avocado.utils, and
while we're at it, drop some legacy stuff that was still there due to
Tesseract 3 support that we already dropped a while ago.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 1 +
tests/avocado/tesseract_utils.py | 46 -------------------
tests/functional/meson.build | 4 ++
tests/functional/qemu_test/tesseract.py | 35 ++++++++++++++
.../test_m68k_nextcube.py} | 29 ++++++------
5 files changed, 56 insertions(+), 59 deletions(-)
delete mode 100644 tests/avocado/tesseract_utils.py
create mode 100644 tests/functional/qemu_test/tesseract.py
rename tests/{avocado/machine_m68k_nextcube.py => functional/test_m68k_nextcube.py} (75%)
mode change 100644 => 100755
diff --git a/MAINTAINERS b/MAINTAINERS
index cca48683ce..394d704515 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1283,6 +1283,7 @@ S: Odd Fixes
F: hw/m68k/next-*.c
F: hw/display/next-fb.c
F: include/hw/m68k/next-cube.h
+F: tests/functional/test_m68k_nextcube.py
q800
M: Laurent Vivier <laurent@vivier.eu>
diff --git a/tests/avocado/tesseract_utils.py b/tests/avocado/tesseract_utils.py
deleted file mode 100644
index 476f528147..0000000000
--- a/tests/avocado/tesseract_utils.py
+++ /dev/null
@@ -1,46 +0,0 @@
-# ...
-#
-# Copyright (c) 2019 Philippe Mathieu-Daudé <f4bug@amsat.org>
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or
-# later. See the COPYING file in the top-level directory.
-
-import re
-import logging
-
-from avocado.utils import process
-from avocado.utils.path import find_command, CmdNotFoundError
-
-def tesseract_available(expected_version):
- try:
- find_command('tesseract')
- except CmdNotFoundError:
- return False
- res = process.run('tesseract --version')
- try:
- version = res.stdout_text.split()[1]
- except IndexError:
- version = res.stderr_text.split()[1]
- return int(version.split('.')[0]) >= expected_version
-
- match = re.match(r'tesseract\s(\d)', res)
- if match is None:
- return False
- # now this is guaranteed to be a digit
- return int(match.groups()[0]) >= expected_version
-
-
-def tesseract_ocr(image_path, tesseract_args='', tesseract_version=3):
- console_logger = logging.getLogger('tesseract')
- console_logger.debug(image_path)
- if tesseract_version == 4:
- tesseract_args += ' --oem 1'
- proc = process.run("tesseract {} {} stdout".format(tesseract_args,
- image_path))
- lines = []
- for line in proc.stdout_text.split('\n'):
- sline = line.strip()
- if len(sline):
- console_logger.debug(sline)
- lines += [sline]
- return lines
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index f90fb08d6a..e95b2c5592 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -38,6 +38,10 @@ tests_loongarch64_thorough = [
'loongarch64_virt',
]
+tests_m68k_thorough = [
+ 'm68k_nextcube'
+]
+
tests_microblaze_thorough = [
'microblaze_s3adsp1800'
]
diff --git a/tests/functional/qemu_test/tesseract.py b/tests/functional/qemu_test/tesseract.py
new file mode 100644
index 0000000000..c4087b7c11
--- /dev/null
+++ b/tests/functional/qemu_test/tesseract.py
@@ -0,0 +1,35 @@
+# ...
+#
+# Copyright (c) 2019 Philippe Mathieu-Daudé <f4bug@amsat.org>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import re
+import logging
+
+from . import has_cmd, run_cmd
+
+def tesseract_available(expected_version):
+ if not has_cmd('tesseract'):
+ return False
+ (stdout, stderr, ret) = run_cmd([ 'tesseract', '--version'])
+ if ret:
+ return False
+ version = stdout.split()[1]
+ return int(version.split('.')[0]) >= expected_version
+
+def tesseract_ocr(image_path, tesseract_args=''):
+ console_logger = logging.getLogger('console')
+ console_logger.debug(image_path)
+ (stdout, stderr, ret) = run_cmd(['tesseract', image_path,
+ 'stdout'])
+ if ret:
+ return None
+ lines = []
+ for line in stdout.split('\n'):
+ sline = line.strip()
+ if len(sline):
+ console_logger.debug(sline)
+ lines += [sline]
+ return lines
diff --git a/tests/avocado/machine_m68k_nextcube.py b/tests/functional/test_m68k_nextcube.py
old mode 100644
new mode 100755
similarity index 75%
rename from tests/avocado/machine_m68k_nextcube.py
rename to tests/functional/test_m68k_nextcube.py
index 1f3c883910..89385a134a
--- a/tests/avocado/machine_m68k_nextcube.py
+++ b/tests/functional/test_m68k_nextcube.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Functional test that boots a VM and run OCR on the framebuffer
#
# Copyright (c) 2019 Philippe Mathieu-Daudé <f4bug@amsat.org>
@@ -8,10 +10,10 @@
import os
import time
-from avocado_qemu import QemuSystemTest
-from avocado import skipUnless
+from qemu_test import QemuSystemTest, Asset
+from unittest import skipUnless
-from tesseract_utils import tesseract_available, tesseract_ocr
+from qemu_test.tesseract import tesseract_available, tesseract_ocr
PIL_AVAILABLE = True
try:
@@ -21,19 +23,15 @@
class NextCubeMachine(QemuSystemTest):
- """
- :avocado: tags=arch:m68k
- :avocado: tags=machine:next-cube
- :avocado: tags=device:framebuffer
- """
timeout = 15
+ ASSET_ROM = Asset(('https://sourceforge.net/p/previous/code/1350/tree/'
+ 'trunk/src/Rev_2.5_v66.BIN?format=raw'),
+ '1b753890b67095b73e104c939ddf62eca9e7d0aedde5108e3893b0ed9d8000a4')
+
def check_bootrom_framebuffer(self, screenshot_path):
- rom_url = ('https://sourceforge.net/p/previous/code/1350/tree/'
- 'trunk/src/Rev_2.5_v66.BIN?format=raw')
- rom_hash = 'b3534796abae238a0111299fc406a9349f7fee24'
- rom_path = self.fetch_asset(rom_url, asset_hash=rom_hash)
+ rom_path = self.ASSET_ROM.fetch()
self.vm.add_args('-bios', rom_path)
self.vm.launch()
@@ -48,6 +46,7 @@ def check_bootrom_framebuffer(self, screenshot_path):
@skipUnless(PIL_AVAILABLE, 'Python PIL not installed')
def test_bootrom_framebuffer_size(self):
+ self.set_machine('next-cube')
screenshot_path = os.path.join(self.workdir, "dump.ppm")
self.check_bootrom_framebuffer(screenshot_path)
@@ -60,11 +59,15 @@ def test_bootrom_framebuffer_size(self):
# that it is still alpha-level software.
@skipUnless(tesseract_available(4), 'tesseract OCR tool not available')
def test_bootrom_framebuffer_ocr_with_tesseract(self):
+ self.set_machine('next-cube')
screenshot_path = os.path.join(self.workdir, "dump.ppm")
self.check_bootrom_framebuffer(screenshot_path)
- lines = tesseract_ocr(screenshot_path, tesseract_version=4)
+ lines = tesseract_ocr(screenshot_path)
text = '\n'.join(lines)
self.assertIn('Testing the FPU', text)
self.assertIn('System test failed. Error code', text)
self.assertIn('Boot command', text)
self.assertIn('Next>', text)
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH v4 28/35] tests/functional: Convert the acpi-bits test into a standalone test
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (26 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 27/35] tests/functional: Convert the m68k nextcube test with tesseract Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 8:27 ` [PATCH v4 29/35] tests/functional: Convert the rx_gdbsim avocado " Thomas Huth
` (6 subsequent siblings)
34 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
Mostly a straight-forward conversion. Looks like we can simply drop
the avocado datadrainer stuff when not using the avocado framework
anymore.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 4 +-
docs/devel/acpi-bits.rst | 62 ++++++--------
.../acpi-bits/bits-config/bits-cfg.txt | 0
.../acpi-bits/bits-tests/smbios.py2 | 0
.../acpi-bits/bits-tests/smilatency.py2 | 0
.../acpi-bits/bits-tests/testacpi.py2 | 0
.../acpi-bits/bits-tests/testcpuid.py2 | 0
tests/functional/meson.build | 2 +
.../test_acpi_bits.py} | 83 ++++++++++---------
9 files changed, 71 insertions(+), 80 deletions(-)
rename tests/{avocado => functional}/acpi-bits/bits-config/bits-cfg.txt (100%)
rename tests/{avocado => functional}/acpi-bits/bits-tests/smbios.py2 (100%)
rename tests/{avocado => functional}/acpi-bits/bits-tests/smilatency.py2 (100%)
rename tests/{avocado => functional}/acpi-bits/bits-tests/testacpi.py2 (100%)
rename tests/{avocado => functional}/acpi-bits/bits-tests/testcpuid.py2 (100%)
rename tests/{avocado/acpi-bits.py => functional/test_acpi_bits.py} (86%)
mode change 100644 => 100755
diff --git a/MAINTAINERS b/MAINTAINERS
index 394d704515..c2dfb3dad3 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2075,8 +2075,8 @@ ACPI/AVOCADO/BIOSBITS
M: Ani Sinha <anisinha@redhat.com>
M: Michael S. Tsirkin <mst@redhat.com>
S: Supported
-F: tests/avocado/acpi-bits/*
-F: tests/avocado/acpi-bits.py
+F: tests/functional/acpi-bits/*
+F: tests/functional/test_acpi_bits.py
F: docs/devel/acpi-bits.rst
ACPI/HEST/GHES
diff --git a/docs/devel/acpi-bits.rst b/docs/devel/acpi-bits.rst
index 1ec394f5fb..78aeb6aa3c 100644
--- a/docs/devel/acpi-bits.rst
+++ b/docs/devel/acpi-bits.rst
@@ -1,6 +1,6 @@
-=============================================================================
-ACPI/SMBIOS avocado tests using biosbits
-=============================================================================
+==================================
+ACPI/SMBIOS testing using biosbits
+==================================
************
Introduction
************
@@ -35,7 +35,7 @@ for developing biosbits and its real life uses can be found in [#a]_ and [#b]_.
For QEMU, we maintain a fork of bios bits in gitlab along with all the
dependent submodules `here <https://gitlab.com/qemu-project/biosbits-bits>`__.
This fork contains numerous fixes, a newer acpica and changes specific to
-running this avocado QEMU tests using bits. The author of this document
+running these functional QEMU tests using bits. The author of this document
is the sole maintainer of the QEMU fork of bios bits repository. For more
information, please see author's `FOSDEM talk on this bios-bits based test
framework <https://fosdem.org/2024/schedule/event/fosdem-2024-2262-exercising-qemu-generated-acpi-smbios-tables-using-biosbits-from-within-a-guest-vm-/>`__.
@@ -44,12 +44,12 @@ framework <https://fosdem.org/2024/schedule/event/fosdem-2024-2262-exercising-qe
Description of the test framework
*********************************
-Under the directory ``tests/avocado/``, ``acpi-bits.py`` is a QEMU avocado
-test that drives all this.
+Under the directory ``tests/functional/``, ``test_acpi_bits.py`` is a QEMU
+functional test that drives all this.
A brief description of the various test files follows.
-Under ``tests/avocado/`` as the root we have:
+Under ``tests/functional/`` as the root we have:
::
@@ -60,12 +60,12 @@ Under ``tests/avocado/`` as the root we have:
│ ├── smbios.py2
│ ├── testacpi.py2
│ └── testcpuid.py2
- ├── acpi-bits.py
+ ├── test_acpi_bits.py
-* ``tests/avocado``:
+* ``tests/functional``:
- ``acpi-bits.py``:
- This is the main python avocado test script that generates a
+ ``test_acpi_bits.py``:
+ This is the main python functional test script that generates a
biosbits iso. It then spawns a QEMU VM with it, collects the log and reports
test failures. This is the script one would be interested in if they wanted
to add or change some component of the log parsing, add a new command line
@@ -79,35 +79,22 @@ Under ``tests/avocado/`` as the root we have:
you to inspect and run the specific commands manually.
In order to run this test, please perform the following steps from the QEMU
- build directory:
+ build directory (assuming that the sources are in ".."):
::
- $ make check-venv (needed only the first time to create the venv)
- $ ./pyvenv/bin/avocado run -t acpi tests/avocado
+ $ export PYTHONPATH=../python:../tests/functional
+ $ export QEMU_TEST_QEMU_BINARY=$PWD/qemu-system-x86_64
+ $ python3 ../tests/functional/test_acpi_bits.py
- The above will run all acpi avocado tests including this one.
- In order to run the individual tests, perform the following:
- ::
-
- $ ./pyvenv/bin/avocado run tests/avocado/acpi-bits.py --tap -
-
- The above will produce output in tap format. You can omit "--tap -" in the
- end and it will produce output like the following:
- ::
-
- $ ./pyvenv/bin/avocado run tests/avocado/acpi-bits.py
- Fetching asset from tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
- JOB ID : eab225724da7b64c012c65705dc2fa14ab1defef
- JOB LOG : /home/anisinha/avocado/job-results/job-2022-10-10T17.58-eab2257/job.log
- (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits: PASS (33.09 s)
- RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
- JOB TIME : 39.22 s
+ The above will run all acpi-bits functional tests (producing output in
+ tap format).
- You can inspect the log file for more information about the run or in order
- to diagnoze issues. If you pass V=1 in the environment, more diagnostic logs
- would be found in the test log.
+ You can inspect the log files in tests/functional/x86_64/test_acpi_bits.*/
+ for more information about the run or in order to diagnoze issues.
+ If you pass V=1 in the environment, more diagnostic logs will be put into
+ the test log.
-* ``tests/avocado/acpi-bits/bits-config``:
+* ``tests/functional/acpi-bits/bits-config``:
This location contains biosbits configuration files that determine how the
software runs the tests.
@@ -117,7 +104,7 @@ Under ``tests/avocado/`` as the root we have:
or actions are performed by bits. The description of the config options are
provided in the file itself.
-* ``tests/avocado/acpi-bits/bits-tests``:
+* ``tests/functional/acpi-bits/bits-tests``:
This directory contains biosbits python based tests that are run from within
the biosbits environment in the spawned VM. New additions of test cases can
@@ -155,7 +142,8 @@ Under ``tests/avocado/`` as the root we have:
(a) They are python2.7 based scripts and not python 3 scripts.
(b) They are run from within the bios bits VM and is not subjected to QEMU
build/test python script maintenance and dependency resolutions.
- (c) They need not be loaded by avocado framework when running tests.
+ (c) They need not be loaded by the test framework by accident when running
+ tests.
Author: Ani Sinha <anisinha@redhat.com>
diff --git a/tests/avocado/acpi-bits/bits-config/bits-cfg.txt b/tests/functional/acpi-bits/bits-config/bits-cfg.txt
similarity index 100%
rename from tests/avocado/acpi-bits/bits-config/bits-cfg.txt
rename to tests/functional/acpi-bits/bits-config/bits-cfg.txt
diff --git a/tests/avocado/acpi-bits/bits-tests/smbios.py2 b/tests/functional/acpi-bits/bits-tests/smbios.py2
similarity index 100%
rename from tests/avocado/acpi-bits/bits-tests/smbios.py2
rename to tests/functional/acpi-bits/bits-tests/smbios.py2
diff --git a/tests/avocado/acpi-bits/bits-tests/smilatency.py2 b/tests/functional/acpi-bits/bits-tests/smilatency.py2
similarity index 100%
rename from tests/avocado/acpi-bits/bits-tests/smilatency.py2
rename to tests/functional/acpi-bits/bits-tests/smilatency.py2
diff --git a/tests/avocado/acpi-bits/bits-tests/testacpi.py2 b/tests/functional/acpi-bits/bits-tests/testacpi.py2
similarity index 100%
rename from tests/avocado/acpi-bits/bits-tests/testacpi.py2
rename to tests/functional/acpi-bits/bits-tests/testacpi.py2
diff --git a/tests/avocado/acpi-bits/bits-tests/testcpuid.py2 b/tests/functional/acpi-bits/bits-tests/testcpuid.py2
similarity index 100%
rename from tests/avocado/acpi-bits/bits-tests/testcpuid.py2
rename to tests/functional/acpi-bits/bits-tests/testcpuid.py2
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index e95b2c5592..aa999cb880 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -11,6 +11,7 @@ endif
# Timeouts for individual tests that can be slow e.g. with debugging enabled
test_timeouts = {
+ 'acpi_bits' : 240,
'netdev_ethtool' : 180,
'ppc_40p' : 240,
'ppc64_hv' : 1000,
@@ -99,6 +100,7 @@ tests_x86_64_quick = [
]
tests_x86_64_thorough = [
+ 'acpi_bits',
'netdev_ethtool',
'virtio_gpu',
]
diff --git a/tests/avocado/acpi-bits.py b/tests/functional/test_acpi_bits.py
old mode 100644
new mode 100755
similarity index 86%
rename from tests/avocado/acpi-bits.py
rename to tests/functional/test_acpi_bits.py
index efe4f52ee0..ee40647d5b
--- a/tests/avocado/acpi-bits.py
+++ b/tests/functional/test_acpi_bits.py
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
-# group: rw quick
+#
# Exercise QEMU generated ACPI/SMBIOS tables using biosbits,
# https://biosbits.org/
#
@@ -24,7 +24,7 @@
# pylint: disable=consider-using-f-string
"""
-This is QEMU ACPI/SMBIOS avocado tests using biosbits.
+This is QEMU ACPI/SMBIOS functional tests using biosbits.
Biosbits is available originally at https://biosbits.org/.
This test uses a fork of the upstream bits and has numerous fixes
including an upgraded acpica. The fork is located here:
@@ -41,15 +41,16 @@
import tempfile
import time
import zipfile
+
+from pathlib import Path
from typing import (
List,
Optional,
Sequence,
)
from qemu.machine import QEMUMachine
-from avocado import skipIf
-from avocado.utils import datadrainer as drainer
-from avocado_qemu import QemuBaseTest
+from unittest import skipIf
+from qemu_test import QemuBaseTest, Asset
deps = ["xorriso", "mformat"] # dependent tools needed in the test setup/box.
supported_platforms = ['x86_64'] # supported test platforms.
@@ -129,34 +130,32 @@ def base_args(self):
class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes
"""
ACPI and SMBIOS tests using biosbits.
-
- :avocado: tags=arch:x86_64
- :avocado: tags=acpi
-
"""
# in slower systems the test can take as long as 3 minutes to complete.
timeout = BITS_TIMEOUT
+ # following are some standard configuration constants
+ # gitlab CI does shallow clones of depth 20
+ BITS_INTERNAL_VER = 2020
+ # commit hash must match the artifact tag below
+ BITS_COMMIT_HASH = 'c7920d2b'
+ # this is the latest bits release as of today.
+ BITS_TAG = "qemu-bits-10262023"
+
+ ASSET_BITS = Asset(("https://gitlab.com/qemu-project/"
+ "biosbits-bits/-/jobs/artifacts/%s/"
+ "download?job=qemu-bits-build" % BITS_TAG),
+ '1b8dd612c6831a6b491716a77acc486666aaa867051cdc34f7ce169c2e25f487')
+
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self._vm = None
self._workDir = None
self._baseDir = None
- # following are some standard configuration constants
- self._bitsInternalVer = 2020 # gitlab CI does shallow clones of depth 20
- self._bitsCommitHash = 'c7920d2b' # commit hash must match
- # the artifact tag below
- self._bitsTag = "qemu-bits-10262023" # this is the latest bits
- # release as of today.
- self._bitsArtSHA1Hash = 'b22cdfcfc7453875297d06d626f5474ee36a343f'
- self._bitsArtURL = ("https://gitlab.com/qemu-project/"
- "biosbits-bits/-/jobs/artifacts/%s/"
- "download?job=qemu-bits-build" %self._bitsTag)
self._debugcon_addr = '0x403'
self._debugcon_log = 'debugcon-log.txt'
- logging.basicConfig(level=logging.INFO)
- self.logger = logging.getLogger('acpi-bits')
+ self.logger = self.log
def _print_log(self, log):
self.logger.info('\nlogs from biosbits follows:')
@@ -171,7 +170,7 @@ def copy_bits_config(self):
bits_config_dir = os.path.join(self._baseDir, 'acpi-bits',
'bits-config')
target_config_dir = os.path.join(self._workDir,
- 'bits-%d' %self._bitsInternalVer,
+ 'bits-%d' %self.BITS_INTERNAL_VER,
'boot')
self.assertTrue(os.path.exists(bits_config_dir))
self.assertTrue(os.path.exists(target_config_dir))
@@ -188,7 +187,7 @@ def copy_test_scripts(self):
bits_test_dir = os.path.join(self._baseDir, 'acpi-bits',
'bits-tests')
target_test_dir = os.path.join(self._workDir,
- 'bits-%d' %self._bitsInternalVer,
+ 'bits-%d' %self.BITS_INTERNAL_VER,
'boot', 'python')
self.assertTrue(os.path.exists(bits_test_dir))
@@ -248,9 +247,9 @@ def generate_bits_iso(self):
test scripts
"""
bits_dir = os.path.join(self._workDir,
- 'bits-%d' %self._bitsInternalVer)
+ 'bits-%d' %self.BITS_INTERNAL_VER)
iso_file = os.path.join(self._workDir,
- 'bits-%d.iso' %self._bitsInternalVer)
+ 'bits-%d.iso' %self.BITS_INTERNAL_VER)
mkrescue_script = os.path.join(self._workDir,
'grub-inst-x86_64-efi', 'bin',
'grub-mkrescue')
@@ -264,8 +263,12 @@ def generate_bits_iso(self):
try:
if os.getenv('V') or os.getenv('BITS_DEBUG'):
- subprocess.check_call([mkrescue_script, '-o', iso_file,
- bits_dir], stderr=subprocess.STDOUT)
+ proc = subprocess.run([mkrescue_script, '-o', iso_file,
+ bits_dir],
+ stdout=subprocess.PIPE,
+ stderr=subprocess.STDOUT,
+ check=True)
+ self.logger.info("grub-mkrescue output %s" % proc.stdout)
else:
subprocess.check_call([mkrescue_script, '-o',
iso_file, bits_dir],
@@ -282,8 +285,9 @@ def generate_bits_iso(self):
def setUp(self): # pylint: disable=arguments-differ
super().setUp('qemu-system-')
+ self.logger = self.log
- self._baseDir = os.getenv('AVOCADO_TEST_BASEDIR')
+ self._baseDir = Path(__file__).parent
# workdir could also be avocado's own workdir in self.workdir.
# At present, I prefer to maintain my own temporary working
@@ -300,15 +304,14 @@ def setUp(self): # pylint: disable=arguments-differ
os.mkdir(prebuiltDir, mode=0o775)
bits_zip_file = os.path.join(prebuiltDir, 'bits-%d-%s.zip'
- %(self._bitsInternalVer,
- self._bitsCommitHash))
+ %(self.BITS_INTERNAL_VER,
+ self.BITS_COMMIT_HASH))
grub_tar_file = os.path.join(prebuiltDir,
'bits-%d-%s-grub.tar.gz'
- %(self._bitsInternalVer,
- self._bitsCommitHash))
+ %(self.BITS_INTERNAL_VER,
+ self.BITS_COMMIT_HASH))
- bitsLocalArtLoc = self.fetch_asset(self._bitsArtURL,
- asset_hash=self._bitsArtSHA1Hash)
+ bitsLocalArtLoc = self.ASSET_BITS.fetch()
self.logger.info("downloaded bits artifacts to %s", bitsLocalArtLoc)
# extract the bits artifact in the temp working directory
@@ -369,7 +372,7 @@ def test_acpi_smbios_bits(self):
"""The main test case implementation."""
iso_file = os.path.join(self._workDir,
- 'bits-%d.iso' %self._bitsInternalVer)
+ 'bits-%d.iso' %self.BITS_INTERNAL_VER)
self.assertTrue(os.access(iso_file, os.R_OK))
@@ -393,12 +396,6 @@ def test_acpi_smbios_bits(self):
self._vm.set_console()
self._vm.launch()
- self.logger.debug("Console output from bits VM follows ...")
- c_drainer = drainer.LineLogger(self._vm.console_socket.fileno(),
- logger=self.logger.getChild("console"),
- stop_check=(lambda :
- not self._vm.is_running()))
- c_drainer.start()
# biosbits has been configured to run all the specified test suites
# in batch mode and then automatically initiate a vm shutdown.
@@ -406,4 +403,8 @@ def test_acpi_smbios_bits(self):
# with the avocado test timeout.
self._vm.event_wait('SHUTDOWN', timeout=BITS_TIMEOUT)
self._vm.wait(timeout=None)
+ self.logger.debug("Checking console output ...")
self.parse_log()
+
+if __name__ == '__main__':
+ QemuBaseTest.main()
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH v4 29/35] tests/functional: Convert the rx_gdbsim avocado test into a standalone test
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (27 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 28/35] tests/functional: Convert the acpi-bits test into a standalone test Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 8:27 ` [PATCH v4 30/35] tests/functional: Convert the linux_initrd " Thomas Huth
` (5 subsequent siblings)
34 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
Provide a "gzip_uncompress" function based on the standard "gzip" module
to avoid the usage of avocado.utils here.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 2 +-
tests/functional/meson.build | 4 ++
tests/functional/qemu_test/utils.py | 12 +++++
.../test_rx_gdbsim.py} | 54 ++++++++++---------
4 files changed, 46 insertions(+), 26 deletions(-)
rename tests/{avocado/machine_rx_gdbsim.py => functional/test_rx_gdbsim.py} (58%)
mode change 100644 => 100755
diff --git a/MAINTAINERS b/MAINTAINERS
index c2dfb3dad3..25e71ac14c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1665,7 +1665,7 @@ R: Yoshinori Sato <ysato@users.sourceforge.jp>
S: Orphan
F: docs/system/target-rx.rst
F: hw/rx/rx-gdbsim.c
-F: tests/avocado/machine_rx_gdbsim.py
+F: tests/functional/test_rx_gdbsim.py
SH4 Machines
------------
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index aa999cb880..e28aa34087 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -82,6 +82,10 @@ tests_riscv64_quick = [
'riscv_opensbi',
]
+tests_rx_thorough = [
+ 'rx_gdbsim',
+]
+
tests_s390x_thorough = [
's390x_ccw_virtio',
's390x_topology',
diff --git a/tests/functional/qemu_test/utils.py b/tests/functional/qemu_test/utils.py
index a12dac51b6..99eae5fc45 100644
--- a/tests/functional/qemu_test/utils.py
+++ b/tests/functional/qemu_test/utils.py
@@ -8,6 +8,7 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
+import gzip
import lzma
import os
import shutil
@@ -23,6 +24,17 @@ def archive_extract(archive, dest_dir, member=None):
else:
tf.extractall(path=dest_dir)
+def gzip_uncompress(gz_path, output_path):
+ if os.path.exists(output_path):
+ return
+ with gzip.open(gz_path, 'rb') as gz_in:
+ try:
+ with open(output_path, 'wb') as raw_out:
+ shutil.copyfileobj(gz_in, raw_out)
+ except:
+ os.remove(output_path)
+ raise
+
def lzma_uncompress(xz_path, output_path):
if os.path.exists(output_path):
return
diff --git a/tests/avocado/machine_rx_gdbsim.py b/tests/functional/test_rx_gdbsim.py
old mode 100644
new mode 100755
similarity index 58%
rename from tests/avocado/machine_rx_gdbsim.py
rename to tests/functional/test_rx_gdbsim.py
index 6bd9ce8199..5687f756bb
--- a/tests/avocado/machine_rx_gdbsim.py
+++ b/tests/functional/test_rx_gdbsim.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Functional test that boots a Linux kernel and checks the console
#
# Copyright (c) 2018 Red Hat, Inc.
@@ -10,11 +12,11 @@
import os
-from avocado import skipUnless
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import exec_command_and_wait_for_pattern
-from avocado_qemu import wait_for_console_pattern
-from avocado.utils import archive
+from unittest import skipUnless
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import exec_command_and_wait_for_pattern
+from qemu_test import wait_for_console_pattern
+from qemu_test.utils import gzip_uncompress
class RxGdbSimMachine(QemuSystemTest):
@@ -22,19 +24,25 @@ class RxGdbSimMachine(QemuSystemTest):
timeout = 30
KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
+ ASSET_UBOOT = Asset(
+ 'https://acc.dl.osdn.jp/users/23/23888/u-boot.bin.gz',
+ '7146567d669e91dbac166384b29aeba1715beb844c8551e904b86831bfd9d046')
+ ASSET_DTB = Asset(
+ 'https://acc.dl.osdn.jp/users/23/23887/rx-virt.dtb',
+ 'aa278d9c1907a4501741d7ee57e7f65c02dd1b3e0323b33c6d4247f1b32cf29a')
+ ASSET_KERNEL = Asset(
+ 'http://acc.dl.osdn.jp/users/23/23845/zImage',
+ 'baa43205e74a7220ed8482188c5e9ce497226712abb7f4e7e4f825ce19ff9656')
+
def test_uboot(self):
"""
U-Boot and checks that the console is operational.
-
- :avocado: tags=arch:rx
- :avocado: tags=machine:gdbsim-r5f562n8
- :avocado: tags=endian:little
- :avocado: tags=flaky
"""
- uboot_url = ('https://acc.dl.osdn.jp/users/23/23888/u-boot.bin.gz')
- uboot_hash = '9b78dbd43b40b2526848c0b1ce9de02c24f4dcdb'
- uboot_path = self.fetch_asset(uboot_url, asset_hash=uboot_hash)
- uboot_path = archive.uncompress(uboot_path, self.workdir)
+ self.set_machine('gdbsim-r5f562n8')
+
+ uboot_path_gz = self.ASSET_UBOOT.fetch()
+ uboot_path = os.path.join(self.workdir, 'u-boot.bin')
+ gzip_uncompress(uboot_path_gz, uboot_path)
self.vm.set_console()
self.vm.add_args('-bios', uboot_path,
@@ -50,18 +58,11 @@ def test_uboot(self):
def test_linux_sash(self):
"""
Boots a Linux kernel and checks that the console is operational.
-
- :avocado: tags=arch:rx
- :avocado: tags=machine:gdbsim-r5f562n7
- :avocado: tags=endian:little
- :avocado: tags=flaky
"""
- dtb_url = ('https://acc.dl.osdn.jp/users/23/23887/rx-virt.dtb')
- dtb_hash = '7b4e4e2c71905da44e86ce47adee2210b026ac18'
- dtb_path = self.fetch_asset(dtb_url, asset_hash=dtb_hash)
- kernel_url = ('http://acc.dl.osdn.jp/users/23/23845/zImage')
- kernel_hash = '39a81067f8d72faad90866ddfefa19165d68fc99'
- kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+ self.set_machine('gdbsim-r5f562n7')
+
+ dtb_path = self.ASSET_DTB.fetch()
+ kernel_path = self.ASSET_KERNEL.fetch()
self.vm.set_console()
kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'earlycon'
@@ -72,3 +73,6 @@ def test_linux_sash(self):
wait_for_console_pattern(self, 'Sash command shell (version 1.1.1)',
failure_message='Kernel panic - not syncing')
exec_command_and_wait_for_pattern(self, 'printenv', 'TERM=linux')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH v4 30/35] tests/functional: Convert the linux_initrd avocado test into a standalone test
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (28 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 29/35] tests/functional: Convert the rx_gdbsim avocado " Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 8:27 ` [PATCH v4 31/35] gitlab-ci: Add "check-functional" to the build tests Thomas Huth
` (4 subsequent siblings)
34 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
While we're at it, update the SHA1 hashes to SHA256 hashes since
SHA1 should not be used anymore nowadays.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/functional/meson.build | 1 +
.../test_linux_initrd.py} | 37 +++++++++++--------
2 files changed, 22 insertions(+), 16 deletions(-)
rename tests/{avocado/linux_initrd.py => functional/test_linux_initrd.py} (76%)
mode change 100644 => 100755
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index e28aa34087..12e08e365b 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -105,6 +105,7 @@ tests_x86_64_quick = [
tests_x86_64_thorough = [
'acpi_bits',
+ 'linux_initrd',
'netdev_ethtool',
'virtio_gpu',
]
diff --git a/tests/avocado/linux_initrd.py b/tests/functional/test_linux_initrd.py
old mode 100644
new mode 100755
similarity index 76%
rename from tests/avocado/linux_initrd.py
rename to tests/functional/test_linux_initrd.py
index 7f47b98ae7..c71a59d4c9
--- a/tests/avocado/linux_initrd.py
+++ b/tests/functional/test_linux_initrd.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Linux initrd integration test.
#
# Copyright (c) 2018 Red Hat, Inc.
@@ -12,20 +14,27 @@
import logging
import tempfile
-from avocado_qemu import QemuSystemTest
-from avocado import skipUnless
+from qemu_test import QemuSystemTest, Asset
+from unittest import skipUnless
class LinuxInitrd(QemuSystemTest):
"""
Checks QEMU evaluates correctly the initrd file passed as -initrd option.
-
- :avocado: tags=arch:x86_64
- :avocado: tags=machine:pc
"""
timeout = 300
+ ASSET_F18_KERNEL = Asset(
+ ('https://archives.fedoraproject.org/pub/archive/fedora/linux/'
+ 'releases/18/Fedora/x86_64/os/images/pxeboot/vmlinuz'),
+ '1a27cb42559ce29237ac186699d063556ad69c8349d732bb1bd8d614e5a8cc2e')
+
+ ASSET_F28_KERNEL = Asset(
+ ('https://archives.fedoraproject.org/pub/archive/fedora/linux/'
+ 'releases/28/Everything/x86_64/os/images/pxeboot/vmlinuz'),
+ 'd05909c9d4a742a6fcc84dcc0361009e4611769619cc187a07107579a035f24e')
+
def test_with_2gib_file_should_exit_error_msg_with_linux_v3_6(self):
"""
Pretends to boot QEMU with an initrd file with size of 2GiB
@@ -33,10 +42,8 @@ def test_with_2gib_file_should_exit_error_msg_with_linux_v3_6(self):
Fedora-18 shipped with linux-3.6 which have not supported xloadflags
cannot support more than 2GiB initrd.
"""
- kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora/li'
- 'nux/releases/18/Fedora/x86_64/os/images/pxeboot/vmlinuz')
- kernel_hash = '41464f68efe42b9991250bed86c7081d2ccdbb21'
- kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+ self.set_machine('pc')
+ kernel_path = self.ASSET_F18_KERNEL.fetch()
max_size = 2 * (1024 ** 3) - 1
with tempfile.NamedTemporaryFile() as initrd:
@@ -56,16 +63,11 @@ def test_with_2gib_file_should_exit_error_msg_with_linux_v3_6(self):
@skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab')
def test_with_2gib_file_should_work_with_linux_v4_16(self):
"""
- :avocado: tags=flaky
-
QEMU has supported up to 4 GiB initrd for recent kernel
Expect guest can reach 'Unpacking initramfs...'
"""
- kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
- '/linux/releases/28/Everything/x86_64/os/images/pxeboot/'
- 'vmlinuz')
- kernel_hash = '238e083e114c48200f80d889f7e32eeb2793e02a'
- kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+ self.set_machine('pc')
+ kernel_path = self.ASSET_F28_KERNEL.fetch()
max_size = 2 * (1024 ** 3) + 1
with tempfile.NamedTemporaryFile() as initrd:
@@ -89,3 +91,6 @@ def test_with_2gib_file_should_work_with_linux_v4_16(self):
break
if 'Kernel panic - not syncing' in msg:
self.fail("Kernel panic reached")
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH v4 31/35] gitlab-ci: Add "check-functional" to the build tests
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (29 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 30/35] tests/functional: Convert the linux_initrd " Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 8:27 ` [PATCH v4 32/35] docs/devel: Split testing docs from the build docs and move to separate folder Thomas Huth
` (3 subsequent siblings)
34 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
Now that we converted many tests from the "check-avocado" test suite
to the "check-functional" test suite, we should make sure that these
also get tested in the CI.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
.gitlab-ci.d/buildtest-template.yml | 14 +++++--
.gitlab-ci.d/buildtest.yml | 60 ++++++++++++++---------------
2 files changed, 41 insertions(+), 33 deletions(-)
diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index 844c26623d..5f2fc7e6f4 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -59,6 +59,10 @@
- cd build
- find . -type f -exec touch {} +
# Avoid recompiling by hiding ninja with NINJA=":"
+ # We also have to pre-cache the functional tests manually in this case
+ - if [ "x${QEMU_TEST_CACHE_DIR}" != "x" ]; then
+ $MAKE precache-functional ;
+ fi
- $MAKE NINJA=":" $MAKE_CHECK_ARGS
.native_test_job_template:
@@ -72,12 +76,13 @@
reports:
junit: build/meson-logs/testlog.junit.xml
-.avocado_test_job_template:
+.functional_test_job_template:
extends: .common_test_job_template
cache:
key: "${CI_JOB_NAME}-cache"
paths:
- ${CI_PROJECT_DIR}/avocado-cache
+ - ${CI_PROJECT_DIR}/functional-cache
policy: pull-push
artifacts:
name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
@@ -86,6 +91,7 @@
paths:
- build/tests/results/latest/results.xml
- build/tests/results/latest/test-results
+ - build/tests/functional/*/*/*.log
reports:
junit: build/tests/results/latest/results.xml
before_script:
@@ -96,11 +102,13 @@
- echo -e '[job.output.testlogs]\nstatuses = ["FAIL", "INTERRUPT"]'
>> ~/.config/avocado/avocado.conf
- if [ -d ${CI_PROJECT_DIR}/avocado-cache ]; then
- du -chs ${CI_PROJECT_DIR}/avocado-cache ;
+ du -chs ${CI_PROJECT_DIR}/*-cache ;
fi
- export AVOCADO_ALLOW_UNTRUSTED_CODE=1
+ - export QEMU_TEST_ALLOW_UNTRUSTED_CODE=1
+ - export QEMU_TEST_CACHE_DIR=${CI_PROJECT_DIR}/functional-cache
after_script:
- cd build
- - du -chs ${CI_PROJECT_DIR}/avocado-cache
+ - du -chs ${CI_PROJECT_DIR}/*-cache
variables:
QEMU_JOB_AVOCADO: 1
diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index aa32782405..1d2afae996 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -22,14 +22,14 @@ check-system-alpine:
IMAGE: alpine
MAKE_CHECK_ARGS: check-unit check-qtest
-avocado-system-alpine:
- extends: .avocado_test_job_template
+functional-system-alpine:
+ extends: .functional_test_job_template
needs:
- job: build-system-alpine
artifacts: true
variables:
IMAGE: alpine
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
AVOCADO_TAGS: arch:avr arch:loongarch64 arch:mips64 arch:mipsel
build-system-ubuntu:
@@ -53,14 +53,14 @@ check-system-ubuntu:
IMAGE: ubuntu2204
MAKE_CHECK_ARGS: check
-avocado-system-ubuntu:
- extends: .avocado_test_job_template
+functional-system-ubuntu:
+ extends: .functional_test_job_template
needs:
- job: build-system-ubuntu
artifacts: true
variables:
IMAGE: ubuntu2204
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
AVOCADO_TAGS: arch:alpha arch:microblazeel arch:mips64el
build-system-debian:
@@ -85,14 +85,14 @@ check-system-debian:
IMAGE: debian
MAKE_CHECK_ARGS: check
-avocado-system-debian:
- extends: .avocado_test_job_template
+functional-system-debian:
+ extends: .functional_test_job_template
needs:
- job: build-system-debian
artifacts: true
variables:
IMAGE: debian
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
AVOCADO_TAGS: arch:arm arch:i386 arch:riscv64 arch:sh4 arch:sparc arch:xtensa
crash-test-debian:
@@ -129,14 +129,14 @@ check-system-fedora:
IMAGE: fedora
MAKE_CHECK_ARGS: check
-avocado-system-fedora:
- extends: .avocado_test_job_template
+functional-system-fedora:
+ extends: .functional_test_job_template
needs:
- job: build-system-fedora
artifacts: true
variables:
IMAGE: fedora
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
AVOCADO_TAGS: arch:microblaze arch:mips arch:xtensa arch:m68k
arch:riscv32 arch:ppc arch:sparc64
@@ -243,14 +243,14 @@ check-system-centos:
IMAGE: centos9
MAKE_CHECK_ARGS: check
-avocado-system-centos:
- extends: .avocado_test_job_template
+functional-system-centos:
+ extends: .functional_test_job_template
needs:
- job: build-system-centos
artifacts: true
variables:
IMAGE: centos9
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
AVOCADO_TAGS: arch:ppc64 arch:or1k arch:s390x arch:x86_64 arch:rx
arch:sh4
@@ -274,14 +274,14 @@ check-system-opensuse:
IMAGE: opensuse-leap
MAKE_CHECK_ARGS: check
-avocado-system-opensuse:
- extends: .avocado_test_job_template
+functional-system-opensuse:
+ extends: .functional_test_job_template
needs:
- job: build-system-opensuse
artifacts: true
variables:
IMAGE: opensuse-leap
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
AVOCADO_TAGS: arch:s390x arch:x86_64 arch:aarch64
#
@@ -302,15 +302,15 @@ build-system-flaky:
ppc64-softmmu rx-softmmu s390x-softmmu sh4-softmmu x86_64-softmmu
MAKE_CHECK_ARGS: check-build
-avocado-system-flaky:
- extends: .avocado_test_job_template
+functional-system-flaky:
+ extends: .functional_test_job_template
needs:
- job: build-system-flaky
artifacts: true
allow_failure: true
variables:
IMAGE: debian
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
QEMU_JOB_OPTIONAL: 1
QEMU_TEST_FLAKY_TESTS: 1
AVOCADO_TAGS: flaky
@@ -485,14 +485,14 @@ check-cfi-aarch64:
IMAGE: fedora
MAKE_CHECK_ARGS: check
-avocado-cfi-aarch64:
- extends: .avocado_test_job_template
+functional-cfi-aarch64:
+ extends: .functional_test_job_template
needs:
- job: build-cfi-aarch64
artifacts: true
variables:
IMAGE: fedora
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
build-cfi-ppc64-s390x:
extends:
@@ -523,14 +523,14 @@ check-cfi-ppc64-s390x:
IMAGE: fedora
MAKE_CHECK_ARGS: check
-avocado-cfi-ppc64-s390x:
- extends: .avocado_test_job_template
+functional-cfi-ppc64-s390x:
+ extends: .functional_test_job_template
needs:
- job: build-cfi-ppc64-s390x
artifacts: true
variables:
IMAGE: fedora
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
build-cfi-x86_64:
extends:
@@ -557,14 +557,14 @@ check-cfi-x86_64:
IMAGE: fedora
MAKE_CHECK_ARGS: check
-avocado-cfi-x86_64:
- extends: .avocado_test_job_template
+functional-cfi-x86_64:
+ extends: .functional_test_job_template
needs:
- job: build-cfi-x86_64
artifacts: true
variables:
IMAGE: fedora
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
tsan-build:
extends: .native_build_job_template
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH v4 32/35] docs/devel: Split testing docs from the build docs and move to separate folder
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (30 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 31/35] gitlab-ci: Add "check-functional" to the build tests Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-21 8:27 ` [PATCH v4 33/35] docs/devel/testing: Split the Avocado documentation into a separate file Thomas Huth
` (2 subsequent siblings)
34 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
Building and testing are two separate topics, so let's split the testing
into a separate category and move the related files into a separate folder.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
docs/devel/index-build.rst | 14 ++++----------
docs/devel/index.rst | 1 +
docs/devel/{ => testing}/acpi-bits.rst | 0
docs/devel/{ => testing}/ci-definitions.rst.inc | 0
docs/devel/{ => testing}/ci-jobs.rst.inc | 0
docs/devel/{ => testing}/ci-runners.rst.inc | 0
docs/devel/{ => testing}/ci.rst | 0
docs/devel/{ => testing}/fuzzing.rst | 0
docs/devel/testing/index.rst | 14 ++++++++++++++
docs/devel/{testing.rst => testing/main.rst} | 0
docs/devel/{ => testing}/qgraph.rst | 0
docs/devel/{ => testing}/qtest.rst | 0
12 files changed, 19 insertions(+), 10 deletions(-)
rename docs/devel/{ => testing}/acpi-bits.rst (100%)
rename docs/devel/{ => testing}/ci-definitions.rst.inc (100%)
rename docs/devel/{ => testing}/ci-jobs.rst.inc (100%)
rename docs/devel/{ => testing}/ci-runners.rst.inc (100%)
rename docs/devel/{ => testing}/ci.rst (100%)
rename docs/devel/{ => testing}/fuzzing.rst (100%)
create mode 100644 docs/devel/testing/index.rst
rename docs/devel/{testing.rst => testing/main.rst} (100%)
rename docs/devel/{ => testing}/qgraph.rst (100%)
rename docs/devel/{ => testing}/qtest.rst (100%)
diff --git a/docs/devel/index-build.rst b/docs/devel/index-build.rst
index 90b406ca0e..0023953be3 100644
--- a/docs/devel/index-build.rst
+++ b/docs/devel/index-build.rst
@@ -1,9 +1,8 @@
-QEMU Build and Test System
---------------------------
+QEMU Build System
+-----------------
-Details about how QEMU's build system works and how it is integrated
-into our testing infrastructure. You will need to understand some of
-the basics if you are adding new files and targets to the build.
+Details about how QEMU's build system works. You will need to understand
+some of the basics if you are adding new files and targets to the build.
.. toctree::
:maxdepth: 3
@@ -11,10 +10,5 @@ the basics if you are adding new files and targets to the build.
build-system
kconfig
docs
- testing
- acpi-bits
- qtest
- ci
qapi-code-gen
- fuzzing
control-flow-integrity
diff --git a/docs/devel/index.rst b/docs/devel/index.rst
index abf60457c2..a53f1bfda5 100644
--- a/docs/devel/index.rst
+++ b/docs/devel/index.rst
@@ -31,6 +31,7 @@ the :ref:`tcg_internals`.
index-process
index-build
+ testing/index
index-api
index-internals
index-tcg
diff --git a/docs/devel/acpi-bits.rst b/docs/devel/testing/acpi-bits.rst
similarity index 100%
rename from docs/devel/acpi-bits.rst
rename to docs/devel/testing/acpi-bits.rst
diff --git a/docs/devel/ci-definitions.rst.inc b/docs/devel/testing/ci-definitions.rst.inc
similarity index 100%
rename from docs/devel/ci-definitions.rst.inc
rename to docs/devel/testing/ci-definitions.rst.inc
diff --git a/docs/devel/ci-jobs.rst.inc b/docs/devel/testing/ci-jobs.rst.inc
similarity index 100%
rename from docs/devel/ci-jobs.rst.inc
rename to docs/devel/testing/ci-jobs.rst.inc
diff --git a/docs/devel/ci-runners.rst.inc b/docs/devel/testing/ci-runners.rst.inc
similarity index 100%
rename from docs/devel/ci-runners.rst.inc
rename to docs/devel/testing/ci-runners.rst.inc
diff --git a/docs/devel/ci.rst b/docs/devel/testing/ci.rst
similarity index 100%
rename from docs/devel/ci.rst
rename to docs/devel/testing/ci.rst
diff --git a/docs/devel/fuzzing.rst b/docs/devel/testing/fuzzing.rst
similarity index 100%
rename from docs/devel/fuzzing.rst
rename to docs/devel/testing/fuzzing.rst
diff --git a/docs/devel/testing/index.rst b/docs/devel/testing/index.rst
new file mode 100644
index 0000000000..2711fd78b7
--- /dev/null
+++ b/docs/devel/testing/index.rst
@@ -0,0 +1,14 @@
+Testing QEMU
+------------
+
+Details about how to test QEMU and how it is integrated into our CI
+testing infrastructure.
+
+.. toctree::
+ :maxdepth: 3
+
+ main
+ qtest
+ acpi-bits
+ ci
+ fuzzing
diff --git a/docs/devel/testing.rst b/docs/devel/testing/main.rst
similarity index 100%
rename from docs/devel/testing.rst
rename to docs/devel/testing/main.rst
diff --git a/docs/devel/qgraph.rst b/docs/devel/testing/qgraph.rst
similarity index 100%
rename from docs/devel/qgraph.rst
rename to docs/devel/testing/qgraph.rst
diff --git a/docs/devel/qtest.rst b/docs/devel/testing/qtest.rst
similarity index 100%
rename from docs/devel/qtest.rst
rename to docs/devel/testing/qtest.rst
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH v4 33/35] docs/devel/testing: Split the Avocado documentation into a separate file
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (31 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 32/35] docs/devel: Split testing docs from the build docs and move to separate folder Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-29 10:18 ` Daniel P. Berrangé
2024-08-21 8:27 ` [PATCH v4 34/35] docs/devel/testing: Rename avocado_qemu.Test class Thomas Huth
2024-08-21 8:27 ` [PATCH v4 35/35] docs/devel/testing: Add documentation for functional tests Thomas Huth
34 siblings, 1 reply; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
The main testing documentation file got very overloaded already.
Thus let's split the Avocado information into a separate file.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
docs/devel/testing/avocado.rst | 581 +++++++++++++++++++++++++++++++++
docs/devel/testing/index.rst | 1 +
docs/devel/testing/main.rst | 565 +-------------------------------
3 files changed, 583 insertions(+), 564 deletions(-)
create mode 100644 docs/devel/testing/avocado.rst
diff --git a/docs/devel/testing/avocado.rst b/docs/devel/testing/avocado.rst
new file mode 100644
index 0000000000..0d207a05c5
--- /dev/null
+++ b/docs/devel/testing/avocado.rst
@@ -0,0 +1,581 @@
+.. _checkavocado-ref:
+
+
+Integration testing with Avocado
+================================
+
+The ``tests/avocado`` directory hosts integration tests. They're usually
+higher level tests, and may interact with external resources and with
+various guest operating systems.
+
+These tests are written using the Avocado Testing Framework (which must
+be installed separately) in conjunction with a the ``avocado_qemu.Test``
+class, implemented at ``tests/avocado/avocado_qemu``.
+
+Tests based on ``avocado_qemu.Test`` can easily:
+
+ * Customize the command line arguments given to the convenience
+ ``self.vm`` attribute (a QEMUMachine instance)
+
+ * Interact with the QEMU monitor, send QMP commands and check
+ their results
+
+ * Interact with the guest OS, using the convenience console device
+ (which may be useful to assert the effectiveness and correctness of
+ command line arguments or QMP commands)
+
+ * Interact with external data files that accompany the test itself
+ (see ``self.get_data()``)
+
+ * Download (and cache) remote data files, such as firmware and kernel
+ images
+
+ * Have access to a library of guest OS images (by means of the
+ ``avocado.utils.vmimage`` library)
+
+ * Make use of various other test related utilities available at the
+ test class itself and at the utility library:
+
+ - http://avocado-framework.readthedocs.io/en/latest/api/test/avocado.html#avocado.Test
+ - http://avocado-framework.readthedocs.io/en/latest/api/utils/avocado.utils.html
+
+Running tests
+-------------
+
+You can run the avocado tests simply by executing:
+
+.. code::
+
+ make check-avocado
+
+This involves the automatic installation, from PyPI, of all the
+necessary avocado-framework dependencies into the QEMU venv within the
+build tree (at ``./pyvenv``). Test results are also saved within the
+build tree (at ``tests/results``).
+
+Note: the build environment must be using a Python 3 stack, and have
+the ``venv`` and ``pip`` packages installed. If necessary, make sure
+``configure`` is called with ``--python=`` and that those modules are
+available. On Debian and Ubuntu based systems, depending on the
+specific version, they may be on packages named ``python3-venv`` and
+``python3-pip``.
+
+It is also possible to run tests based on tags using the
+``make check-avocado`` command and the ``AVOCADO_TAGS`` environment
+variable:
+
+.. code::
+
+ make check-avocado AVOCADO_TAGS=quick
+
+Note that tags separated with commas have an AND behavior, while tags
+separated by spaces have an OR behavior. For more information on Avocado
+tags, see:
+
+ https://avocado-framework.readthedocs.io/en/latest/guides/user/chapters/tags.html
+
+To run a single test file, a couple of them, or a test within a file
+using the ``make check-avocado`` command, set the ``AVOCADO_TESTS``
+environment variable with the test files or test names. To run all
+tests from a single file, use:
+
+ .. code::
+
+ make check-avocado AVOCADO_TESTS=$FILEPATH
+
+The same is valid to run tests from multiple test files:
+
+ .. code::
+
+ make check-avocado AVOCADO_TESTS='$FILEPATH1 $FILEPATH2'
+
+To run a single test within a file, use:
+
+ .. code::
+
+ make check-avocado AVOCADO_TESTS=$FILEPATH:$TESTCLASS.$TESTNAME
+
+The same is valid to run single tests from multiple test files:
+
+ .. code::
+
+ make check-avocado AVOCADO_TESTS='$FILEPATH1:$TESTCLASS1.$TESTNAME1 $FILEPATH2:$TESTCLASS2.$TESTNAME2'
+
+The scripts installed inside the virtual environment may be used
+without an "activation". For instance, the Avocado test runner
+may be invoked by running:
+
+ .. code::
+
+ pyvenv/bin/avocado run $OPTION1 $OPTION2 tests/avocado/
+
+Note that if ``make check-avocado`` was not executed before, it is
+possible to create the Python virtual environment with the dependencies
+needed running:
+
+ .. code::
+
+ make check-venv
+
+It is also possible to run tests from a single file or a single test within
+a test file. To run tests from a single file within the build tree, use:
+
+ .. code::
+
+ pyvenv/bin/avocado run tests/avocado/$TESTFILE
+
+To run a single test within a test file, use:
+
+ .. code::
+
+ pyvenv/bin/avocado run tests/avocado/$TESTFILE:$TESTCLASS.$TESTNAME
+
+Valid test names are visible in the output from any previous execution
+of Avocado or ``make check-avocado``, and can also be queried using:
+
+ .. code::
+
+ pyvenv/bin/avocado list tests/avocado
+
+Manual Installation
+-------------------
+
+To manually install Avocado and its dependencies, run:
+
+.. code::
+
+ pip install --user avocado-framework
+
+Alternatively, follow the instructions on this link:
+
+ https://avocado-framework.readthedocs.io/en/latest/guides/user/chapters/installing.html
+
+Overview
+--------
+
+The ``tests/avocado/avocado_qemu`` directory provides the
+``avocado_qemu`` Python module, containing the ``avocado_qemu.Test``
+class. Here's a simple usage example:
+
+.. code::
+
+ from avocado_qemu import QemuSystemTest
+
+
+ class Version(QemuSystemTest):
+ """
+ :avocado: tags=quick
+ """
+ def test_qmp_human_info_version(self):
+ self.vm.launch()
+ res = self.vm.cmd('human-monitor-command',
+ command_line='info version')
+ self.assertRegex(res, r'^(\d+\.\d+\.\d)')
+
+To execute your test, run:
+
+.. code::
+
+ avocado run version.py
+
+Tests may be classified according to a convention by using docstring
+directives such as ``:avocado: tags=TAG1,TAG2``. To run all tests
+in the current directory, tagged as "quick", run:
+
+.. code::
+
+ avocado run -t quick .
+
+The ``avocado_qemu.Test`` base test class
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The ``avocado_qemu.Test`` class has a number of characteristics that
+are worth being mentioned right away.
+
+First of all, it attempts to give each test a ready to use QEMUMachine
+instance, available at ``self.vm``. Because many tests will tweak the
+QEMU command line, launching the QEMUMachine (by using ``self.vm.launch()``)
+is left to the test writer.
+
+The base test class has also support for tests with more than one
+QEMUMachine. The way to get machines is through the ``self.get_vm()``
+method which will return a QEMUMachine instance. The ``self.get_vm()``
+method accepts arguments that will be passed to the QEMUMachine creation
+and also an optional ``name`` attribute so you can identify a specific
+machine and get it more than once through the tests methods. A simple
+and hypothetical example follows:
+
+.. code::
+
+ from avocado_qemu import QemuSystemTest
+
+
+ class MultipleMachines(QemuSystemTest):
+ def test_multiple_machines(self):
+ first_machine = self.get_vm()
+ second_machine = self.get_vm()
+ self.get_vm(name='third_machine').launch()
+
+ first_machine.launch()
+ second_machine.launch()
+
+ first_res = first_machine.cmd(
+ 'human-monitor-command',
+ command_line='info version')
+
+ second_res = second_machine.cmd(
+ 'human-monitor-command',
+ command_line='info version')
+
+ third_res = self.get_vm(name='third_machine').cmd(
+ 'human-monitor-command',
+ command_line='info version')
+
+ self.assertEqual(first_res, second_res, third_res)
+
+At test "tear down", ``avocado_qemu.Test`` handles all the QEMUMachines
+shutdown.
+
+The ``avocado_qemu.LinuxTest`` base test class
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The ``avocado_qemu.LinuxTest`` is further specialization of the
+``avocado_qemu.Test`` class, so it contains all the characteristics of
+the later plus some extra features.
+
+First of all, this base class is intended for tests that need to
+interact with a fully booted and operational Linux guest. At this
+time, it uses a Fedora 31 guest image. The most basic example looks
+like this:
+
+.. code::
+
+ from avocado_qemu import LinuxTest
+
+
+ class SomeTest(LinuxTest):
+
+ def test(self):
+ self.launch_and_wait()
+ self.ssh_command('some_command_to_be_run_in_the_guest')
+
+Please refer to tests that use ``avocado_qemu.LinuxTest`` under
+``tests/avocado`` for more examples.
+
+QEMUMachine
+-----------
+
+The QEMUMachine API is already widely used in the Python iotests,
+device-crash-test and other Python scripts. It's a wrapper around the
+execution of a QEMU binary, giving its users:
+
+ * the ability to set command line arguments to be given to the QEMU
+ binary
+
+ * a ready to use QMP connection and interface, which can be used to
+ send commands and inspect its results, as well as asynchronous
+ events
+
+ * convenience methods to set commonly used command line arguments in
+ a more succinct and intuitive way
+
+QEMU binary selection
+^^^^^^^^^^^^^^^^^^^^^
+
+The QEMU binary used for the ``self.vm`` QEMUMachine instance will
+primarily depend on the value of the ``qemu_bin`` parameter. If it's
+not explicitly set, its default value will be the result of a dynamic
+probe in the same source tree. A suitable binary will be one that
+targets the architecture matching host machine.
+
+Based on this description, test writers will usually rely on one of
+the following approaches:
+
+1) Set ``qemu_bin``, and use the given binary
+
+2) Do not set ``qemu_bin``, and use a QEMU binary named like
+ "qemu-system-${arch}", either in the current
+ working directory, or in the current source tree.
+
+The resulting ``qemu_bin`` value will be preserved in the
+``avocado_qemu.Test`` as an attribute with the same name.
+
+Attribute reference
+-------------------
+
+Test
+^^^^
+
+Besides the attributes and methods that are part of the base
+``avocado.Test`` class, the following attributes are available on any
+``avocado_qemu.Test`` instance.
+
+vm
+""
+
+A QEMUMachine instance, initially configured according to the given
+``qemu_bin`` parameter.
+
+arch
+""""
+
+The architecture can be used on different levels of the stack, e.g. by
+the framework or by the test itself. At the framework level, it will
+currently influence the selection of a QEMU binary (when one is not
+explicitly given).
+
+Tests are also free to use this attribute value, for their own needs.
+A test may, for instance, use the same value when selecting the
+architecture of a kernel or disk image to boot a VM with.
+
+The ``arch`` attribute will be set to the test parameter of the same
+name. If one is not given explicitly, it will either be set to
+``None``, or, if the test is tagged with one (and only one)
+``:avocado: tags=arch:VALUE`` tag, it will be set to ``VALUE``.
+
+cpu
+"""
+
+The cpu model that will be set to all QEMUMachine instances created
+by the test.
+
+The ``cpu`` attribute will be set to the test parameter of the same
+name. If one is not given explicitly, it will either be set to
+``None ``, or, if the test is tagged with one (and only one)
+``:avocado: tags=cpu:VALUE`` tag, it will be set to ``VALUE``.
+
+machine
+"""""""
+
+The machine type that will be set to all QEMUMachine instances created
+by the test.
+
+The ``machine`` attribute will be set to the test parameter of the same
+name. If one is not given explicitly, it will either be set to
+``None``, or, if the test is tagged with one (and only one)
+``:avocado: tags=machine:VALUE`` tag, it will be set to ``VALUE``.
+
+qemu_bin
+""""""""
+
+The preserved value of the ``qemu_bin`` parameter or the result of the
+dynamic probe for a QEMU binary in the current working directory or
+source tree.
+
+LinuxTest
+^^^^^^^^^
+
+Besides the attributes present on the ``avocado_qemu.Test`` base
+class, the ``avocado_qemu.LinuxTest`` adds the following attributes:
+
+distro
+""""""
+
+The name of the Linux distribution used as the guest image for the
+test. The name should match the **Provider** column on the list
+of images supported by the avocado.utils.vmimage library:
+
+https://avocado-framework.readthedocs.io/en/latest/guides/writer/libs/vmimage.html#supported-images
+
+distro_version
+""""""""""""""
+
+The version of the Linux distribution as the guest image for the
+test. The name should match the **Version** column on the list
+of images supported by the avocado.utils.vmimage library:
+
+https://avocado-framework.readthedocs.io/en/latest/guides/writer/libs/vmimage.html#supported-images
+
+distro_checksum
+"""""""""""""""
+
+The sha256 hash of the guest image file used for the test.
+
+If this value is not set in the code or by a test parameter (with the
+same name), no validation on the integrity of the image will be
+performed.
+
+Parameter reference
+-------------------
+
+To understand how Avocado parameters are accessed by tests, and how
+they can be passed to tests, please refer to::
+
+ https://avocado-framework.readthedocs.io/en/latest/guides/writer/chapters/writing.html#accessing-test-parameters
+
+Parameter values can be easily seen in the log files, and will look
+like the following:
+
+.. code::
+
+ PARAMS (key=qemu_bin, path=*, default=./qemu-system-x86_64) => './qemu-system-x86_64
+
+Test
+^^^^
+
+arch
+""""
+
+The architecture that will influence the selection of a QEMU binary
+(when one is not explicitly given).
+
+Tests are also free to use this parameter value, for their own needs.
+A test may, for instance, use the same value when selecting the
+architecture of a kernel or disk image to boot a VM with.
+
+This parameter has a direct relation with the ``arch`` attribute. If
+not given, it will default to None.
+
+cpu
+"""
+
+The cpu model that will be set to all QEMUMachine instances created
+by the test.
+
+machine
+"""""""
+
+The machine type that will be set to all QEMUMachine instances created
+by the test.
+
+qemu_bin
+""""""""
+
+The exact QEMU binary to be used on QEMUMachine.
+
+LinuxTest
+^^^^^^^^^
+
+Besides the parameters present on the ``avocado_qemu.Test`` base
+class, the ``avocado_qemu.LinuxTest`` adds the following parameters:
+
+distro
+""""""
+
+The name of the Linux distribution used as the guest image for the
+test. The name should match the **Provider** column on the list
+of images supported by the avocado.utils.vmimage library:
+
+https://avocado-framework.readthedocs.io/en/latest/guides/writer/libs/vmimage.html#supported-images
+
+distro_version
+""""""""""""""
+
+The version of the Linux distribution as the guest image for the
+test. The name should match the **Version** column on the list
+of images supported by the avocado.utils.vmimage library:
+
+https://avocado-framework.readthedocs.io/en/latest/guides/writer/libs/vmimage.html#supported-images
+
+distro_checksum
+"""""""""""""""
+
+The sha256 hash of the guest image file used for the test.
+
+If this value is not set in the code or by this parameter no
+validation on the integrity of the image will be performed.
+
+Skipping tests
+--------------
+
+The Avocado framework provides Python decorators which allow for easily skip
+tests running under certain conditions. For example, on the lack of a binary
+on the test system or when the running environment is a CI system. For further
+information about those decorators, please refer to::
+
+ https://avocado-framework.readthedocs.io/en/latest/guides/writer/chapters/writing.html#skipping-tests
+
+While the conditions for skipping tests are often specifics of each one, there
+are recurring scenarios identified by the QEMU developers and the use of
+environment variables became a kind of standard way to enable/disable tests.
+
+Here is a list of the most used variables:
+
+AVOCADO_ALLOW_LARGE_STORAGE
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Tests which are going to fetch or produce assets considered *large* are not
+going to run unless that ``AVOCADO_ALLOW_LARGE_STORAGE=1`` is exported on
+the environment.
+
+The definition of *large* is a bit arbitrary here, but it usually means an
+asset which occupies at least 1GB of size on disk when uncompressed.
+
+SPEED
+^^^^^
+Tests which have a long runtime will not be run unless ``SPEED=slow`` is
+exported on the environment.
+
+The definition of *long* is a bit arbitrary here, and it depends on the
+usefulness of the test too. A unique test is worth spending more time on,
+small variations on existing tests perhaps less so. As a rough guide,
+a test or set of similar tests which take more than 100 seconds to
+complete.
+
+AVOCADO_ALLOW_UNTRUSTED_CODE
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+There are tests which will boot a kernel image or firmware that can be
+considered not safe to run on the developer's workstation, thus they are
+skipped by default. The definition of *not safe* is also arbitrary but
+usually it means a blob which either its source or build process aren't
+public available.
+
+You should export ``AVOCADO_ALLOW_UNTRUSTED_CODE=1`` on the environment in
+order to allow tests which make use of those kind of assets.
+
+AVOCADO_TIMEOUT_EXPECTED
+^^^^^^^^^^^^^^^^^^^^^^^^
+The Avocado framework has a timeout mechanism which interrupts tests to avoid the
+test suite of getting stuck. The timeout value can be set via test parameter or
+property defined in the test class, for further details::
+
+ https://avocado-framework.readthedocs.io/en/latest/guides/writer/chapters/writing.html#setting-a-test-timeout
+
+Even though the timeout can be set by the test developer, there are some tests
+that may not have a well-defined limit of time to finish under certain
+conditions. For example, tests that take longer to execute when QEMU is
+compiled with debug flags. Therefore, the ``AVOCADO_TIMEOUT_EXPECTED`` variable
+has been used to determine whether those tests should run or not.
+
+QEMU_TEST_FLAKY_TESTS
+^^^^^^^^^^^^^^^^^^^^^
+Some tests are not working reliably and thus are disabled by default.
+This includes tests that don't run reliably on GitLab's CI which
+usually expose real issues that are rarely seen on developer machines
+due to the constraints of the CI environment. If you encounter a
+similar situation then raise a bug and then mark the test as shown on
+the code snippet below:
+
+.. code::
+
+ # See https://gitlab.com/qemu-project/qemu/-/issues/nnnn
+ @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab')
+ def test(self):
+ do_something()
+
+You can also add ``:avocado: tags=flaky`` to the test meta-data so
+only the flaky tests can be run as a group:
+
+.. code::
+
+ env QEMU_TEST_FLAKY_TESTS=1 ./pyvenv/bin/avocado \
+ run tests/avocado -filter-by-tags=flaky
+
+Tests should not live in this state forever and should either be fixed
+or eventually removed.
+
+
+Uninstalling Avocado
+--------------------
+
+If you've followed the manual installation instructions above, you can
+easily uninstall Avocado. Start by listing the packages you have
+installed::
+
+ pip list --user
+
+And remove any package you want with::
+
+ pip uninstall <package_name>
+
+If you've used ``make check-avocado``, the Python virtual environment where
+Avocado is installed will be cleaned up as part of ``make check-clean``.
diff --git a/docs/devel/testing/index.rst b/docs/devel/testing/index.rst
index 2711fd78b7..cdf7ba1f8b 100644
--- a/docs/devel/testing/index.rst
+++ b/docs/devel/testing/index.rst
@@ -9,6 +9,7 @@ testing infrastructure.
main
qtest
+ avocado
acpi-bits
ci
fuzzing
diff --git a/docs/devel/testing/main.rst b/docs/devel/testing/main.rst
index af73d3d64f..39b965ecf6 100644
--- a/docs/devel/testing/main.rst
+++ b/docs/devel/testing/main.rst
@@ -869,577 +869,14 @@ The ``tests/avocado`` directory hosts integration tests. They're usually
higher level tests, and may interact with external resources and with
various guest operating systems.
-These tests are written using the Avocado Testing Framework (which must
-be installed separately) in conjunction with a the ``avocado_qemu.Test``
-class, implemented at ``tests/avocado/avocado_qemu``.
-
-Tests based on ``avocado_qemu.Test`` can easily:
-
- * Customize the command line arguments given to the convenience
- ``self.vm`` attribute (a QEMUMachine instance)
-
- * Interact with the QEMU monitor, send QMP commands and check
- their results
-
- * Interact with the guest OS, using the convenience console device
- (which may be useful to assert the effectiveness and correctness of
- command line arguments or QMP commands)
-
- * Interact with external data files that accompany the test itself
- (see ``self.get_data()``)
-
- * Download (and cache) remote data files, such as firmware and kernel
- images
-
- * Have access to a library of guest OS images (by means of the
- ``avocado.utils.vmimage`` library)
-
- * Make use of various other test related utilities available at the
- test class itself and at the utility library:
-
- - http://avocado-framework.readthedocs.io/en/latest/api/test/avocado.html#avocado.Test
- - http://avocado-framework.readthedocs.io/en/latest/api/utils/avocado.utils.html
-
-Running tests
-~~~~~~~~~~~~~
-
You can run the avocado tests simply by executing:
.. code::
make check-avocado
-This involves the automatic installation, from PyPI, of all the
-necessary avocado-framework dependencies into the QEMU venv within the
-build tree (at ``./pyvenv``). Test results are also saved within the
-build tree (at ``tests/results``).
-
-Note: the build environment must be using a Python 3 stack, and have
-the ``venv`` and ``pip`` packages installed. If necessary, make sure
-``configure`` is called with ``--python=`` and that those modules are
-available. On Debian and Ubuntu based systems, depending on the
-specific version, they may be on packages named ``python3-venv`` and
-``python3-pip``.
-
-It is also possible to run tests based on tags using the
-``make check-avocado`` command and the ``AVOCADO_TAGS`` environment
-variable:
-
-.. code::
-
- make check-avocado AVOCADO_TAGS=quick
-
-Note that tags separated with commas have an AND behavior, while tags
-separated by spaces have an OR behavior. For more information on Avocado
-tags, see:
-
- https://avocado-framework.readthedocs.io/en/latest/guides/user/chapters/tags.html
-
-To run a single test file, a couple of them, or a test within a file
-using the ``make check-avocado`` command, set the ``AVOCADO_TESTS``
-environment variable with the test files or test names. To run all
-tests from a single file, use:
-
- .. code::
-
- make check-avocado AVOCADO_TESTS=$FILEPATH
-
-The same is valid to run tests from multiple test files:
-
- .. code::
-
- make check-avocado AVOCADO_TESTS='$FILEPATH1 $FILEPATH2'
-
-To run a single test within a file, use:
-
- .. code::
-
- make check-avocado AVOCADO_TESTS=$FILEPATH:$TESTCLASS.$TESTNAME
-
-The same is valid to run single tests from multiple test files:
-
- .. code::
-
- make check-avocado AVOCADO_TESTS='$FILEPATH1:$TESTCLASS1.$TESTNAME1 $FILEPATH2:$TESTCLASS2.$TESTNAME2'
-
-The scripts installed inside the virtual environment may be used
-without an "activation". For instance, the Avocado test runner
-may be invoked by running:
-
- .. code::
-
- pyvenv/bin/avocado run $OPTION1 $OPTION2 tests/avocado/
-
-Note that if ``make check-avocado`` was not executed before, it is
-possible to create the Python virtual environment with the dependencies
-needed running:
-
- .. code::
-
- make check-venv
-
-It is also possible to run tests from a single file or a single test within
-a test file. To run tests from a single file within the build tree, use:
-
- .. code::
-
- pyvenv/bin/avocado run tests/avocado/$TESTFILE
-
-To run a single test within a test file, use:
-
- .. code::
-
- pyvenv/bin/avocado run tests/avocado/$TESTFILE:$TESTCLASS.$TESTNAME
-
-Valid test names are visible in the output from any previous execution
-of Avocado or ``make check-avocado``, and can also be queried using:
-
- .. code::
-
- pyvenv/bin/avocado list tests/avocado
-
-Manual Installation
-~~~~~~~~~~~~~~~~~~~
-
-To manually install Avocado and its dependencies, run:
-
-.. code::
-
- pip install --user avocado-framework
-
-Alternatively, follow the instructions on this link:
-
- https://avocado-framework.readthedocs.io/en/latest/guides/user/chapters/installing.html
-
-Overview
-~~~~~~~~
-
-The ``tests/avocado/avocado_qemu`` directory provides the
-``avocado_qemu`` Python module, containing the ``avocado_qemu.Test``
-class. Here's a simple usage example:
-
-.. code::
-
- from avocado_qemu import QemuSystemTest
-
-
- class Version(QemuSystemTest):
- """
- :avocado: tags=quick
- """
- def test_qmp_human_info_version(self):
- self.vm.launch()
- res = self.vm.cmd('human-monitor-command',
- command_line='info version')
- self.assertRegex(res, r'^(\d+\.\d+\.\d)')
-
-To execute your test, run:
-
-.. code::
-
- avocado run version.py
-
-Tests may be classified according to a convention by using docstring
-directives such as ``:avocado: tags=TAG1,TAG2``. To run all tests
-in the current directory, tagged as "quick", run:
-
-.. code::
-
- avocado run -t quick .
-
-The ``avocado_qemu.Test`` base test class
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-The ``avocado_qemu.Test`` class has a number of characteristics that
-are worth being mentioned right away.
-
-First of all, it attempts to give each test a ready to use QEMUMachine
-instance, available at ``self.vm``. Because many tests will tweak the
-QEMU command line, launching the QEMUMachine (by using ``self.vm.launch()``)
-is left to the test writer.
-
-The base test class has also support for tests with more than one
-QEMUMachine. The way to get machines is through the ``self.get_vm()``
-method which will return a QEMUMachine instance. The ``self.get_vm()``
-method accepts arguments that will be passed to the QEMUMachine creation
-and also an optional ``name`` attribute so you can identify a specific
-machine and get it more than once through the tests methods. A simple
-and hypothetical example follows:
-
-.. code::
-
- from avocado_qemu import QemuSystemTest
-
-
- class MultipleMachines(QemuSystemTest):
- def test_multiple_machines(self):
- first_machine = self.get_vm()
- second_machine = self.get_vm()
- self.get_vm(name='third_machine').launch()
-
- first_machine.launch()
- second_machine.launch()
-
- first_res = first_machine.cmd(
- 'human-monitor-command',
- command_line='info version')
-
- second_res = second_machine.cmd(
- 'human-monitor-command',
- command_line='info version')
-
- third_res = self.get_vm(name='third_machine').cmd(
- 'human-monitor-command',
- command_line='info version')
-
- self.assertEqual(first_res, second_res, third_res)
-
-At test "tear down", ``avocado_qemu.Test`` handles all the QEMUMachines
-shutdown.
-
-The ``avocado_qemu.LinuxTest`` base test class
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-The ``avocado_qemu.LinuxTest`` is further specialization of the
-``avocado_qemu.Test`` class, so it contains all the characteristics of
-the later plus some extra features.
-
-First of all, this base class is intended for tests that need to
-interact with a fully booted and operational Linux guest. At this
-time, it uses a Fedora 31 guest image. The most basic example looks
-like this:
-
-.. code::
-
- from avocado_qemu import LinuxTest
-
-
- class SomeTest(LinuxTest):
-
- def test(self):
- self.launch_and_wait()
- self.ssh_command('some_command_to_be_run_in_the_guest')
-
-Please refer to tests that use ``avocado_qemu.LinuxTest`` under
-``tests/avocado`` for more examples.
-
-QEMUMachine
-~~~~~~~~~~~
-
-The QEMUMachine API is already widely used in the Python iotests,
-device-crash-test and other Python scripts. It's a wrapper around the
-execution of a QEMU binary, giving its users:
-
- * the ability to set command line arguments to be given to the QEMU
- binary
-
- * a ready to use QMP connection and interface, which can be used to
- send commands and inspect its results, as well as asynchronous
- events
-
- * convenience methods to set commonly used command line arguments in
- a more succinct and intuitive way
-
-QEMU binary selection
-^^^^^^^^^^^^^^^^^^^^^
-
-The QEMU binary used for the ``self.vm`` QEMUMachine instance will
-primarily depend on the value of the ``qemu_bin`` parameter. If it's
-not explicitly set, its default value will be the result of a dynamic
-probe in the same source tree. A suitable binary will be one that
-targets the architecture matching host machine.
-
-Based on this description, test writers will usually rely on one of
-the following approaches:
-
-1) Set ``qemu_bin``, and use the given binary
-
-2) Do not set ``qemu_bin``, and use a QEMU binary named like
- "qemu-system-${arch}", either in the current
- working directory, or in the current source tree.
-
-The resulting ``qemu_bin`` value will be preserved in the
-``avocado_qemu.Test`` as an attribute with the same name.
-
-Attribute reference
-~~~~~~~~~~~~~~~~~~~
-
-Test
-^^^^
-
-Besides the attributes and methods that are part of the base
-``avocado.Test`` class, the following attributes are available on any
-``avocado_qemu.Test`` instance.
-
-vm
-''
-
-A QEMUMachine instance, initially configured according to the given
-``qemu_bin`` parameter.
-
-arch
-''''
-
-The architecture can be used on different levels of the stack, e.g. by
-the framework or by the test itself. At the framework level, it will
-currently influence the selection of a QEMU binary (when one is not
-explicitly given).
-
-Tests are also free to use this attribute value, for their own needs.
-A test may, for instance, use the same value when selecting the
-architecture of a kernel or disk image to boot a VM with.
-
-The ``arch`` attribute will be set to the test parameter of the same
-name. If one is not given explicitly, it will either be set to
-``None``, or, if the test is tagged with one (and only one)
-``:avocado: tags=arch:VALUE`` tag, it will be set to ``VALUE``.
-
-cpu
-'''
-
-The cpu model that will be set to all QEMUMachine instances created
-by the test.
-
-The ``cpu`` attribute will be set to the test parameter of the same
-name. If one is not given explicitly, it will either be set to
-``None ``, or, if the test is tagged with one (and only one)
-``:avocado: tags=cpu:VALUE`` tag, it will be set to ``VALUE``.
-
-machine
-'''''''
-
-The machine type that will be set to all QEMUMachine instances created
-by the test.
-
-The ``machine`` attribute will be set to the test parameter of the same
-name. If one is not given explicitly, it will either be set to
-``None``, or, if the test is tagged with one (and only one)
-``:avocado: tags=machine:VALUE`` tag, it will be set to ``VALUE``.
-
-qemu_bin
-''''''''
-
-The preserved value of the ``qemu_bin`` parameter or the result of the
-dynamic probe for a QEMU binary in the current working directory or
-source tree.
-
-LinuxTest
-^^^^^^^^^
-
-Besides the attributes present on the ``avocado_qemu.Test`` base
-class, the ``avocado_qemu.LinuxTest`` adds the following attributes:
-
-distro
-''''''
-
-The name of the Linux distribution used as the guest image for the
-test. The name should match the **Provider** column on the list
-of images supported by the avocado.utils.vmimage library:
-
-https://avocado-framework.readthedocs.io/en/latest/guides/writer/libs/vmimage.html#supported-images
-
-distro_version
-''''''''''''''
-
-The version of the Linux distribution as the guest image for the
-test. The name should match the **Version** column on the list
-of images supported by the avocado.utils.vmimage library:
-
-https://avocado-framework.readthedocs.io/en/latest/guides/writer/libs/vmimage.html#supported-images
-
-distro_checksum
-'''''''''''''''
-
-The sha256 hash of the guest image file used for the test.
-
-If this value is not set in the code or by a test parameter (with the
-same name), no validation on the integrity of the image will be
-performed.
-
-Parameter reference
-~~~~~~~~~~~~~~~~~~~
-
-To understand how Avocado parameters are accessed by tests, and how
-they can be passed to tests, please refer to::
-
- https://avocado-framework.readthedocs.io/en/latest/guides/writer/chapters/writing.html#accessing-test-parameters
-
-Parameter values can be easily seen in the log files, and will look
-like the following:
-
-.. code::
-
- PARAMS (key=qemu_bin, path=*, default=./qemu-system-x86_64) => './qemu-system-x86_64
-
-Test
-^^^^
-
-arch
-''''
-
-The architecture that will influence the selection of a QEMU binary
-(when one is not explicitly given).
-
-Tests are also free to use this parameter value, for their own needs.
-A test may, for instance, use the same value when selecting the
-architecture of a kernel or disk image to boot a VM with.
-
-This parameter has a direct relation with the ``arch`` attribute. If
-not given, it will default to None.
-
-cpu
-'''
-
-The cpu model that will be set to all QEMUMachine instances created
-by the test.
-
-machine
-'''''''
-
-The machine type that will be set to all QEMUMachine instances created
-by the test.
-
-qemu_bin
-''''''''
-
-The exact QEMU binary to be used on QEMUMachine.
-
-LinuxTest
-^^^^^^^^^
-
-Besides the parameters present on the ``avocado_qemu.Test`` base
-class, the ``avocado_qemu.LinuxTest`` adds the following parameters:
-
-distro
-''''''
-
-The name of the Linux distribution used as the guest image for the
-test. The name should match the **Provider** column on the list
-of images supported by the avocado.utils.vmimage library:
-
-https://avocado-framework.readthedocs.io/en/latest/guides/writer/libs/vmimage.html#supported-images
-
-distro_version
-''''''''''''''
-
-The version of the Linux distribution as the guest image for the
-test. The name should match the **Version** column on the list
-of images supported by the avocado.utils.vmimage library:
-
-https://avocado-framework.readthedocs.io/en/latest/guides/writer/libs/vmimage.html#supported-images
-
-distro_checksum
-'''''''''''''''
-
-The sha256 hash of the guest image file used for the test.
-
-If this value is not set in the code or by this parameter no
-validation on the integrity of the image will be performed.
-
-Skipping tests
-~~~~~~~~~~~~~~
-
-The Avocado framework provides Python decorators which allow for easily skip
-tests running under certain conditions. For example, on the lack of a binary
-on the test system or when the running environment is a CI system. For further
-information about those decorators, please refer to::
-
- https://avocado-framework.readthedocs.io/en/latest/guides/writer/chapters/writing.html#skipping-tests
-
-While the conditions for skipping tests are often specifics of each one, there
-are recurring scenarios identified by the QEMU developers and the use of
-environment variables became a kind of standard way to enable/disable tests.
-
-Here is a list of the most used variables:
-
-AVOCADO_ALLOW_LARGE_STORAGE
-^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Tests which are going to fetch or produce assets considered *large* are not
-going to run unless that ``AVOCADO_ALLOW_LARGE_STORAGE=1`` is exported on
-the environment.
-
-The definition of *large* is a bit arbitrary here, but it usually means an
-asset which occupies at least 1GB of size on disk when uncompressed.
-
-SPEED
-^^^^^
-Tests which have a long runtime will not be run unless ``SPEED=slow`` is
-exported on the environment.
-
-The definition of *long* is a bit arbitrary here, and it depends on the
-usefulness of the test too. A unique test is worth spending more time on,
-small variations on existing tests perhaps less so. As a rough guide,
-a test or set of similar tests which take more than 100 seconds to
-complete.
-
-AVOCADO_ALLOW_UNTRUSTED_CODE
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-There are tests which will boot a kernel image or firmware that can be
-considered not safe to run on the developer's workstation, thus they are
-skipped by default. The definition of *not safe* is also arbitrary but
-usually it means a blob which either its source or build process aren't
-public available.
-
-You should export ``AVOCADO_ALLOW_UNTRUSTED_CODE=1`` on the environment in
-order to allow tests which make use of those kind of assets.
-
-AVOCADO_TIMEOUT_EXPECTED
-^^^^^^^^^^^^^^^^^^^^^^^^
-The Avocado framework has a timeout mechanism which interrupts tests to avoid the
-test suite of getting stuck. The timeout value can be set via test parameter or
-property defined in the test class, for further details::
-
- https://avocado-framework.readthedocs.io/en/latest/guides/writer/chapters/writing.html#setting-a-test-timeout
-
-Even though the timeout can be set by the test developer, there are some tests
-that may not have a well-defined limit of time to finish under certain
-conditions. For example, tests that take longer to execute when QEMU is
-compiled with debug flags. Therefore, the ``AVOCADO_TIMEOUT_EXPECTED`` variable
-has been used to determine whether those tests should run or not.
-
-QEMU_TEST_FLAKY_TESTS
-^^^^^^^^^^^^^^^^^^^^^
-Some tests are not working reliably and thus are disabled by default.
-This includes tests that don't run reliably on GitLab's CI which
-usually expose real issues that are rarely seen on developer machines
-due to the constraints of the CI environment. If you encounter a
-similar situation then raise a bug and then mark the test as shown on
-the code snippet below:
-
-.. code::
-
- # See https://gitlab.com/qemu-project/qemu/-/issues/nnnn
- @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab')
- def test(self):
- do_something()
-
-You can also add ``:avocado: tags=flaky`` to the test meta-data so
-only the flaky tests can be run as a group:
-
-.. code::
-
- env QEMU_TEST_FLAKY_TESTS=1 ./pyvenv/bin/avocado \
- run tests/avocado -filter-by-tags=flaky
-
-Tests should not live in this state forever and should either be fixed
-or eventually removed.
-
-
-Uninstalling Avocado
-~~~~~~~~~~~~~~~~~~~~
-
-If you've followed the manual installation instructions above, you can
-easily uninstall Avocado. Start by listing the packages you have
-installed::
-
- pip list --user
-
-And remove any package you want with::
-
- pip uninstall <package_name>
+See :ref:`checkavocado-ref` for more details.
-If you've used ``make check-avocado``, the Python virtual environment where
-Avocado is installed will be cleaned up as part of ``make check-clean``.
.. _checktcg-ref:
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* Re: [PATCH v4 33/35] docs/devel/testing: Split the Avocado documentation into a separate file
2024-08-21 8:27 ` [PATCH v4 33/35] docs/devel/testing: Split the Avocado documentation into a separate file Thomas Huth
@ 2024-08-29 10:18 ` Daniel P. Berrangé
0 siblings, 0 replies; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-08-29 10:18 UTC (permalink / raw)
To: Thomas Huth
Cc: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On Wed, Aug 21, 2024 at 10:27:34AM +0200, Thomas Huth wrote:
> The main testing documentation file got very overloaded already.
> Thus let's split the Avocado information into a separate file.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> docs/devel/testing/avocado.rst | 581 +++++++++++++++++++++++++++++++++
> docs/devel/testing/index.rst | 1 +
> docs/devel/testing/main.rst | 565 +-------------------------------
> 3 files changed, 583 insertions(+), 564 deletions(-)
> create mode 100644 docs/devel/testing/avocado.rst
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 69+ messages in thread
* [PATCH v4 34/35] docs/devel/testing: Rename avocado_qemu.Test class
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (32 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 33/35] docs/devel/testing: Split the Avocado documentation into a separate file Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-29 10:27 ` Daniel P. Berrangé
2024-08-21 8:27 ` [PATCH v4 35/35] docs/devel/testing: Add documentation for functional tests Thomas Huth
34 siblings, 1 reply; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
The avocado_qemu.Test class has been renamed a while back in commit
2283b627bc ("tests/avocado: Rename avocado_qemu.Test -> QemuSystemTest"),
so we should reflect this now in the documentation, too.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
docs/devel/testing/avocado.rst | 32 ++++++++++++++++----------------
1 file changed, 16 insertions(+), 16 deletions(-)
diff --git a/docs/devel/testing/avocado.rst b/docs/devel/testing/avocado.rst
index 0d207a05c5..eda76fe2db 100644
--- a/docs/devel/testing/avocado.rst
+++ b/docs/devel/testing/avocado.rst
@@ -8,11 +8,11 @@ The ``tests/avocado`` directory hosts integration tests. They're usually
higher level tests, and may interact with external resources and with
various guest operating systems.
-These tests are written using the Avocado Testing Framework (which must
-be installed separately) in conjunction with a the ``avocado_qemu.Test``
+These tests are written using the Avocado Testing Framework (which must be
+installed separately) in conjunction with a the ``avocado_qemu.QemuSystemTest``
class, implemented at ``tests/avocado/avocado_qemu``.
-Tests based on ``avocado_qemu.Test`` can easily:
+Tests based on ``avocado_qemu.QemuSystemTest`` can easily:
* Customize the command line arguments given to the convenience
``self.vm`` attribute (a QEMUMachine instance)
@@ -154,7 +154,7 @@ Overview
--------
The ``tests/avocado/avocado_qemu`` directory provides the
-``avocado_qemu`` Python module, containing the ``avocado_qemu.Test``
+``avocado_qemu`` Python module, containing the ``avocado_qemu.QemuSystemTest``
class. Here's a simple usage example:
.. code::
@@ -186,11 +186,11 @@ in the current directory, tagged as "quick", run:
avocado run -t quick .
-The ``avocado_qemu.Test`` base test class
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+The ``avocado_qemu.QemuSystemTest`` base test class
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-The ``avocado_qemu.Test`` class has a number of characteristics that
-are worth being mentioned right away.
+The ``avocado_qemu.QemuSystemTest`` class has a number of characteristics
+that are worth being mentioned right away.
First of all, it attempts to give each test a ready to use QEMUMachine
instance, available at ``self.vm``. Because many tests will tweak the
@@ -233,15 +233,15 @@ and hypothetical example follows:
self.assertEqual(first_res, second_res, third_res)
-At test "tear down", ``avocado_qemu.Test`` handles all the QEMUMachines
-shutdown.
+At test "tear down", ``avocado_qemu.QemuSystemTest`` handles all the
+QEMUMachines shutdown.
The ``avocado_qemu.LinuxTest`` base test class
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The ``avocado_qemu.LinuxTest`` is further specialization of the
-``avocado_qemu.Test`` class, so it contains all the characteristics of
-the later plus some extra features.
+``avocado_qemu.QemuSystemTest`` class, so it contains all the characteristics
+of the later plus some extra features.
First of all, this base class is intended for tests that need to
interact with a fully booted and operational Linux guest. At this
@@ -298,7 +298,7 @@ the following approaches:
working directory, or in the current source tree.
The resulting ``qemu_bin`` value will be preserved in the
-``avocado_qemu.Test`` as an attribute with the same name.
+``avocado_qemu.QemuSystemTest`` as an attribute with the same name.
Attribute reference
-------------------
@@ -308,7 +308,7 @@ Test
Besides the attributes and methods that are part of the base
``avocado.Test`` class, the following attributes are available on any
-``avocado_qemu.Test`` instance.
+``avocado_qemu.QemuSystemTest`` instance.
vm
""
@@ -365,7 +365,7 @@ source tree.
LinuxTest
^^^^^^^^^
-Besides the attributes present on the ``avocado_qemu.Test`` base
+Besides the attributes present on the ``avocado_qemu.QemuSystemTest`` base
class, the ``avocado_qemu.LinuxTest`` adds the following attributes:
distro
@@ -446,7 +446,7 @@ The exact QEMU binary to be used on QEMUMachine.
LinuxTest
^^^^^^^^^
-Besides the parameters present on the ``avocado_qemu.Test`` base
+Besides the parameters present on the ``avocado_qemu.QemuSystemTest`` base
class, the ``avocado_qemu.LinuxTest`` adds the following parameters:
distro
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* Re: [PATCH v4 34/35] docs/devel/testing: Rename avocado_qemu.Test class
2024-08-21 8:27 ` [PATCH v4 34/35] docs/devel/testing: Rename avocado_qemu.Test class Thomas Huth
@ 2024-08-29 10:27 ` Daniel P. Berrangé
0 siblings, 0 replies; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-08-29 10:27 UTC (permalink / raw)
To: Thomas Huth
Cc: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On Wed, Aug 21, 2024 at 10:27:35AM +0200, Thomas Huth wrote:
> The avocado_qemu.Test class has been renamed a while back in commit
> 2283b627bc ("tests/avocado: Rename avocado_qemu.Test -> QemuSystemTest"),
> so we should reflect this now in the documentation, too.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> docs/devel/testing/avocado.rst | 32 ++++++++++++++++----------------
> 1 file changed, 16 insertions(+), 16 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 69+ messages in thread
* [PATCH v4 35/35] docs/devel/testing: Add documentation for functional tests
2024-08-21 8:27 [PATCH v4 00/35] Convert avocado tests to normal Python unittests Thomas Huth
` (33 preceding siblings ...)
2024-08-21 8:27 ` [PATCH v4 34/35] docs/devel/testing: Rename avocado_qemu.Test class Thomas Huth
@ 2024-08-21 8:27 ` Thomas Huth
2024-08-29 10:34 ` Daniel P. Berrangé
2024-08-29 10:35 ` Daniel P. Berrangé
34 siblings, 2 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-21 8:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Daniel P . Berrange
Cc: Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
Document the new functional testing framework. The text is originally
based on the Avocado documentation, but heavily modified to match the
new framework.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
docs/devel/testing/functional.rst | 269 ++++++++++++++++++++++++++++++
docs/devel/testing/index.rst | 1 +
docs/devel/testing/main.rst | 12 ++
3 files changed, 282 insertions(+)
create mode 100644 docs/devel/testing/functional.rst
diff --git a/docs/devel/testing/functional.rst b/docs/devel/testing/functional.rst
new file mode 100644
index 0000000000..c1f4bbf224
--- /dev/null
+++ b/docs/devel/testing/functional.rst
@@ -0,0 +1,269 @@
+.. _checkfunctional-ref:
+
+Functional testing with Python
+==============================
+
+The ``tests/functional`` directory hosts functional tests written in
+Python. They are usually higher level tests, and may interact with
+external resources and with various guest operating systems.
+The functional tests have initially evolved from the Avocado tests, so there
+is a lot of similarity to those tests here (see :ref:`checkavocado-ref` for
+details about the Avocado tests).
+
+The tests should be written in the style of the Python `unittest`_
+framework, using stdio for the TAP protocol. The folder
+``tests/functional/qemu_test`` provides classes (e.g. the ``QemuBaseTest``
+and the ``QemuSystemTest`` classes) and utility functions that help
+to get your test into the right shape.
+
+Tests based on ``qemu_test.QemuSystemTest`` can easily:
+
+ * Customize the command line arguments given to the convenience
+ ``self.vm`` attribute (a QEMUMachine instance)
+
+ * Interact with the QEMU monitor, send QMP commands and check
+ their results
+
+ * Interact with the guest OS, using the convenience console device
+ (which may be useful to assert the effectiveness and correctness of
+ command line arguments or QMP commands)
+
+ * Download (and cache) remote data files, such as firmware and kernel
+ images
+
+Running tests
+-------------
+
+You can run the functional tests simply by executing:
+
+.. code::
+
+ make check-functional
+
+It is also possible to run tests for a certain target only, for example
+the following line will only run the tests for the x86_64 target:
+
+.. code::
+
+ make check-functional-x86_64
+
+To run a single test file without the meson test runner, you can also
+execute the file directly by specifying two environment variables first,
+the PYTHONPATH that has to include the python folder and the tests/functional
+folder of the source tree, and QEMU_TEST_QEMU_BINARY that has to point
+to the QEMU binary that should be used for the test, for example::
+
+ $ export PYTHONPATH=../python:../tests/functional
+ $ export QEMU_TEST_QEMU_BINARY=$PWD/qemu-system-x86_64
+ $ python3 ../tests/functional/test_file.py
+
+Overview
+--------
+
+The ``tests/functional/qemu_test`` directory provides the ``qemu_test``
+Python module, containing the ``qemu_test.QemuSystemTest`` class.
+Here is a simple usage example:
+
+.. code::
+
+ #!/usr/bin/env python3
+
+ from qemu_test import QemuSystemTest
+
+ class Version(QemuSystemTest):
+
+ def test_qmp_human_info_version(self):
+ self.vm.launch()
+ res = self.vm.cmd('human-monitor-command',
+ command_line='info version')
+ self.assertRegex(res, r'^(\d+\.\d+\.\d)')
+
+ if __name__ == '__main__':
+ QemuSystemTest.main()
+
+By providing the "hash bang" line at the beginning of the script,
+and by calling into QemuSystemTest.main() when it is run directly,
+the test framework makes sure to run all test_*() functions in the
+right fassion (e.g. with TAP output that is required by the meson test
+runner).
+
+The ``qemu_test.QemuSystemTest`` base test class
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The ``qemu_test.QemuSystemTest`` class has a number of characteristics
+that are worth being mentioned.
+
+First of all, it attempts to give each test a ready to use QEMUMachine
+instance, available at ``self.vm``. Because many tests will tweak the
+QEMU command line, launching the QEMUMachine (by using ``self.vm.launch()``)
+is left to the test writer.
+
+The base test class has also support for tests with more than one
+QEMUMachine. The way to get machines is through the ``self.get_vm()``
+method which will return a QEMUMachine instance. The ``self.get_vm()``
+method accepts arguments that will be passed to the QEMUMachine creation
+and also an optional ``name`` attribute so you can identify a specific
+machine and get it more than once through the tests methods. A simple
+and hypothetical example follows:
+
+.. code::
+
+ from qemu_test import QemuSystemTest
+
+ class MultipleMachines(QemuSystemTest):
+ def test_multiple_machines(self):
+ first_machine = self.get_vm()
+ second_machine = self.get_vm()
+ self.get_vm(name='third_machine').launch()
+
+ first_machine.launch()
+ second_machine.launch()
+
+ first_res = first_machine.cmd(
+ 'human-monitor-command',
+ command_line='info version')
+
+ second_res = second_machine.cmd(
+ 'human-monitor-command',
+ command_line='info version')
+
+ third_res = self.get_vm(name='third_machine').cmd(
+ 'human-monitor-command',
+ command_line='info version')
+
+ self.assertEqual(first_res, second_res, third_res)
+
+At test "tear down", ``qemu_test.QemuSystemTest`` handles all the QEMUMachines
+shutdown.
+
+QEMUMachine
+-----------
+
+The QEMUMachine API is already widely used in the Python iotests,
+device-crash-test and other Python scripts. It's a wrapper around the
+execution of a QEMU binary, giving its users:
+
+ * the ability to set command line arguments to be given to the QEMU
+ binary
+
+ * a ready to use QMP connection and interface, which can be used to
+ send commands and inspect its results, as well as asynchronous
+ events
+
+ * convenience methods to set commonly used command line arguments in
+ a more succinct and intuitive way
+
+QEMU binary selection
+^^^^^^^^^^^^^^^^^^^^^
+
+The QEMU binary used for the ``self.vm`` QEMUMachine instance will
+primarily depend on the value of the ``qemu_bin`` class attribute.
+If it is not explicitly set by the test code, its default value will
+be the result the QEMU_TEST_QEMU_BINARY environment variable.
+
+Attribute reference
+-------------------
+
+QemuBaseTest
+^^^^^^^^^^^^
+
+The following attributes are available on any ``qemu_test.QemuBaseTest``
+instance.
+
+arch
+""""
+
+The target architecture of the QEMU binary.
+
+Tests are also free to use this attribute value, for their own needs.
+A test may, for instance, use this value when selecting the architecture
+of a kernel or disk image to boot a VM with.
+
+qemu_bin
+""""""""
+
+The preserved value of the ``QEMU_TEST_QEMU_BINARY`` environment
+variable.
+
+QemuSystemTest
+^^^^^^^^^^^^^^
+
+vm
+""
+
+A QEMUMachine instance, initially configured according to the given
+``qemu_bin`` parameter.
+
+cpu
+"""
+
+The cpu model that will be set to all QEMUMachine instances created
+by the test.
+
+machine
+"""""""
+
+The machine type that will be set to all QEMUMachine instances created
+by the test. By using the set_machine() function of the QemuSystemTest
+class to set this attribute, you can automatically check whether the
+machine is available to skip the test in case it is not built into the
+QEMU binary.
+
+Skipping tests
+--------------
+
+Since the test framework is based on the common Python unittest framework,
+you can use the usual Python decorators which allow for easily skipping
+tests running under certain conditions, for example, on the lack of a binary
+on the test system or when the running environment is a CI system. For further
+information about those decorators, please refer to::
+
+ https://docs.python.org/3/library/unittest.html#skipping-tests-and-expected-failures
+
+While the conditions for skipping tests are often specifics of each one, there
+are recurring scenarios identified by the QEMU developers and the use of
+environment variables became a kind of standard way to enable/disable tests.
+
+Here is a list of the most used variables:
+
+QEMU_TEST_ALLOW_LARGE_STORAGE
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Tests which are going to fetch or produce assets considered *large* are not
+going to run unless that ``QEMU_TEST_ALLOW_LARGE_STORAGE=1`` is exported on
+the environment.
+
+The definition of *large* is a bit arbitrary here, but it usually means an
+asset which occupies at least 1GB of size on disk when uncompressed.
+
+QEMU_TEST_ALLOW_UNTRUSTED_CODE
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+There are tests which will boot a kernel image or firmware that can be
+considered not safe to run on the developer's workstation, thus they are
+skipped by default. The definition of *not safe* is also arbitrary but
+usually it means a blob which either its source or build process aren't
+public available.
+
+You should export ``QEMU_TEST_ALLOW_UNTRUSTED_CODE=1`` on the environment in
+order to allow tests which make use of those kind of assets.
+
+QEMU_TEST_FLAKY_TESTS
+^^^^^^^^^^^^^^^^^^^^^
+Some tests are not working reliably and thus are disabled by default.
+This includes tests that don't run reliably on GitLab's CI which
+usually expose real issues that are rarely seen on developer machines
+due to the constraints of the CI environment. If you encounter a
+similar situation then raise a bug and then mark the test as shown on
+the code snippet below:
+
+.. code::
+
+ # See https://gitlab.com/qemu-project/qemu/-/issues/nnnn
+ @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab')
+ def test(self):
+ do_something()
+
+Tests should not live in this state forever and should either be fixed
+or eventually removed.
+
+
+.. _unittest: https://docs.python.org/3/library/unittest.html
diff --git a/docs/devel/testing/index.rst b/docs/devel/testing/index.rst
index cdf7ba1f8b..45eb4a7181 100644
--- a/docs/devel/testing/index.rst
+++ b/docs/devel/testing/index.rst
@@ -9,6 +9,7 @@ testing infrastructure.
main
qtest
+ functional
avocado
acpi-bits
ci
diff --git a/docs/devel/testing/main.rst b/docs/devel/testing/main.rst
index 39b965ecf6..e9921a4b10 100644
--- a/docs/devel/testing/main.rst
+++ b/docs/devel/testing/main.rst
@@ -862,6 +862,18 @@ supported. To start the fuzzer, run
Alternatively, some command different from ``qemu-img info`` can be tested, by
changing the ``-c`` option.
+Functional tests using Python
+-----------------------------
+
+The ``tests/functional`` directory hosts functional tests written in
+Python. You can run the functional tests simply by executing:
+
+.. code::
+
+ make check-functional
+
+See :ref:`checkfunctional-ref` for more details.
+
Integration tests using the Avocado Framework
---------------------------------------------
--
2.46.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* Re: [PATCH v4 35/35] docs/devel/testing: Add documentation for functional tests
2024-08-21 8:27 ` [PATCH v4 35/35] docs/devel/testing: Add documentation for functional tests Thomas Huth
@ 2024-08-29 10:34 ` Daniel P. Berrangé
2024-08-29 11:35 ` Thomas Huth
2024-08-29 10:35 ` Daniel P. Berrangé
1 sibling, 1 reply; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-08-29 10:34 UTC (permalink / raw)
To: Thomas Huth
Cc: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On Wed, Aug 21, 2024 at 10:27:36AM +0200, Thomas Huth wrote:
> Document the new functional testing framework. The text is originally
> based on the Avocado documentation, but heavily modified to match the
> new framework.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> docs/devel/testing/functional.rst | 269 ++++++++++++++++++++++++++++++
> docs/devel/testing/index.rst | 1 +
> docs/devel/testing/main.rst | 12 ++
> 3 files changed, 282 insertions(+)
> create mode 100644 docs/devel/testing/functional.rst
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
> +The tests should be written in the style of the Python `unittest`_
> +framework, using stdio for the TAP protocol. The folder
> +``tests/functional/qemu_test`` provides classes (e.g. the ``QemuBaseTest``
> +and the ``QemuSystemTest`` classes) and utility functions that help
> +to get your test into the right shape.
One gotcha when using TAP protocol is that you can't just spew
debug info to stdout/stderr. Each line of debug info needs to
be prefixed with '#' so it is interpreted as diagnostic output.
We should point this out and recommend people to exclusively
using the 'logging' framework.
Particular care should be taken when spawning sub-processes
to capture both stdout/stderr and then log the result if
needed.
> +Overview
> +--------
> +
> +The ``tests/functional/qemu_test`` directory provides the ``qemu_test``
> +Python module, containing the ``qemu_test.QemuSystemTest`` class.
> +Here is a simple usage example:
> +
> +.. code::
> +
> + #!/usr/bin/env python3
> +
> + from qemu_test import QemuSystemTest
> +
> + class Version(QemuSystemTest):
> +
> + def test_qmp_human_info_version(self):
> + self.vm.launch()
> + res = self.vm.cmd('human-monitor-command',
> + command_line='info version')
> + self.assertRegex(res, r'^(\d+\.\d+\.\d)')
> +
> + if __name__ == '__main__':
> + QemuSystemTest.main()
> +
> +By providing the "hash bang" line at the beginning of the script,
> +and by calling into QemuSystemTest.main() when it is run directly,
> +the test framework makes sure to run all test_*() functions in the
> +right fassion (e.g. with TAP output that is required by the meson test
> +runner).
Perhaps say that the test file should have execute permissions,
given the hash bang ?
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH v4 35/35] docs/devel/testing: Add documentation for functional tests
2024-08-29 10:34 ` Daniel P. Berrangé
@ 2024-08-29 11:35 ` Thomas Huth
2024-08-29 11:43 ` Daniel P. Berrangé
0 siblings, 1 reply; 69+ messages in thread
From: Thomas Huth @ 2024-08-29 11:35 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On 29/08/2024 12.34, Daniel P. Berrangé wrote:
> On Wed, Aug 21, 2024 at 10:27:36AM +0200, Thomas Huth wrote:
>> Document the new functional testing framework. The text is originally
>> based on the Avocado documentation, but heavily modified to match the
>> new framework.
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>> docs/devel/testing/functional.rst | 269 ++++++++++++++++++++++++++++++
>> docs/devel/testing/index.rst | 1 +
>> docs/devel/testing/main.rst | 12 ++
>> 3 files changed, 282 insertions(+)
>> create mode 100644 docs/devel/testing/functional.rst
>
> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
>
>> +The tests should be written in the style of the Python `unittest`_
>> +framework, using stdio for the TAP protocol. The folder
>> +``tests/functional/qemu_test`` provides classes (e.g. the ``QemuBaseTest``
>> +and the ``QemuSystemTest`` classes) and utility functions that help
>> +to get your test into the right shape.
>
> One gotcha when using TAP protocol is that you can't just spew
> debug info to stdout/stderr. Each line of debug info needs to
> be prefixed with '#' so it is interpreted as diagnostic output.
Actually, that's the great thing about pycotap (in comparison to other
Python TAP implementations that I've seen), it helps you to get this right:
By instantiating the TAPTestRunner like this:
tr = pycotap.TAPTestRunner(message_log = pycotap.LogMode.LogToError,
test_output_log = pycotap.LogMode.LogToError)
The stdio output gets redirected to stderr. And the meson test runner is
fine by collecting the error messages from stderr and showing them to the
user in the right way in case the test failed (and puts them into the log
file if the test succeeded).
But yes, the doc text is a little bit sparse on this topic, I'll try to
elaborate in the next iteration.
Thomas
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH v4 35/35] docs/devel/testing: Add documentation for functional tests
2024-08-29 11:35 ` Thomas Huth
@ 2024-08-29 11:43 ` Daniel P. Berrangé
0 siblings, 0 replies; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-08-29 11:43 UTC (permalink / raw)
To: Thomas Huth
Cc: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On Thu, Aug 29, 2024 at 01:35:21PM +0200, Thomas Huth wrote:
> On 29/08/2024 12.34, Daniel P. Berrangé wrote:
> > On Wed, Aug 21, 2024 at 10:27:36AM +0200, Thomas Huth wrote:
> > > Document the new functional testing framework. The text is originally
> > > based on the Avocado documentation, but heavily modified to match the
> > > new framework.
> > >
> > > Signed-off-by: Thomas Huth <thuth@redhat.com>
> > > ---
> > > docs/devel/testing/functional.rst | 269 ++++++++++++++++++++++++++++++
> > > docs/devel/testing/index.rst | 1 +
> > > docs/devel/testing/main.rst | 12 ++
> > > 3 files changed, 282 insertions(+)
> > > create mode 100644 docs/devel/testing/functional.rst
> >
> > Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
> >
> > > +The tests should be written in the style of the Python `unittest`_
> > > +framework, using stdio for the TAP protocol. The folder
> > > +``tests/functional/qemu_test`` provides classes (e.g. the ``QemuBaseTest``
> > > +and the ``QemuSystemTest`` classes) and utility functions that help
> > > +to get your test into the right shape.
> >
> > One gotcha when using TAP protocol is that you can't just spew
> > debug info to stdout/stderr. Each line of debug info needs to
> > be prefixed with '#' so it is interpreted as diagnostic output.
>
> Actually, that's the great thing about pycotap (in comparison to other
> Python TAP implementations that I've seen), it helps you to get this right:
> By instantiating the TAPTestRunner like this:
>
> tr = pycotap.TAPTestRunner(message_log = pycotap.LogMode.LogToError,
> test_output_log = pycotap.LogMode.LogToError)
>
> The stdio output gets redirected to stderr. And the meson test runner is
> fine by collecting the error messages from stderr and showing them to the
> user in the right way in case the test failed (and puts them into the log
> file if the test succeeded).
I'm not sure that works in all scenarios. In the patch that converts the
acpi-bits test, I had to add this chunk:
@@ -264,8 +263,12 @@ def generate_bits_iso(self):
try:
if os.getenv('V') or os.getenv('BITS_DEBUG'):
- subprocess.check_call([mkrescue_script, '-o', iso_file,
- bits_dir], stderr=subprocess.STDOUT)
+ proc = subprocess.run([mkrescue_script, '-o', iso_file,
+ bits_dir],
+ stdout=subprocess.PIPE,
+ stderr=subprocess.STDOUT,
+ check=True)
+ self.logger.info("grub-mkrescue output %s" % proc.stdout)
else:
subprocess.check_call([mkrescue_script, '-o',
iso_file, bits_dir],
because I saw errors in TAP output parsing when V=1 was set,
due to mkrescue_script printing to the STDOUT file descriptor.
IIUC, I wonder if the pycotap runner is replacing the 'stdout'
python object, but leaving the underlying OS FD 1 open on its
original channel such that it gets inherited by child processes.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v4 35/35] docs/devel/testing: Add documentation for functional tests
2024-08-21 8:27 ` [PATCH v4 35/35] docs/devel/testing: Add documentation for functional tests Thomas Huth
2024-08-29 10:34 ` Daniel P. Berrangé
@ 2024-08-29 10:35 ` Daniel P. Berrangé
1 sibling, 0 replies; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-08-29 10:35 UTC (permalink / raw)
To: Thomas Huth
Cc: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé,
Ani Sinha, Richard Henderson, John Snow, qemu-ppc, Fabiano Rosas
On Wed, Aug 21, 2024 at 10:27:36AM +0200, Thomas Huth wrote:
> Document the new functional testing framework. The text is originally
> based on the Avocado documentation, but heavily modified to match the
> new framework.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> docs/devel/testing/functional.rst | 269 ++++++++++++++++++++++++++++++
> docs/devel/testing/index.rst | 1 +
> docs/devel/testing/main.rst | 12 ++
> 3 files changed, 282 insertions(+)
> create mode 100644 docs/devel/testing/functional.rst
Should also mention the use of the "Asset" class and ASSET_blah class
level variables for caching purposes.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 69+ messages in thread